You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by karl wettin <ka...@gmail.com> on 2007/06/27 02:16:50 UTC

Lucene as primary object storage

Until recently I have only used Lucene as an index. I'm very  
interested to hear what you people have to say about using Lucene as  
the primary and only persistance layer in an application.

I would feel much safer if I had a transactionlog for crash recovery.  
So I might get me one of those. Are there more things I should worry  
about?


-- 
karl

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: Lucene as primary object storage

Posted by Mohammad Norouzi <mn...@gmail.com>.
Hi karl,
we did something like hibernate to map an object (Entity) with lucene by
defining a bunch of annotations just like the Limax project (as far as I
know it is led by you),
the only problem we had was how to make relationship between two or more
separate indexes. I managed to resolve it but I don't think it's very good
idea. if only the Lucene had some feature to facilitate this :)
We use these indexes for generating some dynamic reports and we are going to
create a database crawler to surf the DB and find new or deleted records to
update its index files.
our application uses only index files to persist the information comming
from DB and also uses that index as a resource

I am welcome if you want to know how to make relationship between two or
more indexes.

Good Luck

-- 
Regards,
Mohammad
--------------------------
see my blog: http://brainable.blogspot.com/
another in Persian: http://fekre-motefavet.blogspot.com/

Re: Lucene as primary object storage

Posted by Emmanuel Bernard <em...@hibernate.org>.
While not everything in the following post apply to Lucene as a  
storage solution, it has some interesting points
http://blog.hibernate.org/cgi-bin/blosxom.cgi/2007/05/23#in-defence

I need to trust my data storage for its good support for:
  - transaction
  - reliability
  - scalability
  - readability (being able to read my data from any random language  
on earth 10 years from now)

I trust my RDBMS because it's been 30 years that they deliver on all  
of those criteria.

There are other qualities that explain why I prefer Relational DB to  
keep my data (including relational queries, ability to do correlated  
queries between 2 or more "associated" entities/documents), but that  
should be good enough to give you an idea of that I think.


On  Jun 29, 2007, at 08:48, karl wettin wrote:

> 28 jun 2007 kl. 15.37 skrev Emmanuel Bernard:
>> I don't really like the idea actually: I'm much comfortable with  
>> having my data in a relational DB :)
>
> If you don't mind, please develop that a bit further.
>
>
> I think Lucene is suited pretty well for object storage if you also  
> need it as an index. It is especially great for them heavy  
> polymorphic or inheritive models. One simple term query can span  
> multiple classes. Many ORMs require SQL joins or what unless you  
> choose to store the data in some semi-retarded way.
>
> There are of course some setbacks. I can not update a single field  
> of an instance, I have to update the whole thing. So one have to be  
> careful with bi-directional associations beween classes and such.
>
>
> My hypotheisis in general goes something like "the less, the  
> simpler, the faster, the better it is". That is probably why I sort  
> of got sidetracked by the codemonkey on my shoulder while fooling  
> around with @annotation for the first time and ended up with an  
> object storage somewhere between the Sleepycat transactional API  
> and the Hibernate lazy patterns, but using Lucene as persistency. I  
> still don't know how well it works with lots of data.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: Lucene as primary object storage

Posted by karl wettin <ka...@gmail.com>.
28 jun 2007 kl. 15.37 skrev Emmanuel Bernard:
> I don't really like the idea actually: I'm much comfortable with  
> having my data in a relational DB :)

If you don't mind, please develop that a bit further.


I think Lucene is suited pretty well for object storage if you also  
need it as an index. It is especially great for them heavy  
polymorphic or inheritive models. One simple term query can span  
multiple classes. Many ORMs require SQL joins or what unless you  
choose to store the data in some semi-retarded way.

There are of course some setbacks. I can not update a single field of  
an instance, I have to update the whole thing. So one have to be  
careful with bi-directional associations beween classes and such.


My hypotheisis in general goes something like "the less, the simpler,  
the faster, the better it is". That is probably why I sort of got  
sidetracked by the codemonkey on my shoulder while fooling around  
with @annotation for the first time and ended up with an object  
storage somewhere between the Sleepycat transactional API and the  
Hibernate lazy patterns, but using Lucene as persistency. I still  
don't know how well it works with lots of data.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: Lucene as primary object storage

Posted by Emmanuel Bernard <em...@hibernate.org>.
Hibernate Search (formerly known as Hibernate Lucene) is not designed  
to use Lucene as the primary and only backend. It is designed to  
complement a database.
I don't really like the idea actually: I'm much comfortable with  
having my data in a relational DB :)

So this product will not help for you request.

Emmanuel

On  Jun 27, 2007, at 03:48, karl wettin wrote:

> 27 jun 2007 kl. 03.19 skrev Liu_Andy2@emc.com:
>
>> Perhaps this reference
>> http://www.mail-archive.com/hibernate-devel@lists.sourceforge.net/ 
>> msg062
>> 60.html will give you some help. It's about Hibernate Lucene
>> integration.
>
> Thanks, interesting. Do you know more about this? Was the ORM  
> implemented? Does Hibernate have some sort of built in transaction  
> log it could store on the side?
>
> They don't seem to talk that much about what might go wrong with a  
> directoy though. That is my big concern. I guess my subject was a  
> bit missleading. Up until now I have been able to rebuild the index  
> from a database or something if it dies on me due to bad code,  
> hardware failure or what ever reason. I can't do that if the index  
> was the only place I saved the data.
>
> Perhaps it is a backup-solution I'm looking for. Maybe there are  
> some expensive or sensitive things I need to watch out a bit extra  
> when calling as, or something. I don't know. That is why I'm asking.
>
>>
>> Andy
>>
>> -----Original Message-----
>> From: karl wettin [mailto:karl.wettin@gmail.com]
>> Sent: Wednesday, June 27, 2007 8:17 AM
>> To: java-user@lucene.apache.org
>> Subject: Lucene as primary object storage
>>
>> Until recently I have only used Lucene as an index. I'm very
>> interested to hear what you people have to say about using Lucene as
>> the primary and only persistance layer in an application.
>>
>> I would feel much safer if I had a transactionlog for crash recovery.
>> So I might get me one of those. Are there more things I should worry
>> about?
>>
>>
>> -- 
>> karl
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: Lucene as primary object storage

Posted by karl wettin <ka...@gmail.com>.
27 jun 2007 kl. 03.19 skrev Liu_Andy2@emc.com:

> Perhaps this reference
> http://www.mail-archive.com/hibernate-devel@lists.sourceforge.net/ 
> msg062
> 60.html will give you some help. It's about Hibernate Lucene
> integration.

Thanks, interesting. Do you know more about this? Was the ORM  
implemented? Does Hibernate have some sort of built in transaction  
log it could store on the side?

They don't seem to talk that much about what might go wrong with a  
directoy though. That is my big concern. I guess my subject was a bit  
missleading. Up until now I have been able to rebuild the index from  
a database or something if it dies on me due to bad code, hardware  
failure or what ever reason. I can't do that if the index was the  
only place I saved the data.

Perhaps it is a backup-solution I'm looking for. Maybe there are some  
expensive or sensitive things I need to watch out a bit extra when  
calling as, or something. I don't know. That is why I'm asking.

>
> Andy
>
> -----Original Message-----
> From: karl wettin [mailto:karl.wettin@gmail.com]
> Sent: Wednesday, June 27, 2007 8:17 AM
> To: java-user@lucene.apache.org
> Subject: Lucene as primary object storage
>
> Until recently I have only used Lucene as an index. I'm very
> interested to hear what you people have to say about using Lucene as
> the primary and only persistance layer in an application.
>
> I would feel much safer if I had a transactionlog for crash recovery.
> So I might get me one of those. Are there more things I should worry
> about?
>
>
> -- 
> karl
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


RE: Lucene as primary object storage

Posted by Li...@emc.com.
Perhaps this reference
http://www.mail-archive.com/hibernate-devel@lists.sourceforge.net/msg062
60.html will give you some help. It's about Hibernate Lucene
integration.

Andy

-----Original Message-----
From: karl wettin [mailto:karl.wettin@gmail.com] 
Sent: Wednesday, June 27, 2007 8:17 AM
To: java-user@lucene.apache.org
Subject: Lucene as primary object storage

Until recently I have only used Lucene as an index. I'm very  
interested to hear what you people have to say about using Lucene as  
the primary and only persistance layer in an application.

I would feel much safer if I had a transactionlog for crash recovery.  
So I might get me one of those. Are there more things I should worry  
about?


-- 
karl

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org