Index design question

["N. C. Deepak Ramesh" <nc...@gmail.com>]

Hi list,

Let me describe my issue taking a simpler model. Lets say I were to build a 
blog which allows each post to have multiple keywords. I want to provide a 
search over the posts but restricted to a subset of the keywords (say - 
python, windows, etc.). How can I structure the index in this case. I had 
though of 2 fields, one a list of keyword ids and the other for post 
contents. (The reason why I go for keyword ids is because the keyword is a 
foreign key whose string could be changed independent of the post). What do 
you think?

Also, from a design orientation for the use case I described above, would it 
be better to go for something like tsearch2 (I use postgres) in this case 
because keyword searching is just one way of searching in my app. The data 
could be searched across many other fields which are being done by sql's. 
Does it really help in using something like lucene because I am worried 
about the burden of maintaing the 2 data repositories (db and lucene index) 
in sync. I am asking this because if I go for tsearch2 the data is in only 1 
place and also updates, deletes to the data are handled for free by the db 
for me.

Does anybody have any suggestion?

Regards,
 
Deepak

Re: Index design question

[Otis Gospodnetic <ot...@yahoo.com>]

Hi,

> If you store only IDs in Lucene, you won't be able to search using
> > keywords (text).
> 
> 
> Let me explain better. Suppose I have an index field called
> categories which 
> contains a list of ids for each post. For example - 1 2 45 198. Now I
> can 
> search on the contents field but restricted to all posts under a
> particular 
> category by using -
> 
> "content:java and category:45 or 198". 
> 
> Note - Not exact syntax but I hope you get the picture. Anything
> wrong with this approach?

Nothing wrong with this approach.  You'll likely want/need to write
your own Analyzer to make sure your numeric IDs are not discarded (some
Analyzers discard them).  Perhaps you'll want to use
PerFieldAnalyzerWrapper.

> That way you get PG+Lucene in a single package.
> > 
> 
> That would be a killer combo. 
> 
> But what I would like to know is if anybody on this list has used
> tsearch2 
> but later moved on to lucene because of reasons of speed, search 
> flexibility, etc.

Lots of people did that for reasons of speed, flexibility, as well as
disk usage.  If your application is a small one, you may be able to
stick with just PG.

Otis

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Simpy -- http://www.simpy.com/ -- Find it. Tag it. Share it.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Index design question

["N. C. Deepak Ramesh" <nc...@gmail.com>]

Hi,

If you store only IDs in Lucene, you won't be able to search using
> keywords (text).


Let me explain better. Suppose I have an index field called categories which 
contains a list of ids for each post. For example - 1 2 45 198. Now I can 
search on the contents field but restricted to all posts under a particular 
category by using -

"content:java and category:45 or 198". 

Note - Not exact syntax but I hope you get the picture. Anything wrong with 
this approach?

That way you get PG+Lucene in a single package.
> 

That would be a killer combo. 

But what I would like to know is if anybody on this list has used tsearch2 
but later moved on to lucene because of reasons of speed, search 
flexibility, etc.

Thanks,

Deepak

Re: Index design question

[Otis Gospodnetic <ot...@yahoo.com>]

Hi,

> Let me describe my issue taking a simpler model. Lets say I were to
> build a 
> blog which allows each post to have multiple keywords. I want to
> provide a 
> search over the posts but restricted to a subset of the keywords (say
> - 
> python, windows, etc.). How can I structure the index in this case. I
> had 
> though of 2 fields, one a list of keyword ids and the other for post 
> contents. (The reason why I go for keyword ids is because the keyword
> is a 
> foreign key whose string could be changed independent of the post).
> What do you think?

If you store only IDs in Lucene, you won't be able to search using
keywords (text).

> Also, from a design orientation for the use case I described above,
> would it 
> be better to go for something like tsearch2 (I use postgres) in this
> case 
> because keyword searching is just one way of searching in my app. The
> data 
> could be searched across many other fields which are being done by
> sql's. 
> Does it really help in using something like lucene because I am
> worried 
> about the burden of maintaing the 2 data repositories (db and lucene
> index) 
> in sync. I am asking this because if I go for tsearch2 the data is in
> only 1 
> place and also updates, deletes to the data are handled for free by
> the db 
> for me.
> 
> Does anybody have any suggestion?

It's up to you to pick the route.  Going to PG-only route probably
means less powerful searching.  Going PG+Lucene route means you have to
maintain PG and Lucene data in sync.  In my experience, doing the
latter is not very hard.  Another option to consider is talking to PG
people and exposing them to Lucene4c, which they may want to embed in
PostgreSQL.  That way you get PG+Lucene in a single package.

Otis

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Simpy -- http://www.simpy.com/ -- Find it. Tag it. Share it.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org