You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Shaghayegh Sahebie <sh...@yahoo.com> on 2006/06/20 14:21:41 UTC
Search within multiple different subfolders
Hi all,
I'm a new user of Lucene and wanted to know if Lucne can search multiple data directories which these directories do not have similar parent?
e.g. I have a directory named "father" and i have a subDirectory called "son" and i have another directory called "mother" with a subDirectory called "daughter" and i want to search within both "son" and "daughter" but not in other subfolders and files of "mother" and "father".
Is it possible to limit the search to these two different path directories? should i use any kind of filter for it?
Thanks in advance
--Shaghayegh
---------------------------------
Talk is cheap. Use Yahoo! Messenger to make PC-to-Phone calls. Great rates starting at 1¢/min.
Re: Search within multiple different subfolders
Posted by Erick Erickson <er...@gmail.com>.
>
>
>
> Perhaps for privacy reasons? that only specific users should be able to
> search the whole index.
> Is there a best practice approach to realize this?
>
>
Good point. But I still think you could get the same effect with less
complexity by including a "source" tag (to extend the example) and munging
the query to AND in the source tags the user was authorized to see. Of
course I'm not writing the application and everything seems simpler from the
outside, so.....
The whole permissions thing is "interesting", as in "may you live in
interesting times" <G>... I'm sure the issue has been discussed in the
archive, but since it's not something I have to worry about I didn't track
it. You might try searching on "permissions" or similar in the archive...
Re: Search within multiple different subfolders
Posted by Martin Braun <mb...@uni-hd.de>.
hi,
>
> I'm hardly the lucene expert, but I don't think you can search just a
> portion of the index. But that's effectively what you're doing if you
> restrict the search to "son and.....".
I think there is also the possibility to write a custom search filter
(org.apache.lucene.search.Filter), and to build a bitset with a special
field providing the information from which dir the file comes from.
But I am wondering which is the better solution according to performance?
> But I also have to ask, why are you trying to "search only a portion of the
> index"? If you haven't encountered a bottleneck that's forcing you into
Perhaps for privacy reasons? that only specific users should be able to
search the whole index.
Is there a best practice approach to realize this?
thx,
martin
--
Universitaetsbibliothek Heidelberg Tel: +49 6221 54-2580
Ploeck 107-109, D-69117 Heidelberg Fax: +49 6221 54-2623
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Search within multiple different subfolders
Posted by Erick Erickson <er...@gmail.com>.
Shagheyegh:
I'm hardly the lucene expert, but I don't think you can search just a
portion of the index. But that's effectively what you're doing if you
restrict the search to "son and.....".
However, depending on your problem space, you could build separate indexes.
To continue the example, you could build 4 indexes, "mother", "father",
"son", "daughter" and only search the relevant ones.
You should still be able to aggregate the results. That is, you could search
over all 4 indexes when you needed to and combine the results into a single
response. Then you could only search a subset of them at other times. I
admit that I haven't yet had to use a MultiSearcher, but that sure looks
like what you want if you adopt this approach <G>...
Note: I'm looking at the 2.0 documentation....
But I also have to ask, why are you trying to "search only a portion of the
index"? If you haven't encountered a bottleneck that's forcing you into this
option (or don't have a *very* high expectation that you will encounter such
a bottleneck), this strikes me as work you shouldn't be doing until there's
a demonstrated need. The eXtreme Programming folks look at it this way "make
it work, make it right, make it fast".
>From Tony Hoare and Donald Knuth... "We should forget about small
efficiencies, say about 97% of the time: premature optimization is the root
of all evil."
Best
Erick
Re: Search within multiple different subfolders
Posted by Shaghayegh Sahebie <sh...@yahoo.com>.
Thank you Erick,
You are right! Lucene searches the Index not directories, but in this way we search the whole index and just add a new constraint(e.g. files path) to match proper results. Is there a way to not to search the whole index and just some branches of an index without using constraints?
regards
--Shaghayegh
---------------------------------
Ring'em or ping'em. Make PC-to-phone calls as low as 1¢/min with Yahoo! Messenger with Voice.
Re: Search within multiple different subfolders
Posted by Erick Erickson <er...@gmail.com>.
I'm a little confused. What does a directory have to do with searching? You
can certainly build the *index* from files in a directory, but Lucene
doesn't search files themselves, it searches the *index* built from the
files.
If I'm interpreting your problem correctly, you'll have to build an index,
perhaps putting some designator in each document about where it came from.
Crudely, you could just store the file path that the file was indexed from,
or even just some designation. For instance, a field "origin", with values
"mother", "father", "son", "daughter". Then your search could restrict
itself to documents with the "origin" field however you choose.
Hope that helps
Erick