WebDAV Search

[Dave Bryson <da...@miceda-data.com>]

I'm looking at the possibility of adding search capabilites to
WebDAV. I've looked a the DASL spec.

I think a good search engine ( lucene ) customized to index on the
WebDAV tags would be a good solution.

Does anyone have any thoughts on this approach?  It seems to me this 
would be more efficient in the long run compared to a RDB. 

Thanks,
-- 
Dave Bryson
daveb@miceda-data.com
----------------------

Re: WebDAV Search

[Remy Maucherat <re...@apache.org>]

> On Mon, 13 Nov 2000, you wrote:
> > I would like to use an abstraction layer for the indexing/searching
> > component (similar to the ones for the other Slide components).
> > The WebDAV part of DASL is the handling of the SEARCH method, which can
be
> > done independently of the rest of the DAV servlet (there's a header to
add
> > to OPTIONS). I think it would be cool to support pluggable query
languages.
>
> My first thought is to create a descriptor store that creates an index of
> the properties via a lucene ( www.lucene.com ) index.

I think there should be a new type of store (index store), as well as a
helper (IndexHelper) to provide some higher level function calls.

The IndexStore would have those functions :
- add(NodeRevisionDescriptor, NodeRevisionContent)
- update(NodeRevisionDescriptor, NodeRevisionContent)
- remove(uri, revision number ?)
- search(query), which returns some kind of iterator containing result
entries (probably containg uri + revision) - That's the hard part ;)
We have the problem of having a single query language, and having to
translate it. We could run into some limitations in that area. The store
could advertise its capabilities, and the helper could try to deal with them
...

What do you think ?

> The search interface would be a seperate Servlet that can translate the
> DASL request to a lucene query.  Lucene appears to have all the features
> for a basic search request as required per the DASL spec (
> contains, like, boolean, etc...).

No, I would definitely add a org.apache.slide.webdav.SearchMethod class. It
would be just fine. Why add a new servlet ?


I may have missed something, though. Perhaps you just want to add a DASL
interface on top of Lucene, and add some data from Slide into Lucene's
index. If that's the case, I can help you with the Slide part, then at the
end of the project, I could use a nice DASL servlet being contributed to
Slide (merging the code with the current WebDAV servlet wouldn't require a
huge effort) :)

> However, I'm still trying to get my head around the proper way to pull-out
> the property descriptors from the rest of the request in Slide. :-)

I think it's best to wait for them to be sumbitted by the other helpers when
updates are needed. A background thread could be used to do some updates
(some repositories could be modified without going through Slide).

> I also think this abstraction layer could lend itself to adding
> full-text search capabilties to Slide

Definitely. If the underlying searching engine can handle it, of course ;)

> and the ability to compress some
> content. I think compression of content will become more important as
> bandwith increases.

The contract of the ContentStore is that you can retrieve the content you
stored into it. It does not specify if this is encrypted, compressed ...

> Axiom 1 (at least applies to Office 2000 users): Content size is directly
> proportional to available bandwith.

:)

Remy

Re: WebDAV Search

[Dave Bryson <da...@miceda-data.com>]

On Mon, 13 Nov 2000, you wrote:
> I would like to use an abstraction layer for the indexing/searching
> component (similar to the ones for the other Slide components).
> The WebDAV part of DASL is the handling of the SEARCH method, which can be
> done independently of the rest of the DAV servlet (there's a header to add
> to OPTIONS). I think it would be cool to support pluggable query languages.

My first thought is to create a descriptor store that creates an index of
the properties via a lucene ( www.lucene.com ) index.

The search interface would be a seperate Servlet that can translate the
DASL request to a lucene query.  Lucene appears to have all the features
for a basic search request as required per the DASL spec (
contains, like, boolean, etc...).

However, I'm still trying to get my head around the proper way to pull-out
the property descriptors from the rest of the request in Slide. :-)

I also think this abstraction layer could lend itself to adding
full-text search capabilties to Slide and the ability to compress some
content. I think compression of content will become more important as
bandwith increases. 

Axiom 1 (at least applies to Office 2000 users): Content size is directly
proportional to available bandwith.  

-- 
Dave Bryson
daveb@miceda-data.com
----------------------

Re: WebDAV Search

[Remy Maucherat <re...@apache.org>]

> I'm looking at the possibility of adding search capabilites to
> WebDAV. I've looked a the DASL spec.

I'm interested :))

> I think a good search engine ( lucene ) customized to index on the
> WebDAV tags would be a good solution.

I would like to use an abstraction layer for the indexing/searching
component (similar to the ones for the other Slide components).
The WebDAV part of DASL is the handling of the SEARCH method, which can be
done independently of the rest of the DAV servlet (there's a header to add
to OPTIONS). I think it would be cool to support pluggable query languages.

Right now, my schedule for WebDAV is :
- DAV ACL
- Delta V
- DASL
So if anyone can help by implementing DASL before, that would be great.

> Does anyone have any thoughts on this approach?  It seems to me this
> would be more efficient in the long run compared to a RDB.

Yes, probably.

Remy