You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@forrest.apache.org by thorsten <th...@apache.org> on 2004/09/29 22:30:07 UTC
[RT] Search engines
Hello devs,
I would like to implement a "better" search engine script.
The script should be:
a) page specific
b) extensible
a) Imagine a site that have different topics.
1) sport
2) news
3) open source
Each of this topics would have different search provider.
e.g. open source
Imagine our mailing list site. We can search our site with /marc/ or
/eyebrowse/. The searchbox then would offer this engine to search. That
will be more efficient for the user because he do not have to go to
those site but rather search from the forrest site and got redirect to
the search result.
b) The user should be able to extend the standard search engines by
adding the neccessary information into a xml file. We would just add
some examples that we actually need for our site.
*How it should work*
We need the following components:
I) site.xml - Here we can add the hooks for each site which search
engine should be used. e.g.
<todo label="Todo" href="todo.html" description="Todo List"
searchType="os" searchEngine="all"/>
This would offer the *all* search engines from the categorie *os*.
<todo label="Todo" href="todo.html" description="Todo List"
searchType="os" searchEngine="marc, eyebrowse"/>
This would offer the marc and the eyebrowse search engines from the
categorie *os*.
II) searchEngine.xml - This is the place where all possible search
engines are stored. e.g.
<engine id="marc" href="http://marc.theaimsgroup.com/"
queryParam="query" searchType="os">
<param type="hidden" id="l" value="forrest-dev"/>
</engine>
<engine id="eyebrowse"
href="http://issues.apache.org/eyebrowse/SummarizeList" queryParam="q"
searchType="os">
<param type="option" id="listId" value="260"/>
<param type="option" id="listId" value="259"/>
</engine>
III) searchEngine.xsl - This is the place where the searchEngine.xml got
transformed.
All this will be then injected in the site2xhtml.xsl and to the
resulting page.
I hope you got the idea and WDYT?
king regards
thorsten
Re: [RT] Search engines
Posted by David Crossley <cr...@apache.org>.
Ross Gardler wrote:
>
> So, what does the user see? Is it a text box for search terms and a drop
> down box with options for each of the available engines?
This will need some careful design. It should be
explicit that the user knows the scope of the
document collection that they are searching.
--
David Crossley
Re: [RT] Search engines
Posted by thorsten <th...@apache.org>.
Ross Gardler wrote:
> thorsten wrote:
>
>> I would like to implement a "better" search engine script.
>>
>> The script should be:
>> a) page specific
>> b) extensible
>>
>
> <snip/>
>
>> Each of this topics would have different search provider.
>> e.g. open source
>
>
> <snip/>
>
>> b) The user should be able to extend the standard search engines by
>> adding the neccessary information into a xml file. We would just add
>> some examples that we actually need for our site.
>>
>> *How it should work*
>> We need the following components:
>> I) site.xml - Here we can add the hooks for each site which search
>> engine should be used. e.g.
>> <todo label="Todo" href="todo.html" description="Todo List"
>> searchType="os" searchEngine="all"/>
>> This would offer the *all* search engines from the categorie *os*.
>> <todo label="Todo" href="todo.html" description="Todo List"
>> searchType="os" searchEngine="marc, eyebrowse"/>
>> This would offer the marc and the eyebrowse search engines from the
>> categorie *os*.
>
>
> So, what does the user see? Is it a text box for search terms and a drop
> down box with options for each of the available engines?
>
Yeah. The example of eclipse site is really the one I had in mind when
writting this RT.
>> I hope you got the idea and WDYT?
>
>
> I like the sound of this. One of the things I really like about the
> eclipse site is that you can search on a variety of "levels". That is
> you can search the whole site, just the docs, just the API's, just
> mailing lists, specific mailing list etc. (see
> http://www.eclipse.org/search/search.cgi). It seems you are suggesting
> the same kind of idea but with the added benefit of being able to
> customise the options in each section of the site, so in the developer
> section we default to searching the API (for example).
>
Jupp, you understand me 100%. Nice. :)
> I can see this going further in that a site could use Mozilla search
> plugins (http://mycroft.mozdev.org/download.html). Have you considered
> using the Mozilla format? (docs are at
> http://mycroft.mozdev.org/deepdocs.html)
>
I just had a look on some basic examples on that site and yes I think I
will use the format. Looks good and why reinventing the wheel all the
time. ;-)
> Ross
>
Thanks Ross for that feedback again.
Kind regards
thorsten
Re: [RT] Search engines
Posted by Ross Gardler <rg...@apache.org>.
thorsten wrote:
> I would like to implement a "better" search engine script.
>
> The script should be:
> a) page specific
> b) extensible
>
<snip/>
> Each of this topics would have different search provider.
> e.g. open source
<snip/>
> b) The user should be able to extend the standard search engines by
> adding the neccessary information into a xml file. We would just add
> some examples that we actually need for our site.
>
> *How it should work*
> We need the following components:
> I) site.xml - Here we can add the hooks for each site which search
> engine should be used. e.g.
> <todo label="Todo" href="todo.html" description="Todo List"
> searchType="os" searchEngine="all"/>
> This would offer the *all* search engines from the categorie *os*.
> <todo label="Todo" href="todo.html" description="Todo List"
> searchType="os" searchEngine="marc, eyebrowse"/>
> This would offer the marc and the eyebrowse search engines from the
> categorie *os*.
So, what does the user see? Is it a text box for search terms and a drop
down box with options for each of the available engines?
> I hope you got the idea and WDYT?
I like the sound of this. One of the things I really like about the
eclipse site is that you can search on a variety of "levels". That is
you can search the whole site, just the docs, just the API's, just
mailing lists, specific mailing list etc. (see
http://www.eclipse.org/search/search.cgi). It seems you are suggesting
the same kind of idea but with the added benefit of being able to
customise the options in each section of the site, so in the developer
section we default to searching the API (for example).
I can see this going further in that a site could use Mozilla search
plugins (http://mycroft.mozdev.org/download.html). Have you considered
using the Mozilla format? (docs are at
http://mycroft.mozdev.org/deepdocs.html)
Ross
Re: [RT] Search engines
Posted by thorsten <th...@apache.org>.
David Crossley wrote:
> <snip/>
>
>
> Did you say *or* integrate with Lenya!
> I though that that was always on the schedule.
>
:). Yes, mate. It is on my (and it seems on Ross) personal list but not
yet scheduled. ;-)
I may have a lenya project in the next month and will get into the deep
details of lenya. I reckon afterwards it should be not as hard. Stay
tuned on that. ;-)
King regards
thorsten
Re: [RT] Search engines
Posted by David Crossley <cr...@apache.org>.
thorsten wrote:
> David Crossley wrote:
> > thorsten wrote:
> >>*How it should work*
> >>We need the following components:
> >>I) site.xml - Here we can add the hooks for each site which search
<snip/>
> >
> > Is this starting to overload the site.xml
> > Perhaps there are other other ways.
>
> IMO that would not be as bad. The site.xml is the one part of forrest
> that the user really has to edit to add new content. All other files can
> be set to default.
Yes good to have minimal configuration points.
The worry is that already the site.xml is very free
for all, loose content. This makes it difficult for
validators to constrain the content for editors to
provide hints.
> Anyway that leads me to the approach that Ross uses for his sourceforge
> project. He uses manifest to extract tab and site.xml. He can edit the
> site with a GUI. IMO we need such a think very fast.
> ...or I have to really start the integration of forrest into lenya (et.
> al.). Then the user will not have to edit XML docs directly but through
> Bitflux, Kupu, et. al..
Did you say *or* integrate with Lenya!
I though that that was always on the schedule.
Anyway yes, integrate for sure. We need to be able to
configure Forrest via various mechanisms, always including
the vim command-line option.
--
David Crossley
Re: [RT] Search engines
Posted by Ross Gardler <rg...@apache.org>.
thorsten wrote:
> David Crossley wrote:
>
>> thorsten wrote:
>> <snip/>
>>
>>> *How it should work*
>>> We need the following components:
>>> I) site.xml - Here we can add the hooks for each site which search
>>> engine should be used. e.g.
>>> <todo label="Todo" href="todo.html" description="Todo List"
>>> searchType="os" searchEngine="all"/>
>>> This would offer the *all* search engines from the categorie *os*.
>>> <todo label="Todo" href="todo.html" description="Todo List"
>>> searchType="os" searchEngine="marc, eyebrowse"/>
>>> This would offer the marc and the eyebrowse search engines from the
>>> categorie *os*.
>>
>>
>> <snip/>
>>
>> Is this starting to overload the site.xml
>> Perhaps there are other other ways.
>>
>
> IMO that would not be as bad. The site.xml is the one part of forrest
> that the user really has to edit to add new content. All other files can
> be set to default.
>
> Anyway that leads me to the approach that Ross uses for his sourceforge
> project. He uses manifest to extract tab and site.xml. He can edit the
> site with a GUI. IMO we need such a think very fast.
The manifest stuff is the example plugin currently in the
sitemap-plugins branch
(http://svn.apache.org/viewcvs.cgi/forrest/branches/sitemap-plugins/?root=Apache-SVN).
The GUI portion is part of the Burrokeet project
(http://sourceforge.net/projects/burrokeet/), whether this stays a
separate project or gets donated to some other project can be visited
when it is a little more mature.
> ...or I have to really start the integration of forrest into lenya (et.
> al.). Then the user will not have to edit XML docs directly but through
> Bitflux, Kupu, et. al..
I have this second on my todo list at the moment. My plan of action
(which has not been discussed yet) is to integrate Lenya into Burrokeet,
which already has Forrest integrated through the Eclipse plugin in our
scratchpad here. I plan to do the same with Lenya. In the first instance
there will be two apps running (Forrest + Lenya), but I would hope to
bring the two together into a single app using the sitemap-plugins branch.
What we will then have is Burrokeet (Content Structure and config
editing), Lenya (workflow management and document editing) and Forrest
(content publishing). So, with respect to this RT, I would see the
management of search engines etc. being a part of the functionality of
Burrokeet. What this means, at least to me, is that we don't need to
worry too much about how easy it is to edit Forrests files by hand. We
should focus on a good clean design and really make use of Cocoon's
excellent cacheing facilities when generating the intermediate formats
that Forrest requires. That is, the core of Forrest is starting to get
quite complex within itself, the idea of editing these files by hand is
still acceptable but with more and more features being added, like the
search facilities Thosten is proposing, this editing is going to get
more complex.
We therefore have to decide what Forrest is intending to achieve, is it
going to provide easily edited "intermediate" files, or is it to create
efficient intermediate files that give us the maximum flexibility and
performance in the source -> intermediate -> publish process.
For me we have to go for flexibility and extendability in the
intermediate formats. We can look at ease of editing at the source stage
and is therefore out of the scope of the Forrest project (unless we want
to bring Burrokeet and any similar source editing projects into Forrest
itself).
Having said that, I don't know where these Search configuration files
should be, I'm just saying that personally I am not scared of a loss of
human readability at the the expense of increased flexibility. So, where
should these config files be?
Ross
Re: [RT] Search engines
Posted by thorsten <th...@apache.org>.
David Crossley wrote:
> thorsten wrote:
> <snip/>
>
>>*How it should work*
>>We need the following components:
>>I) site.xml - Here we can add the hooks for each site which search
>>engine should be used. e.g.
>><todo label="Todo" href="todo.html" description="Todo List"
>>searchType="os" searchEngine="all"/>
>>This would offer the *all* search engines from the categorie *os*.
>><todo label="Todo" href="todo.html" description="Todo List"
>>searchType="os" searchEngine="marc, eyebrowse"/>
>>This would offer the marc and the eyebrowse search engines from the
>>categorie *os*.
>
> <snip/>
>
> Is this starting to overload the site.xml
> Perhaps there are other other ways.
>
IMO that would not be as bad. The site.xml is the one part of forrest
that the user really has to edit to add new content. All other files can
be set to default.
Anyway that leads me to the approach that Ross uses for his sourceforge
project. He uses manifest to extract tab and site.xml. He can edit the
site with a GUI. IMO we need such a think very fast.
...or I have to really start the integration of forrest into lenya (et.
al.). Then the user will not have to edit XML docs directly but through
Bitflux, Kupu, et. al..
King regards
thorsten
Re: [RT] Search engines
Posted by David Crossley <cr...@apache.org>.
thorsten wrote:
<snip/>
>
> *How it should work*
> We need the following components:
> I) site.xml - Here we can add the hooks for each site which search
> engine should be used. e.g.
> <todo label="Todo" href="todo.html" description="Todo List"
> searchType="os" searchEngine="all"/>
> This would offer the *all* search engines from the categorie *os*.
> <todo label="Todo" href="todo.html" description="Todo List"
> searchType="os" searchEngine="marc, eyebrowse"/>
> This would offer the marc and the eyebrowse search engines from the
> categorie *os*.
<snip/>
Is this starting to overload the site.xml
Perhaps there are other other ways.
--
David Crossley