You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@roller.apache.org by Brian Levine <le...@hotmail.com> on 2008/04/04 16:50:55 UTC
Custom Search
Has anyone had any luck in implementing a custom search?
I'm interested in searching not just blog entries, but also some other fields that are in my DB. I checked out the manuals and dug around in the code a bit (only found search plugins for google and wiki, but those seem to be something else). There doesn't seem to be any hooks in for adding my own search functionality. Does this sound right?
Since I don't want to change roller source because that would make upgrades/updates a PIA, my current plan is to write a new search servlet to handle what I want to do. But I still want to keep everything within the roller framework (if that makes sense). So, I'm also going to extend PageModel to be able to display the results in a different way -- because I'm going to be displaying more than blog entries. And then right my own velocity template for rendering the page. And it looks like I'll have to implement my own pager since as I mentioned before I'll be returning results that contain more than blog entries.
Has anyone had luck doing anything like this? Does this sound feasible?
Thanks!
/Brian
_________________________________________________________________
Use video conversation to talk face-to-face with Windows Live Messenger.
http://www.windowslive.com/messenger/connect_your_way.html?ocid=TXT_TAGLM_WL_Refresh_messenger_video_042008
RE: Custom Search
Posted by Brian Levine <le...@hotmail.com>.
> Date: Mon, 7 Apr 2008 08:54:42 -0400
> From: snoopdave@gmail.com
> To: user@roller.apache.org
> Subject: Re: Custom Search
>
> On Sun, Apr 6, 2008 at 10:00 PM, Brian Levine <le...@hotmail.com> wrote:
> >
> >
> >
> > > Date: Sat, 5 Apr 2008 14:30:07 -0400
> > > From: snoopdave@gmail.com
> > > To: user@roller.apache.org
> > > Subject: Re: Custom Search
> >
> >
> > >
> > > On Fri, Apr 4, 2008 at 10:50 AM, Brian Levine <le...@hotmail.com> wrote:
> > > > Has anyone had any luck in implementing a custom search?
> > > >
> > > > I'm interested in searching not just blog entries, but also some other fields that are in my DB. I checked out the manuals and dug around in the code a bit (only found search plugins for google and wiki, but those seem to be something else). There doesn't seem to be any hooks in for adding my own search functionality. Does this sound right?
> > >
> > > Yes, unfortunately that is correct. There is currently no search plugin facility
> > >
> > > > Since I don't want to change roller source because that would make upgrades/updates a PIA, my current plan is to write a new search servlet to handle what I want to do. But I still want to keep everything within the roller framework (if that makes sense). So, I'm also going to extend PageModel to be able to display the results in a different way -- because I'm going to be displaying more than blog entries. And then right my own velocity template for rendering the page. And it looks like I'll have to implement my own pager since as I mentioned before I'll be returning results that contain more than blog entries.
> > >
> > > Create your own page models to display your new search results sounds
> > > like a good idea.
> > >
> > > Question are: 1) how do you hook-in to ensure that you index the
> > > things you want to index and 2) how do return your search results so
> > > that they can be included in Roller-generated blog pages and feeds.
> > >
> > > Roller's backend is defined by a set of Java "manager" interfaces that
> > > create, retrieve, update and delete plain old Java objects or POJOs.
> > > The implementations of these interfaces are specified in Roller's
> > > Guice module file, which is called JPAWebloggerModule. By creating
> > > your own module class and plugging it in via roller-custom.properties,
> > > you can replace Roller's managers with your own implementations and
> > > even add entirely new managers.
> > >
> > > Override this property and substitute your own module, naming your
> > > implementations:
> > > guice.backend.module=org.apache.roller.weblogger.business.jpa.JPAWebloggerModule
> > >
> > > So, the answer to question #1 might be for you to override Roller's
> > > WeblogEntryManager with your own implementation, one that extends
> > > Roller's just to hook in your index operations every time an entry is
> > > saved, updated or deleted.
> > >
> > > For question #2, you might add your own new SearchManager interface
> > > with methods that return your new extended seach operations.
> > >
> > > Oh, and one thing to be aware of that Roller's Lucene implementation
> > > will not scale well, i.e. it won't work in a scenario where you have
> > > more than one instance of Roller running.
> > >
> > > - Dave
> >
> > Dave, thanks for your response!
> >
> > For question #1: After taking a look at the Guice file you mentioned, I think I can get away with overriding IndexManagerImp (and pointing to the new implementation in the Guice file), since I don't need to change how blog entries are stored and indexed (I know, not very clear from my description). Instead, I want to be able to search blog entries and I have other documents that I want to search. So I'll probably set up a second Lucene index for these different kinds of documents. Then I'll override the classes that are in the org.apache.roller.weblogger.business.search.operations package (or write new classes for the second Lucene index).
> >
> > For quesiton #2: I think I'm okay here. The extra, umm, stuff that I want people to be able to search for, shouldn't be displayed in feeds. The documents are web pages on other, external websites (it's a niche search engine). Is search being used in other places (besides the feeds), like where you display blog entries by category? Or someplace else?
> >
> > Scalability: Am I going to have trouble running lucene in a hosted environment where other accounts on the same machine are running roller? Will it be slow, or will it not work at all?
>
> No. The problem will occur if you have two instances of the Roller web
> app operating against the same database. Those two instances cannot
> write to the same Lucene index because they'll have a conflict writing
> to the same set of files. And each instance maintains it's own index,
> then those indexes will get out of sync.
>
> - Dave
Sounds good, thanks!
For those that are interested in doing something similar, I thought I'd describe how I added a new servlet to roller.
I basically took the existing SearchServlet class as a template. You'll have to edit web.xml to add the servlet information in the normal way to deploy it.
One important thing you'll have to do is implement your own request mapper (implement the org.apache.roller.weblogger.ui.rendering.RequstMapper interface). Roller does some url forwarding, and you won't be able to access your servlet unless you write this class. I used the WeblogRequstMapper (in the same package as RequestMapper) as a template for this. Basically you have to search for the url where your servlet should be accessed and if you find it, forward to that url and then return true. Returning true means that your request mapper will handle the request. If your class is not going to handle the request, you should return false so that roller can handle the request. To let roller know you have your own request mapper you should add this to your roller-custom.properties file:
rendering.userRequestMappers=\
package.of.your.RequestMapper
It gets called before roller's request mapper, so don't return true unless you want to handle the request, otherwise everything will break.
You can also load up your own templates from the servlet. The SearchServlet uses this code to get the template to use:
page = weblog.getTheme().getTemplateByAction(ThemeTemplate.ACTION_SEARCH);
There are some built-in actions, but unless you want to change rollers code, you'll need something else. Instead of getting a template by action, you need to get a template by name. This is the code I used:
page = weblog.getTheme().getTemplateByName ( MY_SEARCH_TEMPLATE ) ;
MY_SEARCH_TEMPLATE should be the same as the name of a template you create in roller (use the Design:Templates link to create new templates).
To populate the "stuff" that the velocity template will use, you need to create your own page model. For me I used SearchResultsModel as an example to work off of. Okay, this next part I'm a little hazy on, but I think roller uses multiple models to render a page, not just the template you designate. So to access those other models, from your page model you need to add this to your roller-custom.properties file:
rendering.mySearchModels=\
path.to.MySearchResultsModel,\
org.apache.roller.weblogger.ui.rendering.model.ConfigModel,\
org.apache.roller.weblogger.ui.rendering.model.UtilitiesModel,\
org.apache.roller.weblogger.ui.rendering.model.URLModel,\
org.apache.roller.weblogger.ui.rendering.model.MessageModel,\
org.apache.roller.weblogger.ui.rendering.model.CalendarModel,\
org.apache.roller.weblogger.ui.rendering.model.MenuModel
Then in your servlet you need to load these models with this code:
String searchModels = WebloggerConfig.getProperty("rendering.mySearchModels");
ModelLoader.loadModels(searchModels, model, initData, true);
I think that's pretty much it. To use the new code, I created a new jar file and dropped it into webapp/roller/WEB-INF/lib. Re jar'd roller and redployed the roller jar file to my tomcat install. I'm a bit new to deploying web apps, so this may not be a "best practice" way of deploying my jar, but it worked, so that's something.
Hope this is helpful to someone else. At the very least when I forgot what I've done, I can find this message and remind myself :)
/Brian
_________________________________________________________________
Get in touch in an instant. Get Windows Live Messenger now.
http://www.windowslive.com/messenger/overview.html?ocid=TXT_TAGLM_WL_Refresh_getintouch_042008
Re: Custom Search
Posted by Dave <sn...@gmail.com>.
On Sun, Apr 6, 2008 at 10:00 PM, Brian Levine <le...@hotmail.com> wrote:
>
>
>
> > Date: Sat, 5 Apr 2008 14:30:07 -0400
> > From: snoopdave@gmail.com
> > To: user@roller.apache.org
> > Subject: Re: Custom Search
>
>
> >
> > On Fri, Apr 4, 2008 at 10:50 AM, Brian Levine <le...@hotmail.com> wrote:
> > > Has anyone had any luck in implementing a custom search?
> > >
> > > I'm interested in searching not just blog entries, but also some other fields that are in my DB. I checked out the manuals and dug around in the code a bit (only found search plugins for google and wiki, but those seem to be something else). There doesn't seem to be any hooks in for adding my own search functionality. Does this sound right?
> >
> > Yes, unfortunately that is correct. There is currently no search plugin facility
> >
> > > Since I don't want to change roller source because that would make upgrades/updates a PIA, my current plan is to write a new search servlet to handle what I want to do. But I still want to keep everything within the roller framework (if that makes sense). So, I'm also going to extend PageModel to be able to display the results in a different way -- because I'm going to be displaying more than blog entries. And then right my own velocity template for rendering the page. And it looks like I'll have to implement my own pager since as I mentioned before I'll be returning results that contain more than blog entries.
> >
> > Create your own page models to display your new search results sounds
> > like a good idea.
> >
> > Question are: 1) how do you hook-in to ensure that you index the
> > things you want to index and 2) how do return your search results so
> > that they can be included in Roller-generated blog pages and feeds.
> >
> > Roller's backend is defined by a set of Java "manager" interfaces that
> > create, retrieve, update and delete plain old Java objects or POJOs.
> > The implementations of these interfaces are specified in Roller's
> > Guice module file, which is called JPAWebloggerModule. By creating
> > your own module class and plugging it in via roller-custom.properties,
> > you can replace Roller's managers with your own implementations and
> > even add entirely new managers.
> >
> > Override this property and substitute your own module, naming your
> > implementations:
> > guice.backend.module=org.apache.roller.weblogger.business.jpa.JPAWebloggerModule
> >
> > So, the answer to question #1 might be for you to override Roller's
> > WeblogEntryManager with your own implementation, one that extends
> > Roller's just to hook in your index operations every time an entry is
> > saved, updated or deleted.
> >
> > For question #2, you might add your own new SearchManager interface
> > with methods that return your new extended seach operations.
> >
> > Oh, and one thing to be aware of that Roller's Lucene implementation
> > will not scale well, i.e. it won't work in a scenario where you have
> > more than one instance of Roller running.
> >
> > - Dave
>
> Dave, thanks for your response!
>
> For question #1: After taking a look at the Guice file you mentioned, I think I can get away with overriding IndexManagerImp (and pointing to the new implementation in the Guice file), since I don't need to change how blog entries are stored and indexed (I know, not very clear from my description). Instead, I want to be able to search blog entries and I have other documents that I want to search. So I'll probably set up a second Lucene index for these different kinds of documents. Then I'll override the classes that are in the org.apache.roller.weblogger.business.search.operations package (or write new classes for the second Lucene index).
>
> For quesiton #2: I think I'm okay here. The extra, umm, stuff that I want people to be able to search for, shouldn't be displayed in feeds. The documents are web pages on other, external websites (it's a niche search engine). Is search being used in other places (besides the feeds), like where you display blog entries by category? Or someplace else?
>
> Scalability: Am I going to have trouble running lucene in a hosted environment where other accounts on the same machine are running roller? Will it be slow, or will it not work at all?
No. The problem will occur if you have two instances of the Roller web
app operating against the same database. Those two instances cannot
write to the same Lucene index because they'll have a conflict writing
to the same set of files. And each instance maintains it's own index,
then those indexes will get out of sync.
- Dave
RE: Custom Search
Posted by Brian Levine <le...@hotmail.com>.
> Date: Sat, 5 Apr 2008 14:30:07 -0400
> From: snoopdave@gmail.com
> To: user@roller.apache.org
> Subject: Re: Custom Search
>
> On Fri, Apr 4, 2008 at 10:50 AM, Brian Levine <le...@hotmail.com> wrote:
> > Has anyone had any luck in implementing a custom search?
> >
> > I'm interested in searching not just blog entries, but also some other fields that are in my DB. I checked out the manuals and dug around in the code a bit (only found search plugins for google and wiki, but those seem to be something else). There doesn't seem to be any hooks in for adding my own search functionality. Does this sound right?
>
> Yes, unfortunately that is correct. There is currently no search plugin facility
>
>
> > Since I don't want to change roller source because that would make upgrades/updates a PIA, my current plan is to write a new search servlet to handle what I want to do. But I still want to keep everything within the roller framework (if that makes sense). So, I'm also going to extend PageModel to be able to display the results in a different way -- because I'm going to be displaying more than blog entries. And then right my own velocity template for rendering the page. And it looks like I'll have to implement my own pager since as I mentioned before I'll be returning results that contain more than blog entries.
>
> Create your own page models to display your new search results sounds
> like a good idea.
>
> Question are: 1) how do you hook-in to ensure that you index the
> things you want to index and 2) how do return your search results so
> that they can be included in Roller-generated blog pages and feeds.
>
> Roller's backend is defined by a set of Java "manager" interfaces that
> create, retrieve, update and delete plain old Java objects or POJOs.
> The implementations of these interfaces are specified in Roller's
> Guice module file, which is called JPAWebloggerModule. By creating
> your own module class and plugging it in via roller-custom.properties,
> you can replace Roller's managers with your own implementations and
> even add entirely new managers.
>
> Override this property and substitute your own module, naming your
> implementations:
> guice.backend.module=org.apache.roller.weblogger.business.jpa.JPAWebloggerModule
>
> So, the answer to question #1 might be for you to override Roller's
> WeblogEntryManager with your own implementation, one that extends
> Roller's just to hook in your index operations every time an entry is
> saved, updated or deleted.
>
> For question #2, you might add your own new SearchManager interface
> with methods that return your new extended seach operations.
>
> Oh, and one thing to be aware of that Roller's Lucene implementation
> will not scale well, i.e. it won't work in a scenario where you have
> more than one instance of Roller running.
>
> - Dave
Dave, thanks for your response!
For question #1: After taking a look at the Guice file you mentioned, I think I can get away with overriding IndexManagerImp (and pointing to the new implementation in the Guice file), since I don't need to change how blog entries are stored and indexed (I know, not very clear from my description). Instead, I want to be able to search blog entries and I have other documents that I want to search. So I'll probably set up a second Lucene index for these different kinds of documents. Then I'll override the classes that are in the org.apache.roller.weblogger.business.search.operations package (or write new classes for the second Lucene index).
For quesiton #2: I think I'm okay here. The extra, umm, stuff that I want people to be able to search for, shouldn't be displayed in feeds. The documents are web pages on other, external websites (it's a niche search engine). Is search being used in other places (besides the feeds), like where you display blog entries by category? Or someplace else?
Scalability: Am I going to have trouble running lucene in a hosted environment where other accounts on the same machine are running roller? Will it be slow, or will it not work at all?
Thanks again Dave.
/Brian
_________________________________________________________________
Use video conversation to talk face-to-face with Windows Live Messenger.
http://www.windowslive.com/messenger/connect_your_way.html?ocid=TXT_TAGLM_WL_Refresh_messenger_video_042008
Re: Custom Search
Posted by Dave <sn...@gmail.com>.
On Fri, Apr 4, 2008 at 10:50 AM, Brian Levine <le...@hotmail.com> wrote:
> Has anyone had any luck in implementing a custom search?
>
> I'm interested in searching not just blog entries, but also some other fields that are in my DB. I checked out the manuals and dug around in the code a bit (only found search plugins for google and wiki, but those seem to be something else). There doesn't seem to be any hooks in for adding my own search functionality. Does this sound right?
Yes, unfortunately that is correct. There is currently no search plugin facility
> Since I don't want to change roller source because that would make upgrades/updates a PIA, my current plan is to write a new search servlet to handle what I want to do. But I still want to keep everything within the roller framework (if that makes sense). So, I'm also going to extend PageModel to be able to display the results in a different way -- because I'm going to be displaying more than blog entries. And then right my own velocity template for rendering the page. And it looks like I'll have to implement my own pager since as I mentioned before I'll be returning results that contain more than blog entries.
Create your own page models to display your new search results sounds
like a good idea.
Question are: 1) how do you hook-in to ensure that you index the
things you want to index and 2) how do return your search results so
that they can be included in Roller-generated blog pages and feeds.
Roller's backend is defined by a set of Java "manager" interfaces that
create, retrieve, update and delete plain old Java objects or POJOs.
The implementations of these interfaces are specified in Roller's
Guice module file, which is called JPAWebloggerModule. By creating
your own module class and plugging it in via roller-custom.properties,
you can replace Roller's managers with your own implementations and
even add entirely new managers.
Override this property and substitute your own module, naming your
implementations:
guice.backend.module=org.apache.roller.weblogger.business.jpa.JPAWebloggerModule
So, the answer to question #1 might be for you to override Roller's
WeblogEntryManager with your own implementation, one that extends
Roller's just to hook in your index operations every time an entry is
saved, updated or deleted.
For question #2, you might add your own new SearchManager interface
with methods that return your new extended seach operations.
Oh, and one thing to be aware of that Roller's Lucene implementation
will not scale well, i.e. it won't work in a scenario where you have
more than one instance of Roller running.
- Dave