Extending Solr's Admin functionality

[Otis Gospodnetic <ot...@yahoo.com>]

Hello,

I may need to add functionality to Solr's admin pages.  The functionality that I'm looking to add is the ability to trigger certain indexing functions and monitor their progress.  I'm wondering if people have thoughts about the best way to do this.  Here are my initial ideas:

1. Add additional admin screens/JSPs, make them call custom classes that trigger indexing (e.g. go to a DB, retrieve some data, index it, maybe optimize when done), have that execute in a separate thread, and have these classes call Solr via custom HTTP requests that report progress, so that this progress/status can be viewed through another admin page for monitoring of this stuff.

2. Forget about triggering things from the UI.  Write generic/command-line-type classes, have them invoked independently of Solr, but still have them call Solr via custom HTTP requests that report progress, so that this progress/status can be viewed through another admin page for monitoring of this stuff.

I like 1, because everything is contained in Solr, but I fear it may be hard to maintain this extended version with Solr, unless the stuff I write ends up being generic enough that I can contribute it back.  I guess 2 would have some of these problems because I'd still need an admin page for monitoring.

Any thoughts?
Has anyone already envisioned a good way to extend Solr's functionality with custom admin screens?

Thanks,
Otis

Re: Extending Solr's Admin functionality

[Chris Hostetter <ho...@fucit.org>]

: I may need to add functionality to Solr's admin pages.  The
: functionality that I'm looking to add is the ability to trigger certain
: indexing functions and monitor their progress.  I'm wondering if people
: have thoughts about the best way to do this.  Here are my initial ideas:
:
: 1. Add additional admin screens/JSPs, make them call custom classes that
: trigger indexing (e.g. go to a DB, retrieve some data, index it, maybe
	...
: 2. Forget about triggering things from the UI.  Write
: generic/command-line-type classes, have them invoked independently of
	...

My advice would be to implement this as a seperate war, which could run in
the same servlet container instance (or not) and triggers
adds/deletes/commits over HTTP (as a "remote" client) ... you can have
your gui for triggering actions, you can have your (own) admin screen for
monitoring hte progress of thosee actions, you can link to the Solr
admin screens to see the low level progress, and you still have the nice
self contained system that can be shutdown as a single entity.  (so your
indexer is only running when you know your Solr instance is running)

: Any thoughts? Has anyone already envisioned a good way to extend Solr's
: functionality with custom admin screens?

way, way, *WAY* back in the day when Solr was just a gleam in Yonik's eye,
and i was fighting to convince him and Clay that i needed to be able to
run custom Java code on the server (a really advanced "scripting language"
for sending instructions was their suggestion to doing faceted searching)
i tried convincing them that "Solr core" should just be a really big jar,
filled with all of the code currently in Solr -- including the servlets
and JSPs -- and an extremely thin WAR with the basic path mappings for the
servlets that people could use if they wanted "bare bones" Solr.  but for
people who wanted super custom indexing functionality (like loading data
straight from a DB), or complex monitoring tasks, or admin screens, could
roll their own WAR around the SolrCore, picking and choosing the servlets
they want to use for interacting with it (or writting their own from
scratch)

We didn't go that route (obviously) and i eventually came arround to
realizing that was acctually the right decission.  Among other things;
it's allowed for a lot of "update" optimizations that probably wouldn't
have been possible with a totally open API.


If you *really* want to embed all of this in Solr, you could really do
just about anything you want in a request handler (including spinning up
background threads, accessing the SolrCore, and issuing update commands)
... monitoring could be done using the handleRequest method, or from the
stats page using the getStatistics method ... the one hitch you might run
into is that there is no "close" method on SolrRequestHandler at the
moment, so in a hot-deploy environment you might run into resource leaks
... but that should be easy to solve with a small API addition.

	...seems kind of "dirty" to me though.





-Hoss