Dynamic RequestHandler loading

[Ryan McKinley <ry...@gmail.com>]

I know I'm pushing solr to do things it was never designed to do, so
shut me up quick if this is not where you want things to go - I could
quietly implement this with quick hacks, but i'd rather not...

Currently SolrCore loads all the request handlers in a final variable
as the instance is initialized:

private final RequestHandlers reqHandlers = new
RequestHandlers(SolrConfig.config);

This is a little strange because forces the request handlers to be
loaded as the instance is initialized - not in a more appropriate
place like after initWriters() in the constructor.  The bad side
affects to this are that handlers can not know about the schema or
config directory in the init() method.  They are forced to do some
sort of lazy loading.

The other thing I would like is to be able to dynamically register
handlers.  For example, i have one handler that wants to register 6
related handlers (each one maps a different action - it only makes
sense to have them as a collection).  While i could put six entries in
solrconfig, this will quickly become unruly.

How do you feel about:

1. move the request handler initialization into the constructor after
initWriters()?

2. Exposing the following functions in SolrCore:

 // this already exists
  SolrRequestHandler getRequestHandler(String handlerName);

 // this will register the handler and return whatever used to be at
that path (or null)
  SolrRequestHandler registerRequestHandler(String handlerName,
SolrRequestHandler )

 // get all the registered handlers by class
  Collection<SolrRequestHandler> getRequestHandlers( Class<? extends,
SolrRequestHandler> clazz );

3. Give request handlers a standard way to know what path/name they
are registered to.  The simplest way i can think to do this is to add
parameters to the NamedList passed to init()


thoughts?

ryan

Re: Dynamic RequestHandler loading

[Chris Hostetter <ho...@fucit.org>]

: getRequestHandlers() would be equivolent to:
: getRequestHandlers( SolrRequestHandler.class )
:
: We will need some way to ask what is registered without knowing the
: path it is registered to.

getting instances by class seems like a pretty special case situation ...
i'd rather not add a bunch of methods that really only have one use case
which isn't even in the "main" code base.

Adding a "Map<String,SolrRequestHandler> getRequestHandlers()" method to
the core seems useful enough in a broad case to solve any special needs
custom code might have -- find instances by interface etc...

As long as we change the SolrCore initialization to construct all
SOlrRequestHandler instances and build up that Map prior to calling hte
"init" method on them, it would also solve the "what name am i registered
with" question for RequestHandlers without needing to change the INterface
in a backwards incompatible way.  (handlers that want to know could get
the Map and look for a value they are "==" to)


-Hoss

Re: Dynamic RequestHandler loading

[Ryan McKinley <ry...@gmail.com>]

> >
> >  // get all the registered handlers by class
> >   Collection<SolrRequestHandler> getRequestHandlers( Class<? extends,
> > SolrRequestHandler> clazz );
>
> By class?  What's that for?
>

It was useful to check what else is configured.  The alternative is to have a
  Collection<SolrRequestHandler> getRequestHandlers() and have let the
client sort out what is what.  In the case I am looking at, I want to
check if a handler of given type is already registered - if not, i
register one.

getRequestHandlers() would be equivolent to:
getRequestHandlers( SolrRequestHandler.class )

We will need some way to ask what is registered without knowing the
path it is registered to.

> Everything else seems to make sense... it would mean an extra
> synchronization per Solr request, but that shouldn't be measurable
> given everything else we are doing.
>

would access to getRequestHandler( name ) need to be synchronized?

Are there any real problems if not?  I don't imagine much would change
after startup.


> One thing we lost going from individual servlets to a filter was the
> potential for load-on-demand handlers.  If/when CSV and SQL update
> handlers get put in the core, it might be nice if they weren't loaded
> at startup.
>

perhaps we could add:
 <requestHandler name="..." class=".." statup="lazy" />

Then we could have a LazyRequestHandlerWrapper that only hangs on to
configuration parameters, but done not initialize the delegate handler
until its first request.  This would be transparent to the
Filter/Servlet framework

But I'll save that for another day :)

Re: Dynamic RequestHandler loading

[Yonik Seeley <yo...@apache.org>]

On 3/5/07, Ryan McKinley <ry...@gmail.com> wrote:
> I know I'm pushing solr to do things it was never designed to do, so
> shut me up quick if this is not where you want things to go - I could
> quietly implement this with quick hacks, but i'd rather not...
>
> Currently SolrCore loads all the request handlers in a final variable
> as the instance is initialized:
>
> private final RequestHandlers reqHandlers = new
> RequestHandlers(SolrConfig.config);
>
> This is a little strange because forces the request handlers to be
> loaded as the instance is initialized - not in a more appropriate
> place like after initWriters() in the constructor.  The bad side
> affects to this are that handlers can not know about the schema or
> config directory in the init() method.  They are forced to do some
> sort of lazy loading.
>
> The other thing I would like is to be able to dynamically register
> handlers.  For example, i have one handler that wants to register 6
> related handlers (each one maps a different action - it only makes
> sense to have them as a collection).  While i could put six entries in
> solrconfig, this will quickly become unruly.
>
> How do you feel about:
>
> 1. move the request handler initialization into the constructor after
> initWriters()?
>
> 2. Exposing the following functions in SolrCore:
>
>  // this already exists
>   SolrRequestHandler getRequestHandler(String handlerName);
>
>  // this will register the handler and return whatever used to be at
> that path (or null)
>   SolrRequestHandler registerRequestHandler(String handlerName,
> SolrRequestHandler )
>
>  // get all the registered handlers by class
>   Collection<SolrRequestHandler> getRequestHandlers( Class<? extends,
> SolrRequestHandler> clazz );

By class?  What's that for?

Everything else seems to make sense... it would mean an extra
synchronization per Solr request, but that shouldn't be measurable
given everything else we are doing.

One thing we lost going from individual servlets to a filter was the
potential for load-on-demand handlers.  If/when CSV and SQL update
handlers get put in the core, it might be nice if they weren't loaded
at startup.

-Yonik