Re: SolrConfig.Initializable

[Henrib <hb...@gmail.com>]

Just pinging on the subject and the related solr-399 patch; I understand the
priority for this is low, just seeking some comment to determine whether
this is the expected track (and how low the priority is).

-- 
View this message in context: http://www.nabble.com/SolrConfig.Initializable-tf4665036.html#a13586685
Sent from the Solr - Dev mailing list archive at Nabble.com.

Re: SolrConfig.Initializable

[Henrib <hb...@gmail.com>]

ryantxu wrote:
> 
> but I fear we are turning the initialization into a jumbled spaghetti of
> interface looparounds, callbacks and pending lists 
> of soon-to-be-initialized stuff.
> (not just this patch - but the combination of requirements we have had for
> all the plugin stuff)
> 
I agree this is "messy"; a common/simple void init(SolrCore core, ...) would
be much better.
I'm no fan of a thread local solution and I believe we can smoothly
transition the API the same way we did when we introduced the
SolrConfig.Initializable (deprecation plus default implementation).


ryantxu wrote:
> 
> One key design choice is if we can pass a not fully initialized SolrCore
> to an init method.
> 
I think we should pass the core even if not fully initialized since there
are already ways today to access the core during init phase anyway
(SolrCore.getSolrCore).
We should document the config/schema/core initialization order so users know
what they can access from a core/schema/config during init (before the core
really starts).
Plus, if we are able to keep the solr-399 feature, besides cyclic or
unsolvable dependencies, the initialization order might just adapt.


ryantxu wrote:
> 
> The solrconfig stuff definitely needs access to the current core. 
> I'm not sure if schema related things need access (it currently does not). 
> Both cases need access to the 'config' in order to access the ClassLoader.
> 
One of the starting point was to allow a tokenizer to use a fieldType;
schema related objects thus need access to others and passing the core would
allow that (getSchema).
Just the same way the core let's us access the solrconfig (getSolrConfig).


ryantxu wrote:
> 
> alternatively, we have to do something like SOLR-399 that queues up
> everything until the core is completely initialized.
> 
I guess solr-399 algorithm can be adapted & kept; if nothing prevents
initialization do it, otherwise postpone it.
Instead of a boolean init(...), we might try to use an exception to indicate
initialization is to be postponed & retried (considering this case as an
exceptional but recoverable workflow).
And we could capture the init continuation in the exception (ala
PluginInitInfo for instance).
Let me know if I should investigate feasibility more thoroughly.
-- 
View this message in context: http://www.nabble.com/SolrConfig.Initializable-tf4665036.html#a13743457
Sent from the Solr - Dev mailing list archive at Nabble.com.

Re: SolrConfig.Initializable

[Ryan McKinley <ry...@gmail.com>]

Henrib wrote:
> Just pinging on the subject and the related solr-399 patch; I understand the
> priority for this is low, just seeking some comment to determine whether
> this is the expected track (and how low the priority is).
> 

I started working on adding configuration to the search components in 
SOLR-281.  This is a case where we need access to the core for 
initialization.

I checked out the SOLR-399 patch - it looks good considering all our 
discussion... but I fear we are turning the initialization into a 
jumbled spaghetti of interface looparounds, callbacks and pending lists 
of soon-to-be-initialized stuff.  (not just this patch - but the 
combination of requirements we have had for all the plugin stuff)

I think we need to have a clear initialization strategy and make sure 
everything conforms to that before releasing 1.3.

Looking over our plugin space, we have two main categories:
1. stuff defined in schema.xml (fieldTypes fields ...)
2. stuff defined in solrconfig.xml (requestHandlers, processors, 
components, writers, ...)

The solrconfig stuff definitely needs access to the current core.  I'm 
not sure if schema related things need access (it currently does not). 
Both cases need access to the 'config' in order to access the ClassLoader.

One key design choice is if we can pass a not fully initialized SolrCore 
to an init method.  alternatively, we have to do something like SOLR-399 
that queues up everything until the core is completely initialized.


Rather then jumble things up with more hooks, queues and callbacks, I 
vote we migrate the NamedList/Map/NodeInitializedPlugin from:
   void init( final NamedList args )
to:
   void init( final SolrCore core, final NamedList args )


I'm not sure what the best approach for the schema family is.  As is, 
IndexSchema can be fully initialized independent of SolrCore -- for 
consistency, it may be nice to use:
   void init( final SolrCore core, final Map<String,String> args )
rather then:
   void init(SolrConfig solrConfig, Map<String,String> args)
If fields/fieldTypes will never need access to the current SolrCore, it 
is fine to leave as is.

- - - -

Another possibility is to perhaps use ThreadLocal to register the 
current core/config while initializing.  That seems a bit dangerous as 
it would assume preconditions that are not required by the API.  But 
this would make it possible to pass things around and not break/migrate 
the existing API.


ryan

Re: SolrConfig.Initializable

[Henrib <hb...@gmail.com>]

ryantxu wrote:
> 
> Is the patch ready for a serious review? or are you still working on it?
> 

Not sure if this I'm supposed to answer but anyway, I was expecting comments
before 'generalizing' the patch. Right now, it only applies to filter &
tokenizer factories. The foundation should be solid though but since this
introduces new interfaces, some review is needed.

Let me know if there is anything you'd like me to change.

-- 
View this message in context: http://www.nabble.com/SolrConfig.Initializable-tf4665036.html#a13704330
Sent from the Solr - Dev mailing list archive at Nabble.com.

Re: SolrConfig.Initializable

[Chris Hostetter <ho...@fucit.org>]

: dooh - I was hoping Chris may take this one up as he started the thread on
: changing the init method :)
: 
: If this is not an issue for Chris, I can take a look sometime in the next few

sorry ... i didn't mean to suggest that i had a specific need to change 
the method ... i was approaching it more from the "because of multi-core, 
the init signature and semantics will change from 1.2 to 1.3; so what do 
we *really* want init methods to look like moving forward?" (as opposed to 
the init signature currently in the trunk that was made to be the minimum 
neccessary changes to support both multi-cores and the existing factories)


-Hoss

Re: SolrConfig.Initializable

[Ryan McKinley <ry...@gmail.com>]

Chris Hostetter wrote:
> : Just pinging on the subject and the related solr-399 patch; I understand the
> : priority for this is low, just seeking some comment to determine whether
> : this is the expected track (and how low the priority is).
> 
> It's in my mailbox of things to look at .... but there are things on that 
> list from september -- when my mail server died and i had to restarted my 
> list from scratch.
> 
> As i recall, the ideas you mentioned in email made sense to me ... I was 
> kind of hopeing Ryan would take the ball and run with it since he's really 
> become the "multi-core warrior"  :)
> 

dooh - I was hoping Chris may take this one up as he started the thread 
on changing the init method :)

If this is not an issue for Chris, I can take a look sometime in the 
next few days.  A quick look at the patch and it looks ok (i have not 
gone into detail yet)

Is the patch ready for a serious review? or are you still working on it?

ryan

Re: SolrConfig.Initializable

[Chris Hostetter <ho...@fucit.org>]

: Just pinging on the subject and the related solr-399 patch; I understand the
: priority for this is low, just seeking some comment to determine whether
: this is the expected track (and how low the priority is).

It's in my mailbox of things to look at .... but there are things on that 
list from september -- when my mail server died and i had to restarted my 
list from scratch.

As i recall, the ideas you mentioned in email made sense to me ... I was 
kind of hopeing Ryan would take the ball and run with it since he's really 
become the "multi-core warrior"  :)


-Hoss