Accessing SolrZkClient instance from a plug-in?

[Timothy Potter <th...@gmail.com>]

I have a custom ValueSourceParser that sets up a Zookeeper Watcher on some
frequently changing metadata that a custom ValueSource depends on.

Basic flow of events is - VSP watches for metadata changes, which triggers
a refresh of some expensive data that my custom ValueSource uses at query
time. Think of the data in Zookeeper as a pointer to some larger dataset
that is computed offline and then loaded into memory for use by my custom
ValueSource.

In my ValueSourceParser, I connect to Zookeeper using an instance of the
SolrZkClient class and am receiving WatchedEvents when my metadata changes
(as expected).

All this works great until core reload happens. From what I can tell,
there's no shutdown hook for ValueSourceParsers, so what's happening is
that my code ends up adding multiple Watchers and thus receives multiple
update events when the metadata changes.

What I need is either

1) a shutdown hook in my VSP that allows me to clean-up the SolrZkClient
instance my code is managing, or

2) access to the ZkController instance owned by the CoreContainer from my
VSP.

For me #2 is better as I'd prefer to just re-use Solr's instance of
SolrZkClient.

I can go and hack either of these in pretty easily but wanted to see if
someone knows a better way to get 1 or 2?

In general, it might be handy to allow plug-ins to get access to the
Zookeeper client SolrCloud is using.

Thanks.
Tim

Re: Accessing SolrZkClient instance from a plug-in?

[Timothy Potter <th...@gmail.com>]

Brilliant! Thank you - I was focusing on the init method and totally
ignored the FunctionQParser passed to the parse method.

Cheers,
Tim

On Mon, Mar 25, 2013 at 4:16 PM, Mark Miller <ma...@gmail.com> wrote:

> I don't know the ValueSourceParser from a hole in my head, but it looks
> like it has access to the solrcore with fp.req.getCore?
>
> If so, it's easy to get the zk stuff
>
> core.getCoreDescriptor.getCoreContainer.getZkController(.getZkClient).
>
> From memory, so perhaps with some minor misname.
>
> - Mark
>
> On Mar 25, 2013, at 6:03 PM, Timothy Potter <th...@gmail.com> wrote:
>
> > I have a custom ValueSourceParser that sets up a Zookeeper Watcher on
> some
> > frequently changing metadata that a custom ValueSource depends on.
> >
> > Basic flow of events is - VSP watches for metadata changes, which
> triggers
> > a refresh of some expensive data that my custom ValueSource uses at query
> > time. Think of the data in Zookeeper as a pointer to some larger dataset
> > that is computed offline and then loaded into memory for use by my custom
> > ValueSource.
> >
> > In my ValueSourceParser, I connect to Zookeeper using an instance of the
> > SolrZkClient class and am receiving WatchedEvents when my metadata
> changes
> > (as expected).
> >
> > All this works great until core reload happens. From what I can tell,
> > there's no shutdown hook for ValueSourceParsers, so what's happening is
> > that my code ends up adding multiple Watchers and thus receives multiple
> > update events when the metadata changes.
> >
> > What I need is either
> >
> > 1) a shutdown hook in my VSP that allows me to clean-up the SolrZkClient
> > instance my code is managing, or
> >
> > 2) access to the ZkController instance owned by the CoreContainer from my
> > VSP.
> >
> > For me #2 is better as I'd prefer to just re-use Solr's instance of
> > SolrZkClient.
> >
> > I can go and hack either of these in pretty easily but wanted to see if
> > someone knows a better way to get 1 or 2?
> >
> > In general, it might be handy to allow plug-ins to get access to the
> > Zookeeper client SolrCloud is using.
> >
> > Thanks.
> > Tim
>
>

Re: Accessing SolrZkClient instance from a plug-in?

[Mark Miller <ma...@gmail.com>]

I don't know the ValueSourceParser from a hole in my head, but it looks like it has access to the solrcore with fp.req.getCore?

If so, it's easy to get the zk stuff

core.getCoreDescriptor.getCoreContainer.getZkController(.getZkClient).

From memory, so perhaps with some minor misname.

- Mark

On Mar 25, 2013, at 6:03 PM, Timothy Potter <th...@gmail.com> wrote:

> I have a custom ValueSourceParser that sets up a Zookeeper Watcher on some
> frequently changing metadata that a custom ValueSource depends on.
> 
> Basic flow of events is - VSP watches for metadata changes, which triggers
> a refresh of some expensive data that my custom ValueSource uses at query
> time. Think of the data in Zookeeper as a pointer to some larger dataset
> that is computed offline and then loaded into memory for use by my custom
> ValueSource.
> 
> In my ValueSourceParser, I connect to Zookeeper using an instance of the
> SolrZkClient class and am receiving WatchedEvents when my metadata changes
> (as expected).
> 
> All this works great until core reload happens. From what I can tell,
> there's no shutdown hook for ValueSourceParsers, so what's happening is
> that my code ends up adding multiple Watchers and thus receives multiple
> update events when the metadata changes.
> 
> What I need is either
> 
> 1) a shutdown hook in my VSP that allows me to clean-up the SolrZkClient
> instance my code is managing, or
> 
> 2) access to the ZkController instance owned by the CoreContainer from my
> VSP.
> 
> For me #2 is better as I'd prefer to just re-use Solr's instance of
> SolrZkClient.
> 
> I can go and hack either of these in pretty easily but wanted to see if
> someone knows a better way to get 1 or 2?
> 
> In general, it might be handy to allow plug-ins to get access to the
> Zookeeper client SolrCloud is using.
> 
> Thanks.
> Tim