SolrJ client changes

[Kirk True <ki...@gmail.com>]

Hi all,

First off, thanks for the great project. We're really excited about
incorporating Solr into our stack. Thanks!

We're looking at making some changes to the SolrJ client to meet our
operational needs:


   - routing of requests to specific nodes for better cache hit ratios
   - adding instrumentation via JMX (list of availability of Solr servers,
   response time, various counters)
    - implementing connection pooling (failure detection and recovery);
   this is probably achievable on a layer atop the SolrJ client vs. as a part
   of it
    - logging errors more consistently
   - miscellaneous changes (debugging, use of POST, optional custom
   User-Agent, etc.)


So far, it appears that these changes can be made by adding hooks or
optional APIs to SolrJ that we would then implement with our custom logic.
So there's no secret sauce or anything.

We are curious for your recommendations about how best to make these
changes.

A few questions:


   1. It seems that the SolrJ client is part of the overall Solr project,
   so it would be harder/slower for us to get our changes in, committed, and
   released in a timely manner.
   2. We're specifically interested in the load balancing SolrJ client
   (LBHttpSolrServer) but it doesn't appear to be recommended for production
   use. Thoughts on its usage?
   3. Are there perhaps other clients for Solr for Java other than SolrJ
   and scalikesolr?


Thanks,
Kirk

Re: SolrJ client changes

[Mark Bennett <ma...@lucidworks.com>]

Kirk,

Welcome to the project.  Taking a stab at a couple items:

On Mar 4, 2013, at 7:50 AM, Kirk True <ki...@gmail.com> wrote:

> Hi all,
> 
> First off, thanks for the great project. We're really excited about incorporating Solr into our stack. Thanks!
> 
> We're looking at making some changes to the SolrJ client to meet our operational needs:
> 
> routing of requests to specific nodes for better cache hit ratios
Sounds more like the prefix routing in 4.1 (http://wiki.apache.org/solr/ReleaseNote41)
> adding instrumentation via JMX (list of availability of Solr servers, response time, various counters)
> implementing connection pooling (failure detection and recovery); this is probably achievable on a layer atop the SolrJ client vs. as a part of it
> logging errors more consistently
On the client (SolrJ) or server side?
> miscellaneous changes (debugging, use of POST, optional custom User-Agent, etc.)
I would have thought POST and user-agent were controllable already… though they don't jump out at me in the javadoc.  Debugging, again, client vs. solr side?

> So far, it appears that these changes can be made by adding hooks or optional APIs to SolrJ that we would then implement with our custom logic. So there's no secret sauce or anything.
> 
> We are curious for your recommendations about how best to make these changes.
> 
> A few questions:
> 
> It seems that the SolrJ client is part of the overall Solr project, so it would be harder/slower for us to get our changes in, committed, and released in a timely manner.
Open a JIRA for what you've done, and attach a patch.  If that seems to languish, maybe ping the group.
> We're specifically interested in the load balancing SolrJ client (LBHttpSolrServer) but it doesn't appear to be recommended for production use. Thoughts on its usage?
> Are there perhaps other clients for Solr for Java other than SolrJ and scalikesolr?
> 
> Thanks,
> Kirk


--
Mark Bennett / LucidWorks: Search & Big Data / mark.bennett@lucidworks.com
Office: 408-898-4201 / Telecommute: 408-733-0387 / Cell: 408-829-6513