Extensibility of HttpSolrServer

[Eugen Paraschiv <eu...@odesk.com>]

Hi,
Quick question about the HttpSolrServer implementation - I would like to
extend some of the functionality of this class - but when I extend it - I'm
having issues with how extensible it is.
For example - some of the details are not visible externally - setters
exist for maxRetries and followRedirects but no getters.
It would really help to make this class a bit more extensible - I'm sure it
usually enough, but when it does need to be extended - it would make sense
to allow that rather than the client implement alternative version of it
via copy-paste (which looks like the only option available right now).
Hope this makes sense.
Cheers,
Eugen.

Re: Extensibility of HttpSolrServer

[Eugen Paraschiv <eu...@odesk.com>]

Will do - thanks for the quick feedback.
Eugen.


On Thu, Nov 21, 2013 at 12:06 AM, Mark Miller <ma...@gmail.com> wrote:

> Feel free to file a JIRA issue with the changes you think make sense.
>
> - Mark
>
> On Nov 20, 2013, at 4:21 PM, Eugen Paraschiv <eu...@odesk.com> wrote:
>
> > Hi,
> > Quick question about the HttpSolrServer implementation - I would like to
> > extend some of the functionality of this class - but when I extend it -
> I'm
> > having issues with how extensible it is.
> > For example - some of the details are not visible externally - setters
> > exist for maxRetries and followRedirects but no getters.
> > It would really help to make this class a bit more extensible - I'm sure
> it
> > usually enough, but when it does need to be extended - it would make
> sense
> > to allow that rather than the client implement alternative version of it
> > via copy-paste (which looks like the only option available right now).
> > Hope this makes sense.
> > Cheers,
> > Eugen.
>
>

Re: Extensibility of HttpSolrServer

[Mark Miller <ma...@gmail.com>]

Feel free to file a JIRA issue with the changes you think make sense.

- Mark

On Nov 20, 2013, at 4:21 PM, Eugen Paraschiv <eu...@odesk.com> wrote:

> Hi,
> Quick question about the HttpSolrServer implementation - I would like to
> extend some of the functionality of this class - but when I extend it - I'm
> having issues with how extensible it is.
> For example - some of the details are not visible externally - setters
> exist for maxRetries and followRedirects but no getters.
> It would really help to make this class a bit more extensible - I'm sure it
> usually enough, but when it does need to be extended - it would make sense
> to allow that rather than the client implement alternative version of it
> via copy-paste (which looks like the only option available right now).
> Hope this makes sense.
> Cheers,
> Eugen.

Re: Extensibility of HttpSolrServer

[Eugen Paraschiv <eu...@odesk.com>]

First - I completely agree with keeping the moving parts to a minimum - but
I do think that's a case by case decision, and in this particular case - it
may just be worth opening up a little.
Then - adding in a custom HttpClient may work - but HttpHeaders are set on
a request (and may differ from one request to the other) - so doing it in a
custom http client may be technically doable, but it's probably going to be
(and look like - from the API POV) a workaround.
As a quick example - I'm using a custom header to activate some
functionality in SORL, and another - to activate another type of
functionality. Ideally, I think the API should be:
solrServer.request(request, headers); // where headers is = final
Map<String, String> headers
Or if that's to specific:
solrServer.request(request, new HttpGet(...));
Now - I can definitely see how that may or may not be OK with the direction
of the API - which is why I was just thinking of extending the
HttpSolrServer and adding that method for my own version. And that is where
I ran into the problems with extensibility.
I think that - without going into adding overloading methods - a quick
change would be to carefully open some of the internal details up so that
they can be used by an extending class - either by making them protected,
or by adding in getters. Since setters do exist - I also think getters
won't add to much conceptual load or moving parts to the logic.
Cheers,
Eugen.


On Thu, Nov 21, 2013 at 12:49 AM, Shawn Heisey <so...@elyograg.org> wrote:

> On 11/20/2013 3:28 PM, Eugen Paraschiv wrote:
>
>> The reason I needed access to internal details of the class - and it's not
>> just these 2 fields (I used these just as a quick example) - was that I
>> was
>> trying to extend the class and overload the request method. As soon as I
>> tried to do that, I noticed that I really couldn't easily do so - because
>> much of the fields has no getters and were not protected either (as you
>> pointed out).
>> So - it's not specifically about this to particular fields - it's more
>> about the overall extensibility of the class.
>> The class is very closed off in terms of the API - so it stands to reason
>> that it may be extended for some specific usecases (for example I am
>> trying
>> to allow setting custom HTTP Headers on the GET request before sending it)
>> - this is mainly why I was asking if it would make sense to try to open it
>> up a little and make it more extensible.
>>
>
> That makes perfect sense.  As Mark suggested, please file an issue in
> Jira.  We can then figure out on the back end exactly what to do.
>
> What kind of HTTP headers are you wanting to send?  SolrJ should already
> send all the headers that Solr requires.  If there's a compelling general
> use-case, we might want a Jira issue that makes it possible to define
> custom headers for all SolrServer implementations.
>
> Thanks,
> Shawn
>
>

Re: Extensibility of HttpSolrServer

[Shawn Heisey <so...@elyograg.org>]

On 11/20/2013 3:28 PM, Eugen Paraschiv wrote:
> The reason I needed access to internal details of the class - and it's not
> just these 2 fields (I used these just as a quick example) - was that I was
> trying to extend the class and overload the request method. As soon as I
> tried to do that, I noticed that I really couldn't easily do so - because
> much of the fields has no getters and were not protected either (as you
> pointed out).
> So - it's not specifically about this to particular fields - it's more
> about the overall extensibility of the class.
> The class is very closed off in terms of the API - so it stands to reason
> that it may be extended for some specific usecases (for example I am trying
> to allow setting custom HTTP Headers on the GET request before sending it)
> - this is mainly why I was asking if it would make sense to try to open it
> up a little and make it more extensible.

That makes perfect sense.  As Mark suggested, please file an issue in 
Jira.  We can then figure out on the back end exactly what to do.

What kind of HTTP headers are you wanting to send?  SolrJ should already 
send all the headers that Solr requires.  If there's a compelling 
general use-case, we might want a Jira issue that makes it possible to 
define custom headers for all SolrServer implementations.

Thanks,
Shawn

Re: Extensibility of HttpSolrServer

[Chris Hostetter <ho...@fucit.org>]

: The reason I needed access to internal details of the class - and it's not
: just these 2 fields (I used these just as a quick example) - was that I was
: trying to extend the class and overload the request method. As soon as I
: tried to do that, I noticed that I really couldn't easily do so - because
: much of the fields has no getters and were not protected either (as you
: pointed out).

In a lot of cases, this is generally intentional to help reduce the 
surface area of the API.  The less you can do in a subclass, the more 
flexibility there is in modifying the internals of a base class, and the 
less likeley that changes to the base class break your subclass.

IIRC the key touch point for clients ot customize HttpSolrServer is by 
being able to specifying an arbitrary HttpClient that the HttpSolrServer 
uses for all of it's communication.  By specifying your own HttpClient, 
you can do just about anything you might want, and i'm hard pressed to 
think of anything we'd want to let clients do that *can't* be done by 
specifying an arbitrary HttpClient.

: that it may be extended for some specific usecases (for example I am trying
: to allow setting custom HTTP Headers on the GET request before sending it)

This is pretty easy at the HttpClient level using RequestWrapper arround 
any GET request you are asked to execute.


-Hoss

Re: Extensibility of HttpSolrServer

[Eugen Paraschiv <eu...@odesk.com>]

The reason I needed access to internal details of the class - and it's not
just these 2 fields (I used these just as a quick example) - was that I was
trying to extend the class and overload the request method. As soon as I
tried to do that, I noticed that I really couldn't easily do so - because
much of the fields has no getters and were not protected either (as you
pointed out).
So - it's not specifically about this to particular fields - it's more
about the overall extensibility of the class.
The class is very closed off in terms of the API - so it stands to reason
that it may be extended for some specific usecases (for example I am trying
to allow setting custom HTTP Headers on the GET request before sending it)
- this is mainly why I was asking if it would make sense to try to open it
up a little and make it more extensible.
Hope that makes sense.
Cheers,
Eugen.


On Thu, Nov 21, 2013 at 12:21 AM, Shawn Heisey <so...@elyograg.org> wrote:

> On 11/20/2013 2:21 PM, Eugen Paraschiv wrote:
>
>> Quick question about the HttpSolrServer implementation - I would like to
>> extend some of the functionality of this class - but when I extend it -
>> I'm
>> having issues with how extensible it is.
>> For example - some of the details are not visible externally - setters
>> exist for maxRetries and followRedirects but no getters.
>> It would really help to make this class a bit more extensible - I'm sure
>> it
>> usually enough, but when it does need to be extended - it would make sense
>> to allow that rather than the client implement alternative version of it
>> via copy-paste (which looks like the only option available right now).
>>
>
> The specific examples that you have given are things that typically get
> set when the object is created and neverget changed.  If you really need
> access to maxRetries or other things that are private but have no getter,
> you can implement local fields, pass-through setters, and the getters you
> want.At the URL below, you can see an implementation that exposes a getter
> for maxRetries and has no warnings.  This will allow you to expose internal
> details that are not available upstream:
>
> http://apaste.info/70Jh
>
> For followRedirects, this is a parameter for the HttpClient.  There is a
> getter for the HttpClient, and with that, you can look up and change just
> about anything you want.
>
> A question for java experts and committers with a lot of experience ...
> are there compelling reasons to keep so many details in HttpSolrServer
> private rather than protected?  Should getters be implemented for all
> private fields?
>
> Thanks,
> Shawn
>
>

Re: Extensibility of HttpSolrServer

[Shawn Heisey <so...@elyograg.org>]

On 11/20/2013 2:21 PM, Eugen Paraschiv wrote:
> Quick question about the HttpSolrServer implementation - I would like to
> extend some of the functionality of this class - but when I extend it - I'm
> having issues with how extensible it is.
> For example - some of the details are not visible externally - setters
> exist for maxRetries and followRedirects but no getters.
> It would really help to make this class a bit more extensible - I'm sure it
> usually enough, but when it does need to be extended - it would make sense
> to allow that rather than the client implement alternative version of it
> via copy-paste (which looks like the only option available right now).

The specific examples that you have given are things that typically get 
set when the object is created and neverget changed.  If you really need 
access to maxRetries or other things that are private but have no 
getter, you can implement local fields, pass-through setters, and the 
getters you want.At the URL below, you can see an implementation that 
exposes a getter for maxRetries and has no warnings.  This will allow 
you to expose internal details that are not available upstream:

http://apaste.info/70Jh

For followRedirects, this is a parameter for the HttpClient.  There is a 
getter for the HttpClient, and with that, you can look up and change 
just about anything you want.

A question for java experts and committers with a lot of experience ... 
are there compelling reasons to keep so many details in HttpSolrServer 
private rather than protected?  Should getters be implemented for all 
private fields?

Thanks,
Shawn