custom request handler

[Lee Carroll <le...@googlemail.com>]

Hi we are extending SearchHandler to provide a custom search request
handler. Basically we've added NamedLists called allowed , whiteList,
maxMinList etc.

These look like the default, append and invariant namedLists in the
standard search handler config. In handleRequestBody we then remove params
not listed in the allowed named list, white list values as per the white
list and so on.

The idea is to have a "safe" request handler which the big bad world could
be exposed to. I'm worried. What have we missed that a front end app could
give us ?

Also removing params in SolrParams is a bit clunky. We are basically
converting SolrParams into NamedList processing a new NamedList from this
and then .setParams(SolrParams.toSolrParams(nlNew)) Is their a better way?
In particular namedLists are not set up for key look ups...

Anyway basically is having a custom request handler doing the above the way
to go ?

Cheers

Re: custom request handler

[Amit Nithian <an...@gmail.com>]

Hi Lee,

So the query component would be a subclass of SearchComponent and you can
define the list of components executed during a search handler.
http://wiki.apache.org/solr/SearchComponent

I *think* you can have a custom component do what you want as long as it's
the first component in the list so you can inspect and re-set the
parameters before it goes downstream to the other components. However, it's
still not clear how you are going to prevent users from POSTing bad queries
or looking at things they probably shouldn't be like the schema.xml or
solrconfig.xml or the admin console. Maybe there are ways in Solr to
prevent this but then you'd have to allow it for internal admins but
exclude it for the "public".

If you are exposing your slaves to the actual world wide public then I'd
strongly suggest an app layer between solr and the public. I treat Solr
like my database meaning that I don't expose access to my database publicly
but rather through some app layer (say some CMS tools or what not).

HTH!
Amit


On Sun, Nov 11, 2012 at 5:23 AM, Lee Carroll
<le...@googlemail.com>wrote:

> Only slaves are public facing and they are read only, with limited query
> request handlers defined. The above approach is to prevent abusive / in
> appropriate queries by clients. A query component sounds interesting would
> this be implemented through an interface so could be separate from solr or
> would it be sub classing a base component ?
>
> cheers lee c
>
>
> On 9 November 2012 17:24, Amit Nithian <an...@gmail.com> wrote:
>
> > Lee,
> >
> > I guess my question was if you are trying to prevent the "big bad world"
> > from doing stuff they aren't supposed to in Solr, how are you going to
> > prevent the big bad world from POSTing a "delete all" query? Or restrict
> > them from hitting the admin console, looking at the schema.xml,
> > solrconfig.xml.
> >
> > I guess the question here is who is the "big bad world"? The internet at
> > large or employees/colleagues in your organization? If it's the internet
> at
> > large then I'd totally decouple this from Solr b/c I want to be 100% sure
> > that the *only* thing that the internet has access to is a GET on /select
> > with some restrictions and this could be done in many places but it's not
> > clear that coupling this to Solr is the place to do it.
> >
> > If the "big bad world" is just within your organization and you want some
> > basic protections around what they can and can't see then what you are
> > saying is reasonable to me. Also perhaps another option is to consider a
> > query component rather than creating a sublcass of the request handler
> as a
> > query component promotes more re-use and flexibility. You could make the
> > necessary parameter changes in the prepare() method and just make sure
> that
> > this "safe parameter" component comes before the query component in the
> > list of components for a handler and you should be fine.
> >
> > Cheers!
> > Amit
> >
> >
> > On Fri, Nov 9, 2012 at 5:39 AM, Lee Carroll <
> lee.a.carroll@googlemail.com
> > >wrote:
> >
> > > Hi Amit
> > >
> > > I did not do this via a servlet filter as I wanted the solr devs to be
> > > concerned with solr config and keep them out of any concerns of the
> > > container. By specifying declarative data in a request handler that
> would
> > > be enough to produce a service uri for an application.
> > >
> > > Or have  I missed a point ? We have several cores with several apps all
> > > with different data query needs. Maybe 20 request handlers needed to
> > > support this with active development on going. Basically I want it easy
> > for
> > > devs to create a specific request handler suited to their needs. I
> > thought
> > > a servletfilter developed and mainatined every time would be over kill.
> > > Again though I may have missed a point / over emphasised a difficulty?
> > >
> > > Are you saying my custom request handler is to tightly bound to solr?
> so
> > > the parameters my apps talk is not de-coupled enough from solr?
> > >
> > > Lee C
> > >
> > > On 7 November 2012 19:49, Amit Nithian <an...@gmail.com> wrote:
> > >
> > > > Why not do this in a ServletFilter? Alternatively, I'd just write a
> > front
> > > > end application servlet to do this so that you don't firewall your
> > > internal
> > > > admins off from accessing the core Solr admin pages. I guess you
> could
> > > > solve this using some form of security but I don't know this well
> > enough.
> > > >
> > > > If I were to restrict access to certain parts of Solr, I'd do this
> > > outside
> > > > of Solr itself and do this in a servlet or a filter, inspecting the
> > > > parameters. It's easy to create a "modifiable" parameters class and
> > > > populate that with acceptable parameters before the Solr filter
> > operates
> > > on
> > > > it.
> > > >
> > > > HTH
> > > > Amit
> > > >
> > > >
> > > >
> > > >
> > >
> >
>

Re: custom request handler

[Lee Carroll <le...@googlemail.com>]

Only slaves are public facing and they are read only, with limited query
request handlers defined. The above approach is to prevent abusive / in
appropriate queries by clients. A query component sounds interesting would
this be implemented through an interface so could be separate from solr or
would it be sub classing a base component ?

cheers lee c


On 9 November 2012 17:24, Amit Nithian <an...@gmail.com> wrote:

> Lee,
>
> I guess my question was if you are trying to prevent the "big bad world"
> from doing stuff they aren't supposed to in Solr, how are you going to
> prevent the big bad world from POSTing a "delete all" query? Or restrict
> them from hitting the admin console, looking at the schema.xml,
> solrconfig.xml.
>
> I guess the question here is who is the "big bad world"? The internet at
> large or employees/colleagues in your organization? If it's the internet at
> large then I'd totally decouple this from Solr b/c I want to be 100% sure
> that the *only* thing that the internet has access to is a GET on /select
> with some restrictions and this could be done in many places but it's not
> clear that coupling this to Solr is the place to do it.
>
> If the "big bad world" is just within your organization and you want some
> basic protections around what they can and can't see then what you are
> saying is reasonable to me. Also perhaps another option is to consider a
> query component rather than creating a sublcass of the request handler as a
> query component promotes more re-use and flexibility. You could make the
> necessary parameter changes in the prepare() method and just make sure that
> this "safe parameter" component comes before the query component in the
> list of components for a handler and you should be fine.
>
> Cheers!
> Amit
>
>
> On Fri, Nov 9, 2012 at 5:39 AM, Lee Carroll <lee.a.carroll@googlemail.com
> >wrote:
>
> > Hi Amit
> >
> > I did not do this via a servlet filter as I wanted the solr devs to be
> > concerned with solr config and keep them out of any concerns of the
> > container. By specifying declarative data in a request handler that would
> > be enough to produce a service uri for an application.
> >
> > Or have  I missed a point ? We have several cores with several apps all
> > with different data query needs. Maybe 20 request handlers needed to
> > support this with active development on going. Basically I want it easy
> for
> > devs to create a specific request handler suited to their needs. I
> thought
> > a servletfilter developed and mainatined every time would be over kill.
> > Again though I may have missed a point / over emphasised a difficulty?
> >
> > Are you saying my custom request handler is to tightly bound to solr? so
> > the parameters my apps talk is not de-coupled enough from solr?
> >
> > Lee C
> >
> > On 7 November 2012 19:49, Amit Nithian <an...@gmail.com> wrote:
> >
> > > Why not do this in a ServletFilter? Alternatively, I'd just write a
> front
> > > end application servlet to do this so that you don't firewall your
> > internal
> > > admins off from accessing the core Solr admin pages. I guess you could
> > > solve this using some form of security but I don't know this well
> enough.
> > >
> > > If I were to restrict access to certain parts of Solr, I'd do this
> > outside
> > > of Solr itself and do this in a servlet or a filter, inspecting the
> > > parameters. It's easy to create a "modifiable" parameters class and
> > > populate that with acceptable parameters before the Solr filter
> operates
> > on
> > > it.
> > >
> > > HTH
> > > Amit
> > >
> > >
> > >
> > >
> >
>

Re: custom request handler

[Amit Nithian <an...@gmail.com>]

Lee,

I guess my question was if you are trying to prevent the "big bad world"
from doing stuff they aren't supposed to in Solr, how are you going to
prevent the big bad world from POSTing a "delete all" query? Or restrict
them from hitting the admin console, looking at the schema.xml,
solrconfig.xml.

I guess the question here is who is the "big bad world"? The internet at
large or employees/colleagues in your organization? If it's the internet at
large then I'd totally decouple this from Solr b/c I want to be 100% sure
that the *only* thing that the internet has access to is a GET on /select
with some restrictions and this could be done in many places but it's not
clear that coupling this to Solr is the place to do it.

If the "big bad world" is just within your organization and you want some
basic protections around what they can and can't see then what you are
saying is reasonable to me. Also perhaps another option is to consider a
query component rather than creating a sublcass of the request handler as a
query component promotes more re-use and flexibility. You could make the
necessary parameter changes in the prepare() method and just make sure that
this "safe parameter" component comes before the query component in the
list of components for a handler and you should be fine.

Cheers!
Amit


On Fri, Nov 9, 2012 at 5:39 AM, Lee Carroll <le...@googlemail.com>wrote:

> Hi Amit
>
> I did not do this via a servlet filter as I wanted the solr devs to be
> concerned with solr config and keep them out of any concerns of the
> container. By specifying declarative data in a request handler that would
> be enough to produce a service uri for an application.
>
> Or have  I missed a point ? We have several cores with several apps all
> with different data query needs. Maybe 20 request handlers needed to
> support this with active development on going. Basically I want it easy for
> devs to create a specific request handler suited to their needs. I thought
> a servletfilter developed and mainatined every time would be over kill.
> Again though I may have missed a point / over emphasised a difficulty?
>
> Are you saying my custom request handler is to tightly bound to solr? so
> the parameters my apps talk is not de-coupled enough from solr?
>
> Lee C
>
> On 7 November 2012 19:49, Amit Nithian <an...@gmail.com> wrote:
>
> > Why not do this in a ServletFilter? Alternatively, I'd just write a front
> > end application servlet to do this so that you don't firewall your
> internal
> > admins off from accessing the core Solr admin pages. I guess you could
> > solve this using some form of security but I don't know this well enough.
> >
> > If I were to restrict access to certain parts of Solr, I'd do this
> outside
> > of Solr itself and do this in a servlet or a filter, inspecting the
> > parameters. It's easy to create a "modifiable" parameters class and
> > populate that with acceptable parameters before the Solr filter operates
> on
> > it.
> >
> > HTH
> > Amit
> >
> >
> >
> >
>

Re: custom request handler

[Lee Carroll <le...@googlemail.com>]

Hi Amit

I did not do this via a servlet filter as I wanted the solr devs to be
concerned with solr config and keep them out of any concerns of the
container. By specifying declarative data in a request handler that would
be enough to produce a service uri for an application.

Or have  I missed a point ? We have several cores with several apps all
with different data query needs. Maybe 20 request handlers needed to
support this with active development on going. Basically I want it easy for
devs to create a specific request handler suited to their needs. I thought
a servletfilter developed and mainatined every time would be over kill.
Again though I may have missed a point / over emphasised a difficulty?

Are you saying my custom request handler is to tightly bound to solr? so
the parameters my apps talk is not de-coupled enough from solr?

Lee C

On 7 November 2012 19:49, Amit Nithian <an...@gmail.com> wrote:

> Why not do this in a ServletFilter? Alternatively, I'd just write a front
> end application servlet to do this so that you don't firewall your internal
> admins off from accessing the core Solr admin pages. I guess you could
> solve this using some form of security but I don't know this well enough.
>
> If I were to restrict access to certain parts of Solr, I'd do this outside
> of Solr itself and do this in a servlet or a filter, inspecting the
> parameters. It's easy to create a "modifiable" parameters class and
> populate that with acceptable parameters before the Solr filter operates on
> it.
>
> HTH
> Amit
>
>
>
>

Re: custom request handler

[Amit Nithian <an...@gmail.com>]

Why not do this in a ServletFilter? Alternatively, I'd just write a front
end application servlet to do this so that you don't firewall your internal
admins off from accessing the core Solr admin pages. I guess you could
solve this using some form of security but I don't know this well enough.

If I were to restrict access to certain parts of Solr, I'd do this outside
of Solr itself and do this in a servlet or a filter, inspecting the
parameters. It's easy to create a "modifiable" parameters class and
populate that with acceptable parameters before the Solr filter operates on
it.

HTH
Amit


On Tue, Nov 6, 2012 at 6:46 AM, Lee Carroll <le...@googlemail.com>wrote:

> Hi we are extending SearchHandler to provide a custom search request
> handler. Basically we've added NamedLists called allowed , whiteList,
> maxMinList etc.
>
> These look like the default, append and invariant namedLists in the
> standard search handler config. In handleRequestBody we then remove params
> not listed in the allowed named list, white list values as per the white
> list and so on.
>
> The idea is to have a "safe" request handler which the big bad world could
> be exposed to. I'm worried. What have we missed that a front end app could
> give us ?
>
> Also removing params in SolrParams is a bit clunky. We are basically
> converting SolrParams into NamedList processing a new NamedList from this
> and then .setParams(SolrParams.toSolrParams(nlNew)) Is their a better way?
> In particular namedLists are not set up for key look ups...
>
> Anyway basically is having a custom request handler doing the above the way
> to go ?
>
> Cheers
>