You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Eric Dain <er...@gmail.com> on 2015/12/25 20:17:03 UTC
AJAX access to Solr Server
Hi all,
Does allowing javascript direct access to SolrCloud raise security concern?
should I build a REST service in between?
I need to provide async search capability to web pages. the pages will be
public with no authentication.
Happy searching,
Eric
Re: AJAX access to Solr Server
Posted by GW <th...@gmail.com>.
I would put in a basic iptables statement to allow only your webserver to
prevent
http://172.16.0.22:8983/solr/products/update?stream.body=%3Cdelete%3E%3Cquery%3E*:*%3C/query%3E%3C/delete%3E&commit=true
On 25 December 2015 at 14:58, Eric Dain <er...@gmail.com> wrote:
> Thanks, that is very helpful.
>
> Have you tried denying access to some fields in the documents?
>
> On Fri, Dec 25, 2015 at 11:31 AM, Doug Turnbull <
> dturnbull@opensourceconnections.com> wrote:
>
> > We do this all the time, whitelisting only the readonly search end points
> > we want to support and disallowing excessively large paging.
> >
> > Here is a template for an nginx solr proxy. The read me describes more of
> > our philosophy
> >
> > https://github.com/o19s/solr_nginx
> >
> > On Friday, December 25, 2015, Eric Dain <er...@gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > Does allowing javascript direct access to SolrCloud raise security
> > concern?
> > > should I build a REST service in between?
> > >
> > > I need to provide async search capability to web pages. the pages will
> be
> > > public with no authentication.
> > >
> > > Happy searching,
> > > Eric
> > >
> >
> >
> > --
> > *Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
> > <http://opensourceconnections.com>, LLC | 240.476.9983
> > Author: Relevant Search <http://manning.com/turnbull>
> > This e-mail and all contents, including attachments, is considered to be
> > Company Confidential unless explicitly stated otherwise, regardless
> > of whether attachments are marked as such.
> >
>
Re: AJAX access to Solr Server
Posted by Eric Dain <er...@gmail.com>.
Thanks, that is very helpful.
Have you tried denying access to some fields in the documents?
On Fri, Dec 25, 2015 at 11:31 AM, Doug Turnbull <
dturnbull@opensourceconnections.com> wrote:
> We do this all the time, whitelisting only the readonly search end points
> we want to support and disallowing excessively large paging.
>
> Here is a template for an nginx solr proxy. The read me describes more of
> our philosophy
>
> https://github.com/o19s/solr_nginx
>
> On Friday, December 25, 2015, Eric Dain <er...@gmail.com> wrote:
>
> > Hi all,
> >
> > Does allowing javascript direct access to SolrCloud raise security
> concern?
> > should I build a REST service in between?
> >
> > I need to provide async search capability to web pages. the pages will be
> > public with no authentication.
> >
> > Happy searching,
> > Eric
> >
>
>
> --
> *Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
> <http://opensourceconnections.com>, LLC | 240.476.9983
> Author: Relevant Search <http://manning.com/turnbull>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless
> of whether attachments are marked as such.
>
Re: AJAX access to Solr Server
Posted by Doug Turnbull <dt...@opensourceconnections.com>.
We do this all the time, whitelisting only the readonly search end points
we want to support and disallowing excessively large paging.
Here is a template for an nginx solr proxy. The read me describes more of
our philosophy
https://github.com/o19s/solr_nginx
On Friday, December 25, 2015, Eric Dain <er...@gmail.com> wrote:
> Hi all,
>
> Does allowing javascript direct access to SolrCloud raise security concern?
> should I build a REST service in between?
>
> I need to provide async search capability to web pages. the pages will be
> public with no authentication.
>
> Happy searching,
> Eric
>
--
*Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
<http://opensourceconnections.com>, LLC | 240.476.9983
Author: Relevant Search <http://manning.com/turnbull>
This e-mail and all contents, including attachments, is considered to be
Company Confidential unless explicitly stated otherwise, regardless
of whether attachments are marked as such.
Re: AJAX access to Solr Server
Posted by GW <th...@gmail.com>.
What are you using for a client?
I generally use a REST client written in PHP or Perl and then prevent cross
scripting so only the client can do the work.
My Solr cluster is running behind OpenVPN on 172.16.0.0/24
I use a jquery in the following to get an infinite scroll
http://www.frogshopping.com
cross scripting work not in place yet.
On 26 December 2015 at 09:59, Doug Turnbull <
dturnbull@opensourceconnections.com> wrote:
> True though you could also query an API in front of Solr to a stand still
> pretty easily. DoSing is a pretty easy thing to do to anything that needs
> to be open to the public.
>
> The biggest issue with the proxy approach is an attacker with Solr
> knowledge that doesn't need to DoS, just send a handful of really slow
> queries to Solr. This is something that can be mitigated, but the more
> skilled the attacker the more interesting the slow queries get.
>
> I should note this is a problem to mitigate with any system that handles
> user queries by passing then directly to edismax. Solr sort of encourages
> you to talk straight to edismax, and most systems don't prepare or escape
> the query. Instead they want to support the full range of query operations.
> An attacker can still put nasty function queries in the query box enough
> times to make a Solr server crawl.
>
> Doug
>
> On Saturday, December 26, 2015, GW <th...@gmail.com> wrote:
>
> > Yes, your proxy seems to work.
> >
> > The only thing that bothers me is anyone can query your Solr
> installation.
> >
> > The world is not a nice place and I can't tell you how many DOS attacks
> > I've fended off in the last 30 years.
> >
> > If I thought you were an a-hole I could set up a few machines and query
> > your server to a standstill.
> >
> > About ten years ago I was working on a contract. The competitor that lost
> > the bid did a email DOS attack on me after they took out a whole bunch
> car
> > adds (hot deals) in the local paper. My email was f###'d and my phone
> was
> > ringing off the hook.
> >
> > Cheers,
> >
> > GW
> >
> >
> > On 25 December 2015 at 21:55, Doug Turnbull <
> > dturnbull@opensourceconnections.com <javascript:;>> wrote:
> >
> > > Yeah I prefer a whitelist of locked down query request handlers via a
> > > proxy that are reasonably well protected. I would never expose update
> to
> > > the web or allow any updating over a public interface.
> > >
> > > If you want an example, you can checkout
> > >
> > >
> > >
> >
> http://solr.quepid.com/solr/statedecoded/select?q=*:*&qt=update&stream.body=
> > > <delete><query>*:*</query></delete>&commit=true
> > >
> > > http://solr.quepid.com/solr/statedecoded/update?stream.body=
> > > <delete><query>*:*</query></delete>&commit=true
> > >
> > > But still get search results back:
> > > http://solr.quepid.com/solr/statedecoded/select?q=*:*
> > >
> > > Click all those all day long. And do let me know if you find holes...
> I'm
> > > sure there's room for improvement
> > >
> > > Cheers,
> > > -Doug
> > >
> > > On Friday, December 25, 2015, GW <thegeoforce@gmail.com
> <javascript:;>>
> > wrote:
> > >
> > > > If you are using Linux a simple one liner in IP tables
> > > >
> > > > iptables -I INPUT \! --src www.yourwebserver.com -m tcp -p tcp
> --dport
> > > > 8983 -j DROP
> > > >
> > > >
> > > > If windows, you can do something similar
> > > >
> > > > otherwise it is very easy for anyone to delete all your documents
> with
> > > >
> > > > http://yoursolrserver.com:8983/solr/your-core/update?stream.body=
> > > > <delete><query>*:*</query></delete>&commit=true
> > > >
> > > >
> > > >
> > > >
> > > > On 25 December 2015 at 20:42, Doug Turnbull <
> > > > dturnbull@opensourceconnections.com <javascript:;> <javascript:;>>
> > wrote:
> > > >
> > > > > Hi Shawn
> > > > >
> > > > > Maybe I should have qualified the parameters of scenarios this make
> > me
> > > > > comfortable just proxying Solr directly w/o an API
> > > > >
> > > > > These situations include:
> > > > >
> > > > > 1. I've got no qualms about giving the whole world access to every
> > > > document
> > > > > in the index. There's nothing protected about anything.
> > > > > 2. The content can be easily rebuilt , it's not very large. (I can
> > > easily
> > > > > push a button and make a new one)
> > > > >
> > > > > Sure you can denial of service Solr, and I might lose my search
> > index.
> > > > But
> > > > > you can denial of service anything. This includes just about
> anything
> > > you
> > > > > put in front of Solr. Moreover, the added complexity of a
> > > > > Drupal/Wordpress/your API might only add to the security problems
> > with
> > > > > their own security issues. I'd rather keep it simple and have fewer
> > > > moving
> > > > > parts.
> > > > >
> > > > > Cases where I would want an API in front of Solr (these are just
> the
> > > > > security ones):
> > > > > - I want to protect the content (ie based on some notion of a
> "user"
> > or
> > > > > other permissions)
> > > > > - Rebuilding the content would be very hard and time consuming
> > > > >
> > > > > I would also say to expose Solr directly to everyone you probably
> > > should
> > > > > know about Solr's bugaboos:
> > > > > - the lovely qt parameter and the request dispatcher (the nginx
> proxy
> > > > below
> > > > > disallows qt)
> > > > > - deep paging (prevented by the nginx proxy)
> > > > > - how to lock down a request handler fairly robustly, how to use
> > > > invariants
> > > > > - mitigating intentionally malicious queries (such as the lovely
> > > "sleep"
> > > > > function query).
> > > > >
> > > > > I'm also curious to hear what the websolr people do, or anyone else
> > > that
> > > > > hosts Solr for the JavaScript app development crowd.
> > > > >
> > > > > Cheers
> > > > > -Doug
> > > > >
> > > > >
> > > > > On Friday, December 25, 2015, Shawn Heisey <apache@elyograg.org
> > <javascript:;>
> > > > <javascript:;>> wrote:
> > > > >
> > > > > > On 12/25/2015 12:17 PM, Eric Dain wrote:
> > > > > > > Does allowing javascript direct access to SolrCloud raise
> > security
> > > > > > concern?
> > > > > > > should I build a REST service in between?
> > > > > > >
> > > > > > > I need to provide async search capability to web pages. the
> pages
> > > > will
> > > > > be
> > > > > > > public with no authentication.
> > > > > >
> > > > > > End users should never have access to Solr. Access to Solr from
> > the
> > > > > > end-user machine is required if you want to accept Solr responses
> > > > > directly.
> > > > > >
> > > > > > In one of the other replies that you received, Doug has given you
> > an
> > > > > > nginx config for proxying access to Solr -- indirect access.
> This
> > > can
> > > > > > protect against *changes* to the index, and it has protection
> > against
> > > > > > high start/rows values, but there are many other ways that an
> > > attacker
> > > > > > can construct denial of service queries, which this proxy config
> > will
> > > > > > not prevent.
> > > > > >
> > > > > > I think that indirect access (through a proxy) should not be
> > allowed
> > > > > > either, unless you can trust all the people that will have
> access.
> > > > > >
> > > > > > If Solr is open to a sufficiently wide audience (especially the
> > > > > > Internet), someone will find a way to abuse the service even
> with a
> > > > > > proxy, either to cause harm or to learn things they shouldn't
> know.
> > > > > >
> > > > > > The most secure option is to only allow the webservers and
> trusted
> > > > > > administrators to access Solr. All end user (Internet) access to
> > > Solr
> > > > > > should be handled through a custom web application. This might
> be
> > > > > > something that you find and install (such as wordpress, drupal,
> > etc),
> > > > or
> > > > > > one that you write yourself.
> > > > > >
> > > > > > You can still do AJAX while maintaining security. You'll need to
> > > write
> > > > > > something in a server-side web programming language like PHP,
> Java,
> > > > etc.
> > > > > > This code will need to accept the AJAX requests from your
> > > client-side
> > > > > > javascript code, validate the request parameters to make sure
> > they're
> > > > > > sane, get a response from Solr, and return relevant data. If the
> > > > > > parameters don't validate, return an error, and handle that error
> > > > > > appropriately in the javascript code.
> > > > > >
> > > > > > Thanks,
> > > > > > Shawn
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > *Doug Turnbull **| *Search Relevance Consultant | OpenSource
> > > Connections
> > > > > <http://opensourceconnections.com>, LLC | 240.476.9983
> > > > > Author: Relevant Search <http://manning.com/turnbull>
> > > > > This e-mail and all contents, including attachments, is considered
> to
> > > be
> > > > > Company Confidential unless explicitly stated otherwise, regardless
> > > > > of whether attachments are marked as such.
> > > > >
> > > >
> > >
> > >
> > > --
> > > *Doug Turnbull **| *Search Relevance Consultant | OpenSource
> Connections
> > > <http://opensourceconnections.com>, LLC | 240.476.9983
> > > Author: Relevant Search <http://manning.com/turnbull>
> > > This e-mail and all contents, including attachments, is considered to
> be
> > > Company Confidential unless explicitly stated otherwise, regardless
> > > of whether attachments are marked as such.
> > >
> >
>
>
> --
> *Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
> <http://opensourceconnections.com>, LLC | 240.476.9983
> Author: Relevant Search <http://manning.com/turnbull>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless
> of whether attachments are marked as such.
>
Re: AJAX access to Solr Server
Posted by Doug Turnbull <dt...@opensourceconnections.com>.
True though you could also query an API in front of Solr to a stand still
pretty easily. DoSing is a pretty easy thing to do to anything that needs
to be open to the public.
The biggest issue with the proxy approach is an attacker with Solr
knowledge that doesn't need to DoS, just send a handful of really slow
queries to Solr. This is something that can be mitigated, but the more
skilled the attacker the more interesting the slow queries get.
I should note this is a problem to mitigate with any system that handles
user queries by passing then directly to edismax. Solr sort of encourages
you to talk straight to edismax, and most systems don't prepare or escape
the query. Instead they want to support the full range of query operations.
An attacker can still put nasty function queries in the query box enough
times to make a Solr server crawl.
Doug
On Saturday, December 26, 2015, GW <th...@gmail.com> wrote:
> Yes, your proxy seems to work.
>
> The only thing that bothers me is anyone can query your Solr installation.
>
> The world is not a nice place and I can't tell you how many DOS attacks
> I've fended off in the last 30 years.
>
> If I thought you were an a-hole I could set up a few machines and query
> your server to a standstill.
>
> About ten years ago I was working on a contract. The competitor that lost
> the bid did a email DOS attack on me after they took out a whole bunch car
> adds (hot deals) in the local paper. My email was f###'d and my phone was
> ringing off the hook.
>
> Cheers,
>
> GW
>
>
> On 25 December 2015 at 21:55, Doug Turnbull <
> dturnbull@opensourceconnections.com <javascript:;>> wrote:
>
> > Yeah I prefer a whitelist of locked down query request handlers via a
> > proxy that are reasonably well protected. I would never expose update to
> > the web or allow any updating over a public interface.
> >
> > If you want an example, you can checkout
> >
> >
> >
> http://solr.quepid.com/solr/statedecoded/select?q=*:*&qt=update&stream.body=
> > <delete><query>*:*</query></delete>&commit=true
> >
> > http://solr.quepid.com/solr/statedecoded/update?stream.body=
> > <delete><query>*:*</query></delete>&commit=true
> >
> > But still get search results back:
> > http://solr.quepid.com/solr/statedecoded/select?q=*:*
> >
> > Click all those all day long. And do let me know if you find holes... I'm
> > sure there's room for improvement
> >
> > Cheers,
> > -Doug
> >
> > On Friday, December 25, 2015, GW <thegeoforce@gmail.com <javascript:;>>
> wrote:
> >
> > > If you are using Linux a simple one liner in IP tables
> > >
> > > iptables -I INPUT \! --src www.yourwebserver.com -m tcp -p tcp --dport
> > > 8983 -j DROP
> > >
> > >
> > > If windows, you can do something similar
> > >
> > > otherwise it is very easy for anyone to delete all your documents with
> > >
> > > http://yoursolrserver.com:8983/solr/your-core/update?stream.body=
> > > <delete><query>*:*</query></delete>&commit=true
> > >
> > >
> > >
> > >
> > > On 25 December 2015 at 20:42, Doug Turnbull <
> > > dturnbull@opensourceconnections.com <javascript:;> <javascript:;>>
> wrote:
> > >
> > > > Hi Shawn
> > > >
> > > > Maybe I should have qualified the parameters of scenarios this make
> me
> > > > comfortable just proxying Solr directly w/o an API
> > > >
> > > > These situations include:
> > > >
> > > > 1. I've got no qualms about giving the whole world access to every
> > > document
> > > > in the index. There's nothing protected about anything.
> > > > 2. The content can be easily rebuilt , it's not very large. (I can
> > easily
> > > > push a button and make a new one)
> > > >
> > > > Sure you can denial of service Solr, and I might lose my search
> index.
> > > But
> > > > you can denial of service anything. This includes just about anything
> > you
> > > > put in front of Solr. Moreover, the added complexity of a
> > > > Drupal/Wordpress/your API might only add to the security problems
> with
> > > > their own security issues. I'd rather keep it simple and have fewer
> > > moving
> > > > parts.
> > > >
> > > > Cases where I would want an API in front of Solr (these are just the
> > > > security ones):
> > > > - I want to protect the content (ie based on some notion of a "user"
> or
> > > > other permissions)
> > > > - Rebuilding the content would be very hard and time consuming
> > > >
> > > > I would also say to expose Solr directly to everyone you probably
> > should
> > > > know about Solr's bugaboos:
> > > > - the lovely qt parameter and the request dispatcher (the nginx proxy
> > > below
> > > > disallows qt)
> > > > - deep paging (prevented by the nginx proxy)
> > > > - how to lock down a request handler fairly robustly, how to use
> > > invariants
> > > > - mitigating intentionally malicious queries (such as the lovely
> > "sleep"
> > > > function query).
> > > >
> > > > I'm also curious to hear what the websolr people do, or anyone else
> > that
> > > > hosts Solr for the JavaScript app development crowd.
> > > >
> > > > Cheers
> > > > -Doug
> > > >
> > > >
> > > > On Friday, December 25, 2015, Shawn Heisey <apache@elyograg.org
> <javascript:;>
> > > <javascript:;>> wrote:
> > > >
> > > > > On 12/25/2015 12:17 PM, Eric Dain wrote:
> > > > > > Does allowing javascript direct access to SolrCloud raise
> security
> > > > > concern?
> > > > > > should I build a REST service in between?
> > > > > >
> > > > > > I need to provide async search capability to web pages. the pages
> > > will
> > > > be
> > > > > > public with no authentication.
> > > > >
> > > > > End users should never have access to Solr. Access to Solr from
> the
> > > > > end-user machine is required if you want to accept Solr responses
> > > > directly.
> > > > >
> > > > > In one of the other replies that you received, Doug has given you
> an
> > > > > nginx config for proxying access to Solr -- indirect access. This
> > can
> > > > > protect against *changes* to the index, and it has protection
> against
> > > > > high start/rows values, but there are many other ways that an
> > attacker
> > > > > can construct denial of service queries, which this proxy config
> will
> > > > > not prevent.
> > > > >
> > > > > I think that indirect access (through a proxy) should not be
> allowed
> > > > > either, unless you can trust all the people that will have access.
> > > > >
> > > > > If Solr is open to a sufficiently wide audience (especially the
> > > > > Internet), someone will find a way to abuse the service even with a
> > > > > proxy, either to cause harm or to learn things they shouldn't know.
> > > > >
> > > > > The most secure option is to only allow the webservers and trusted
> > > > > administrators to access Solr. All end user (Internet) access to
> > Solr
> > > > > should be handled through a custom web application. This might be
> > > > > something that you find and install (such as wordpress, drupal,
> etc),
> > > or
> > > > > one that you write yourself.
> > > > >
> > > > > You can still do AJAX while maintaining security. You'll need to
> > write
> > > > > something in a server-side web programming language like PHP, Java,
> > > etc.
> > > > > This code will need to accept the AJAX requests from your
> > client-side
> > > > > javascript code, validate the request parameters to make sure
> they're
> > > > > sane, get a response from Solr, and return relevant data. If the
> > > > > parameters don't validate, return an error, and handle that error
> > > > > appropriately in the javascript code.
> > > > >
> > > > > Thanks,
> > > > > Shawn
> > > > >
> > > > >
> > > >
> > > > --
> > > > *Doug Turnbull **| *Search Relevance Consultant | OpenSource
> > Connections
> > > > <http://opensourceconnections.com>, LLC | 240.476.9983
> > > > Author: Relevant Search <http://manning.com/turnbull>
> > > > This e-mail and all contents, including attachments, is considered to
> > be
> > > > Company Confidential unless explicitly stated otherwise, regardless
> > > > of whether attachments are marked as such.
> > > >
> > >
> >
> >
> > --
> > *Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
> > <http://opensourceconnections.com>, LLC | 240.476.9983
> > Author: Relevant Search <http://manning.com/turnbull>
> > This e-mail and all contents, including attachments, is considered to be
> > Company Confidential unless explicitly stated otherwise, regardless
> > of whether attachments are marked as such.
> >
>
--
*Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
<http://opensourceconnections.com>, LLC | 240.476.9983
Author: Relevant Search <http://manning.com/turnbull>
This e-mail and all contents, including attachments, is considered to be
Company Confidential unless explicitly stated otherwise, regardless
of whether attachments are marked as such.
Re: AJAX access to Solr Server
Posted by GW <th...@gmail.com>.
Yes, your proxy seems to work.
The only thing that bothers me is anyone can query your Solr installation.
The world is not a nice place and I can't tell you how many DOS attacks
I've fended off in the last 30 years.
If I thought you were an a-hole I could set up a few machines and query
your server to a standstill.
About ten years ago I was working on a contract. The competitor that lost
the bid did a email DOS attack on me after they took out a whole bunch car
adds (hot deals) in the local paper. My email was f###'d and my phone was
ringing off the hook.
Cheers,
GW
On 25 December 2015 at 21:55, Doug Turnbull <
dturnbull@opensourceconnections.com> wrote:
> Yeah I prefer a whitelist of locked down query request handlers via a
> proxy that are reasonably well protected. I would never expose update to
> the web or allow any updating over a public interface.
>
> If you want an example, you can checkout
>
>
> http://solr.quepid.com/solr/statedecoded/select?q=*:*&qt=update&stream.body=
> <delete><query>*:*</query></delete>&commit=true
>
> http://solr.quepid.com/solr/statedecoded/update?stream.body=
> <delete><query>*:*</query></delete>&commit=true
>
> But still get search results back:
> http://solr.quepid.com/solr/statedecoded/select?q=*:*
>
> Click all those all day long. And do let me know if you find holes... I'm
> sure there's room for improvement
>
> Cheers,
> -Doug
>
> On Friday, December 25, 2015, GW <th...@gmail.com> wrote:
>
> > If you are using Linux a simple one liner in IP tables
> >
> > iptables -I INPUT \! --src www.yourwebserver.com -m tcp -p tcp --dport
> > 8983 -j DROP
> >
> >
> > If windows, you can do something similar
> >
> > otherwise it is very easy for anyone to delete all your documents with
> >
> > http://yoursolrserver.com:8983/solr/your-core/update?stream.body=
> > <delete><query>*:*</query></delete>&commit=true
> >
> >
> >
> >
> > On 25 December 2015 at 20:42, Doug Turnbull <
> > dturnbull@opensourceconnections.com <javascript:;>> wrote:
> >
> > > Hi Shawn
> > >
> > > Maybe I should have qualified the parameters of scenarios this make me
> > > comfortable just proxying Solr directly w/o an API
> > >
> > > These situations include:
> > >
> > > 1. I've got no qualms about giving the whole world access to every
> > document
> > > in the index. There's nothing protected about anything.
> > > 2. The content can be easily rebuilt , it's not very large. (I can
> easily
> > > push a button and make a new one)
> > >
> > > Sure you can denial of service Solr, and I might lose my search index.
> > But
> > > you can denial of service anything. This includes just about anything
> you
> > > put in front of Solr. Moreover, the added complexity of a
> > > Drupal/Wordpress/your API might only add to the security problems with
> > > their own security issues. I'd rather keep it simple and have fewer
> > moving
> > > parts.
> > >
> > > Cases where I would want an API in front of Solr (these are just the
> > > security ones):
> > > - I want to protect the content (ie based on some notion of a "user" or
> > > other permissions)
> > > - Rebuilding the content would be very hard and time consuming
> > >
> > > I would also say to expose Solr directly to everyone you probably
> should
> > > know about Solr's bugaboos:
> > > - the lovely qt parameter and the request dispatcher (the nginx proxy
> > below
> > > disallows qt)
> > > - deep paging (prevented by the nginx proxy)
> > > - how to lock down a request handler fairly robustly, how to use
> > invariants
> > > - mitigating intentionally malicious queries (such as the lovely
> "sleep"
> > > function query).
> > >
> > > I'm also curious to hear what the websolr people do, or anyone else
> that
> > > hosts Solr for the JavaScript app development crowd.
> > >
> > > Cheers
> > > -Doug
> > >
> > >
> > > On Friday, December 25, 2015, Shawn Heisey <apache@elyograg.org
> > <javascript:;>> wrote:
> > >
> > > > On 12/25/2015 12:17 PM, Eric Dain wrote:
> > > > > Does allowing javascript direct access to SolrCloud raise security
> > > > concern?
> > > > > should I build a REST service in between?
> > > > >
> > > > > I need to provide async search capability to web pages. the pages
> > will
> > > be
> > > > > public with no authentication.
> > > >
> > > > End users should never have access to Solr. Access to Solr from the
> > > > end-user machine is required if you want to accept Solr responses
> > > directly.
> > > >
> > > > In one of the other replies that you received, Doug has given you an
> > > > nginx config for proxying access to Solr -- indirect access. This
> can
> > > > protect against *changes* to the index, and it has protection against
> > > > high start/rows values, but there are many other ways that an
> attacker
> > > > can construct denial of service queries, which this proxy config will
> > > > not prevent.
> > > >
> > > > I think that indirect access (through a proxy) should not be allowed
> > > > either, unless you can trust all the people that will have access.
> > > >
> > > > If Solr is open to a sufficiently wide audience (especially the
> > > > Internet), someone will find a way to abuse the service even with a
> > > > proxy, either to cause harm or to learn things they shouldn't know.
> > > >
> > > > The most secure option is to only allow the webservers and trusted
> > > > administrators to access Solr. All end user (Internet) access to
> Solr
> > > > should be handled through a custom web application. This might be
> > > > something that you find and install (such as wordpress, drupal, etc),
> > or
> > > > one that you write yourself.
> > > >
> > > > You can still do AJAX while maintaining security. You'll need to
> write
> > > > something in a server-side web programming language like PHP, Java,
> > etc.
> > > > This code will need to accept the AJAX requests from your
> client-side
> > > > javascript code, validate the request parameters to make sure they're
> > > > sane, get a response from Solr, and return relevant data. If the
> > > > parameters don't validate, return an error, and handle that error
> > > > appropriately in the javascript code.
> > > >
> > > > Thanks,
> > > > Shawn
> > > >
> > > >
> > >
> > > --
> > > *Doug Turnbull **| *Search Relevance Consultant | OpenSource
> Connections
> > > <http://opensourceconnections.com>, LLC | 240.476.9983
> > > Author: Relevant Search <http://manning.com/turnbull>
> > > This e-mail and all contents, including attachments, is considered to
> be
> > > Company Confidential unless explicitly stated otherwise, regardless
> > > of whether attachments are marked as such.
> > >
> >
>
>
> --
> *Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
> <http://opensourceconnections.com>, LLC | 240.476.9983
> Author: Relevant Search <http://manning.com/turnbull>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless
> of whether attachments are marked as such.
>
Re: AJAX access to Solr Server
Posted by Doug Turnbull <dt...@opensourceconnections.com>.
Yeah I prefer a whitelist of locked down query request handlers via a
proxy that are reasonably well protected. I would never expose update to
the web or allow any updating over a public interface.
If you want an example, you can checkout
http://solr.quepid.com/solr/statedecoded/select?q=*:*&qt=update&stream.body=
<delete><query>*:*</query></delete>&commit=true
http://solr.quepid.com/solr/statedecoded/update?stream.body=
<delete><query>*:*</query></delete>&commit=true
But still get search results back:
http://solr.quepid.com/solr/statedecoded/select?q=*:*
Click all those all day long. And do let me know if you find holes... I'm
sure there's room for improvement
Cheers,
-Doug
On Friday, December 25, 2015, GW <th...@gmail.com> wrote:
> If you are using Linux a simple one liner in IP tables
>
> iptables -I INPUT \! --src www.yourwebserver.com -m tcp -p tcp --dport
> 8983 -j DROP
>
>
> If windows, you can do something similar
>
> otherwise it is very easy for anyone to delete all your documents with
>
> http://yoursolrserver.com:8983/solr/your-core/update?stream.body=
> <delete><query>*:*</query></delete>&commit=true
>
>
>
>
> On 25 December 2015 at 20:42, Doug Turnbull <
> dturnbull@opensourceconnections.com <javascript:;>> wrote:
>
> > Hi Shawn
> >
> > Maybe I should have qualified the parameters of scenarios this make me
> > comfortable just proxying Solr directly w/o an API
> >
> > These situations include:
> >
> > 1. I've got no qualms about giving the whole world access to every
> document
> > in the index. There's nothing protected about anything.
> > 2. The content can be easily rebuilt , it's not very large. (I can easily
> > push a button and make a new one)
> >
> > Sure you can denial of service Solr, and I might lose my search index.
> But
> > you can denial of service anything. This includes just about anything you
> > put in front of Solr. Moreover, the added complexity of a
> > Drupal/Wordpress/your API might only add to the security problems with
> > their own security issues. I'd rather keep it simple and have fewer
> moving
> > parts.
> >
> > Cases where I would want an API in front of Solr (these are just the
> > security ones):
> > - I want to protect the content (ie based on some notion of a "user" or
> > other permissions)
> > - Rebuilding the content would be very hard and time consuming
> >
> > I would also say to expose Solr directly to everyone you probably should
> > know about Solr's bugaboos:
> > - the lovely qt parameter and the request dispatcher (the nginx proxy
> below
> > disallows qt)
> > - deep paging (prevented by the nginx proxy)
> > - how to lock down a request handler fairly robustly, how to use
> invariants
> > - mitigating intentionally malicious queries (such as the lovely "sleep"
> > function query).
> >
> > I'm also curious to hear what the websolr people do, or anyone else that
> > hosts Solr for the JavaScript app development crowd.
> >
> > Cheers
> > -Doug
> >
> >
> > On Friday, December 25, 2015, Shawn Heisey <apache@elyograg.org
> <javascript:;>> wrote:
> >
> > > On 12/25/2015 12:17 PM, Eric Dain wrote:
> > > > Does allowing javascript direct access to SolrCloud raise security
> > > concern?
> > > > should I build a REST service in between?
> > > >
> > > > I need to provide async search capability to web pages. the pages
> will
> > be
> > > > public with no authentication.
> > >
> > > End users should never have access to Solr. Access to Solr from the
> > > end-user machine is required if you want to accept Solr responses
> > directly.
> > >
> > > In one of the other replies that you received, Doug has given you an
> > > nginx config for proxying access to Solr -- indirect access. This can
> > > protect against *changes* to the index, and it has protection against
> > > high start/rows values, but there are many other ways that an attacker
> > > can construct denial of service queries, which this proxy config will
> > > not prevent.
> > >
> > > I think that indirect access (through a proxy) should not be allowed
> > > either, unless you can trust all the people that will have access.
> > >
> > > If Solr is open to a sufficiently wide audience (especially the
> > > Internet), someone will find a way to abuse the service even with a
> > > proxy, either to cause harm or to learn things they shouldn't know.
> > >
> > > The most secure option is to only allow the webservers and trusted
> > > administrators to access Solr. All end user (Internet) access to Solr
> > > should be handled through a custom web application. This might be
> > > something that you find and install (such as wordpress, drupal, etc),
> or
> > > one that you write yourself.
> > >
> > > You can still do AJAX while maintaining security. You'll need to write
> > > something in a server-side web programming language like PHP, Java,
> etc.
> > > This code will need to accept the AJAX requests from your client-side
> > > javascript code, validate the request parameters to make sure they're
> > > sane, get a response from Solr, and return relevant data. If the
> > > parameters don't validate, return an error, and handle that error
> > > appropriately in the javascript code.
> > >
> > > Thanks,
> > > Shawn
> > >
> > >
> >
> > --
> > *Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
> > <http://opensourceconnections.com>, LLC | 240.476.9983
> > Author: Relevant Search <http://manning.com/turnbull>
> > This e-mail and all contents, including attachments, is considered to be
> > Company Confidential unless explicitly stated otherwise, regardless
> > of whether attachments are marked as such.
> >
>
--
*Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
<http://opensourceconnections.com>, LLC | 240.476.9983
Author: Relevant Search <http://manning.com/turnbull>
This e-mail and all contents, including attachments, is considered to be
Company Confidential unless explicitly stated otherwise, regardless
of whether attachments are marked as such.
Re: AJAX access to Solr Server
Posted by GW <th...@gmail.com>.
If you are using Linux a simple one liner in IP tables
iptables -I INPUT \! --src www.yourwebserver.com -m tcp -p tcp --dport
8983 -j DROP
If windows, you can do something similar
otherwise it is very easy for anyone to delete all your documents with
http://yoursolrserver.com:8983/solr/your-core/update?stream.body=
<delete><query>*:*</query></delete>&commit=true
On 25 December 2015 at 20:42, Doug Turnbull <
dturnbull@opensourceconnections.com> wrote:
> Hi Shawn
>
> Maybe I should have qualified the parameters of scenarios this make me
> comfortable just proxying Solr directly w/o an API
>
> These situations include:
>
> 1. I've got no qualms about giving the whole world access to every document
> in the index. There's nothing protected about anything.
> 2. The content can be easily rebuilt , it's not very large. (I can easily
> push a button and make a new one)
>
> Sure you can denial of service Solr, and I might lose my search index. But
> you can denial of service anything. This includes just about anything you
> put in front of Solr. Moreover, the added complexity of a
> Drupal/Wordpress/your API might only add to the security problems with
> their own security issues. I'd rather keep it simple and have fewer moving
> parts.
>
> Cases where I would want an API in front of Solr (these are just the
> security ones):
> - I want to protect the content (ie based on some notion of a "user" or
> other permissions)
> - Rebuilding the content would be very hard and time consuming
>
> I would also say to expose Solr directly to everyone you probably should
> know about Solr's bugaboos:
> - the lovely qt parameter and the request dispatcher (the nginx proxy below
> disallows qt)
> - deep paging (prevented by the nginx proxy)
> - how to lock down a request handler fairly robustly, how to use invariants
> - mitigating intentionally malicious queries (such as the lovely "sleep"
> function query).
>
> I'm also curious to hear what the websolr people do, or anyone else that
> hosts Solr for the JavaScript app development crowd.
>
> Cheers
> -Doug
>
>
> On Friday, December 25, 2015, Shawn Heisey <ap...@elyograg.org> wrote:
>
> > On 12/25/2015 12:17 PM, Eric Dain wrote:
> > > Does allowing javascript direct access to SolrCloud raise security
> > concern?
> > > should I build a REST service in between?
> > >
> > > I need to provide async search capability to web pages. the pages will
> be
> > > public with no authentication.
> >
> > End users should never have access to Solr. Access to Solr from the
> > end-user machine is required if you want to accept Solr responses
> directly.
> >
> > In one of the other replies that you received, Doug has given you an
> > nginx config for proxying access to Solr -- indirect access. This can
> > protect against *changes* to the index, and it has protection against
> > high start/rows values, but there are many other ways that an attacker
> > can construct denial of service queries, which this proxy config will
> > not prevent.
> >
> > I think that indirect access (through a proxy) should not be allowed
> > either, unless you can trust all the people that will have access.
> >
> > If Solr is open to a sufficiently wide audience (especially the
> > Internet), someone will find a way to abuse the service even with a
> > proxy, either to cause harm or to learn things they shouldn't know.
> >
> > The most secure option is to only allow the webservers and trusted
> > administrators to access Solr. All end user (Internet) access to Solr
> > should be handled through a custom web application. This might be
> > something that you find and install (such as wordpress, drupal, etc), or
> > one that you write yourself.
> >
> > You can still do AJAX while maintaining security. You'll need to write
> > something in a server-side web programming language like PHP, Java, etc.
> > This code will need to accept the AJAX requests from your client-side
> > javascript code, validate the request parameters to make sure they're
> > sane, get a response from Solr, and return relevant data. If the
> > parameters don't validate, return an error, and handle that error
> > appropriately in the javascript code.
> >
> > Thanks,
> > Shawn
> >
> >
>
> --
> *Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
> <http://opensourceconnections.com>, LLC | 240.476.9983
> Author: Relevant Search <http://manning.com/turnbull>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless
> of whether attachments are marked as such.
>
Re: AJAX access to Solr Server
Posted by Doug Turnbull <dt...@opensourceconnections.com>.
Hi Shawn
Maybe I should have qualified the parameters of scenarios this make me
comfortable just proxying Solr directly w/o an API
These situations include:
1. I've got no qualms about giving the whole world access to every document
in the index. There's nothing protected about anything.
2. The content can be easily rebuilt , it's not very large. (I can easily
push a button and make a new one)
Sure you can denial of service Solr, and I might lose my search index. But
you can denial of service anything. This includes just about anything you
put in front of Solr. Moreover, the added complexity of a
Drupal/Wordpress/your API might only add to the security problems with
their own security issues. I'd rather keep it simple and have fewer moving
parts.
Cases where I would want an API in front of Solr (these are just the
security ones):
- I want to protect the content (ie based on some notion of a "user" or
other permissions)
- Rebuilding the content would be very hard and time consuming
I would also say to expose Solr directly to everyone you probably should
know about Solr's bugaboos:
- the lovely qt parameter and the request dispatcher (the nginx proxy below
disallows qt)
- deep paging (prevented by the nginx proxy)
- how to lock down a request handler fairly robustly, how to use invariants
- mitigating intentionally malicious queries (such as the lovely "sleep"
function query).
I'm also curious to hear what the websolr people do, or anyone else that
hosts Solr for the JavaScript app development crowd.
Cheers
-Doug
On Friday, December 25, 2015, Shawn Heisey <ap...@elyograg.org> wrote:
> On 12/25/2015 12:17 PM, Eric Dain wrote:
> > Does allowing javascript direct access to SolrCloud raise security
> concern?
> > should I build a REST service in between?
> >
> > I need to provide async search capability to web pages. the pages will be
> > public with no authentication.
>
> End users should never have access to Solr. Access to Solr from the
> end-user machine is required if you want to accept Solr responses directly.
>
> In one of the other replies that you received, Doug has given you an
> nginx config for proxying access to Solr -- indirect access. This can
> protect against *changes* to the index, and it has protection against
> high start/rows values, but there are many other ways that an attacker
> can construct denial of service queries, which this proxy config will
> not prevent.
>
> I think that indirect access (through a proxy) should not be allowed
> either, unless you can trust all the people that will have access.
>
> If Solr is open to a sufficiently wide audience (especially the
> Internet), someone will find a way to abuse the service even with a
> proxy, either to cause harm or to learn things they shouldn't know.
>
> The most secure option is to only allow the webservers and trusted
> administrators to access Solr. All end user (Internet) access to Solr
> should be handled through a custom web application. This might be
> something that you find and install (such as wordpress, drupal, etc), or
> one that you write yourself.
>
> You can still do AJAX while maintaining security. You'll need to write
> something in a server-side web programming language like PHP, Java, etc.
> This code will need to accept the AJAX requests from your client-side
> javascript code, validate the request parameters to make sure they're
> sane, get a response from Solr, and return relevant data. If the
> parameters don't validate, return an error, and handle that error
> appropriately in the javascript code.
>
> Thanks,
> Shawn
>
>
--
*Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
<http://opensourceconnections.com>, LLC | 240.476.9983
Author: Relevant Search <http://manning.com/turnbull>
This e-mail and all contents, including attachments, is considered to be
Company Confidential unless explicitly stated otherwise, regardless
of whether attachments are marked as such.
Re: AJAX access to Solr Server
Posted by Shawn Heisey <ap...@elyograg.org>.
On 12/25/2015 12:17 PM, Eric Dain wrote:
> Does allowing javascript direct access to SolrCloud raise security concern?
> should I build a REST service in between?
>
> I need to provide async search capability to web pages. the pages will be
> public with no authentication.
End users should never have access to Solr. Access to Solr from the
end-user machine is required if you want to accept Solr responses directly.
In one of the other replies that you received, Doug has given you an
nginx config for proxying access to Solr -- indirect access. This can
protect against *changes* to the index, and it has protection against
high start/rows values, but there are many other ways that an attacker
can construct denial of service queries, which this proxy config will
not prevent.
I think that indirect access (through a proxy) should not be allowed
either, unless you can trust all the people that will have access.
If Solr is open to a sufficiently wide audience (especially the
Internet), someone will find a way to abuse the service even with a
proxy, either to cause harm or to learn things they shouldn't know.
The most secure option is to only allow the webservers and trusted
administrators to access Solr. All end user (Internet) access to Solr
should be handled through a custom web application. This might be
something that you find and install (such as wordpress, drupal, etc), or
one that you write yourself.
You can still do AJAX while maintaining security. You'll need to write
something in a server-side web programming language like PHP, Java, etc.
This code will need to accept the AJAX requests from your client-side
javascript code, validate the request parameters to make sure they're
sane, get a response from Solr, and return relevant data. If the
parameters don't validate, return an error, and handle that error
appropriately in the javascript code.
Thanks,
Shawn