You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Douglas McGilvray <dm...@weemondo.com> on 2015/10/28 23:58:14 UTC
Securing field level access permission by filtering the query itself
Hi all,
First I’d like to say the nested facets and the json facet api in particular have made my world much better, I thank everyone involved, you are all awesome.
In my implementation has much of the solr query building working on the browser, solr is behind a php server which acts as “proxy” and doorman, filtering at the document level according to user role and supplying some sensible maximums …
However we now wish to filter just one or two potentially sensitive fields in one document type according to user role (as determined in the php proxy). Duplicating documents (or cores) seems like overkill for just two fields in one document type .. I wondered if it would be feasible (in the interests of preventing malicious activity) to filter the query itself whether it be parameters (fl, facet.fields, terms, etc) … or even deny any request in which fieldname occurs …
Is there someway someone might obscure a fieldname in a request?
Kind Regards & thanks in davacne,
Douglas
Re: Securing field level access permission by filtering the query itself
Posted by Alessandro Benedetti <ab...@apache.org>.
Are you basically saying that you are going to model 3 collections, 1 per
role .
Each collection schema will contain only the sensitive field.
When you query you simply search in the related collection and retrieve all
the fields.
that's it ?
Cheers
On 6 November 2015 at 15:05, Douglas McGilvray <dm...@weemondo.com> wrote:
> You know what guys, I have had a change in perspective…
>
> I previously thought: do I want to index all these documents multiple
> times just to protect 3 fields
> I am now thinking: do I really want to try to parse all the fields in a
> query when there are only 3 roles.
>
> I have only 4k documents and 3 roles, so thats 8k more documents and I
> doubt I will need to cross query with the other documents …
>
> Until I have more or more complex roles, or more protected documents, I
> think multiple cores is the best option …
>
> Cheers
> D
>
>
> > On 5 Nov 2015, at 12:50, Alessandro Benedetti <ab...@apache.org>
> wrote:
> >
> > Be careful to the suggester as well. You don't want to show suggestions
> > coming from sensitive fields.
> >
> > Cheers
> >
> > On 5 November 2015 at 15:28, Scott Stults <
> sstults@opensourceconnections.com
> >> wrote:
> >
> >> Good to hear! Depending on how far you want to take it, you can then
> scan
> >> the initial request coming in from the client (and the final response)
> for
> >> raw Solr fields -- that shouldn't happen. I've used mod_security as a
> >> general-purpose application firewall and would recommend it.
> >>
> >> k/r,
> >> Scott
> >>
> >> On Wed, Nov 4, 2015 at 1:40 PM, Douglas McGilvray <dm...@weemondo.com>
> wrote:
> >>
> >>>
> >>> Thanks Alessandro, I had overlooked the highlighting component.
> >>>
> >>> I will also add a reminder to exclude these fields from spellcheck
> >> fields,
> >>> (or maintain different spellcheck fields for different roles).
> >>>
> >>> @Scott - Once I started planning my code the penny finally dropped
> >>> regarding your point about aliasing the fields - it removes the need
> for
> >>> calculating which fields to request in the app itself.
> >>>
> >>> Regards,
> >>> D
> >>>
> >>>
> >>>> On 4 Nov 2015, at 14:53, Alessandro Benedetti <ab...@apache.org>
> >>> wrote:
> >>>>
> >>>> Of course it depends of all the query parameter you use and you
> process
> >>> in
> >>>> the response.
> >>>> The list you wrote should be ok if you use only those components.
> >>>>
> >>>> For example if you use highlight, it's not ok and you need to take
> care
> >>> of
> >>>> the highlighted fields as well.
> >>>>
> >>>> Cheers
> >>>>
> >>>> On 30 October 2015 at 14:51, Douglas McGilvray <dm...@weemondo.com>
> >> wrote:
> >>>>
> >>>>>
> >>>>> Scott thanks for the reply. I like the idea of mapping all the
> >>> fieldnames
> >>>>> internally, adding security through obscurity. My question therefore
> >>> would
> >>>>> be what is the definitive list of query parameters that one must
> >> filter
> >>> to
> >>>>> ensure a particular field is not exposed in the query response? Am I
> >>>>> missing in the following?
> >>>>>
> >>>>> fl
> >>>>> facect.field
> >>>>> facet.pivot
> >>>>> json.facet
> >>>>> terms.fl
> >>>>>
> >>>>>
> >>>>> kr
> >>>>> Douglas
> >>>>>
> >>>>>
> >>>>>> On 30 Oct 2015, at 07:37, Scott Stults <
> >>>>> sstults@opensourceconnections.com> wrote:
> >>>>>>
> >>>>>> Douglas,
> >>>>>>
> >>>>>> Managing a per-user-group whitelist of fields outside of Solr seems
> >> the
> >>>>>> best approach. When the query comes in you can then filter out any
> >>> fields
> >>>>>> not contained in the whitelist before you send the request to Solr.
> >> The
> >>>>>> easy part will be to do that on URL parameters like fl. Depending on
> >>> how
> >>>>>> your app generates the actual query string, you may want to also
> scan
> >>>>> that
> >>>>>> for fielded query clauses (eg "badfield:value") and localParams (eg
> >>>>>> "{!dismax qf=badfield}value").
> >>>>>>
> >>>>>> Secondly, you can map internal Solr fields to aliases using this
> >> syntax
> >>>>> in
> >>>>>> the fl parameter: "display_name:real_solr_name". So when the request
> >>>>> comes
> >>>>>> in from your app, first you'll map from the requested field alias
> >> names
> >>>>> to
> >>>>>> internal Solr names (while enforcing the whitelist), and then in the
> >> fl
> >>>>>> parameter supply the aliases you want sent in the response.
> >>>>>>
> >>>>>>
> >>>>>> k/r,
> >>>>>> Scott
> >>>>>>
> >>>>>> On Wed, Oct 28, 2015 at 6:58 PM, Douglas McGilvray <dm@weemondo.com
> >
> >>>>> wrote:
> >>>>>>
> >>>>>>> Hi all,
> >>>>>>>
> >>>>>>> First I’d like to say the nested facets and the json facet api in
> >>>>>>> particular have made my world much better, I thank everyone
> >> involved,
> >>>>> you
> >>>>>>> are all awesome.
> >>>>>>>
> >>>>>>> In my implementation has much of the solr query building working on
> >>> the
> >>>>>>> browser, solr is behind a php server which acts as “proxy” and
> >>> doorman,
> >>>>>>> filtering at the document level according to user role and
> supplying
> >>>>> some
> >>>>>>> sensible maximums …
> >>>>>>>
> >>>>>>> However we now wish to filter just one or two potentially sensitive
> >>>>> fields
> >>>>>>> in one document type according to user role (as determined in the
> >> php
> >>>>>>> proxy). Duplicating documents (or cores) seems like overkill for
> >> just
> >>>>> two
> >>>>>>> fields in one document type .. I wondered if it would be feasible
> >> (in
> >>>>> the
> >>>>>>> interests of preventing malicious activity) to filter the query
> >> itself
> >>>>>>> whether it be parameters (fl, facet.fields, terms, etc) … or even
> >> deny
> >>>>> any
> >>>>>>> request in which fieldname occurs …
> >>>>>>>
> >>>>>>> Is there someway someone might obscure a fieldname in a request?
> >>>>>>>
> >>>>>>> Kind Regards & thanks in davacne,
> >>>>>>> Douglas
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Scott Stults | Founder & Solutions Architect | OpenSource
> >> Connections,
> >>>>> LLC
> >>>>>> | 434.409.2780
> >>>>>> http://www.opensourceconnections.com
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> --------------------------
> >>>>
> >>>> Benedetti Alessandro
> >>>> Visiting card : http://about.me/alessandro_benedetti
> >>>>
> >>>> "Tyger, tyger burning bright
> >>>> In the forests of the night,
> >>>> What immortal hand or eye
> >>>> Could frame thy fearful symmetry?"
> >>>>
> >>>> William Blake - Songs of Experience -1794 England
> >>>
> >>>
> >>
> >>
> >> --
> >> Scott Stults | Founder & Solutions Architect | OpenSource Connections,
> LLC
> >> | 434.409.2780
> >> http://www.opensourceconnections.com
> >>
> >
> >
> >
> > --
> > --------------------------
> >
> > Benedetti Alessandro
> > Visiting card : http://about.me/alessandro_benedetti
> >
> > "Tyger, tyger burning bright
> > In the forests of the night,
> > What immortal hand or eye
> > Could frame thy fearful symmetry?"
> >
> > William Blake - Songs of Experience -1794 England
>
>
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Securing field level access permission by filtering the query itself
Posted by Douglas McGilvray <dm...@weemondo.com>.
You know what guys, I have had a change in perspective…
I previously thought: do I want to index all these documents multiple times just to protect 3 fields
I am now thinking: do I really want to try to parse all the fields in a query when there are only 3 roles.
I have only 4k documents and 3 roles, so thats 8k more documents and I doubt I will need to cross query with the other documents …
Until I have more or more complex roles, or more protected documents, I think multiple cores is the best option …
Cheers
D
> On 5 Nov 2015, at 12:50, Alessandro Benedetti <ab...@apache.org> wrote:
>
> Be careful to the suggester as well. You don't want to show suggestions
> coming from sensitive fields.
>
> Cheers
>
> On 5 November 2015 at 15:28, Scott Stults <sstults@opensourceconnections.com
>> wrote:
>
>> Good to hear! Depending on how far you want to take it, you can then scan
>> the initial request coming in from the client (and the final response) for
>> raw Solr fields -- that shouldn't happen. I've used mod_security as a
>> general-purpose application firewall and would recommend it.
>>
>> k/r,
>> Scott
>>
>> On Wed, Nov 4, 2015 at 1:40 PM, Douglas McGilvray <dm...@weemondo.com> wrote:
>>
>>>
>>> Thanks Alessandro, I had overlooked the highlighting component.
>>>
>>> I will also add a reminder to exclude these fields from spellcheck
>> fields,
>>> (or maintain different spellcheck fields for different roles).
>>>
>>> @Scott - Once I started planning my code the penny finally dropped
>>> regarding your point about aliasing the fields - it removes the need for
>>> calculating which fields to request in the app itself.
>>>
>>> Regards,
>>> D
>>>
>>>
>>>> On 4 Nov 2015, at 14:53, Alessandro Benedetti <ab...@apache.org>
>>> wrote:
>>>>
>>>> Of course it depends of all the query parameter you use and you process
>>> in
>>>> the response.
>>>> The list you wrote should be ok if you use only those components.
>>>>
>>>> For example if you use highlight, it's not ok and you need to take care
>>> of
>>>> the highlighted fields as well.
>>>>
>>>> Cheers
>>>>
>>>> On 30 October 2015 at 14:51, Douglas McGilvray <dm...@weemondo.com>
>> wrote:
>>>>
>>>>>
>>>>> Scott thanks for the reply. I like the idea of mapping all the
>>> fieldnames
>>>>> internally, adding security through obscurity. My question therefore
>>> would
>>>>> be what is the definitive list of query parameters that one must
>> filter
>>> to
>>>>> ensure a particular field is not exposed in the query response? Am I
>>>>> missing in the following?
>>>>>
>>>>> fl
>>>>> facect.field
>>>>> facet.pivot
>>>>> json.facet
>>>>> terms.fl
>>>>>
>>>>>
>>>>> kr
>>>>> Douglas
>>>>>
>>>>>
>>>>>> On 30 Oct 2015, at 07:37, Scott Stults <
>>>>> sstults@opensourceconnections.com> wrote:
>>>>>>
>>>>>> Douglas,
>>>>>>
>>>>>> Managing a per-user-group whitelist of fields outside of Solr seems
>> the
>>>>>> best approach. When the query comes in you can then filter out any
>>> fields
>>>>>> not contained in the whitelist before you send the request to Solr.
>> The
>>>>>> easy part will be to do that on URL parameters like fl. Depending on
>>> how
>>>>>> your app generates the actual query string, you may want to also scan
>>>>> that
>>>>>> for fielded query clauses (eg "badfield:value") and localParams (eg
>>>>>> "{!dismax qf=badfield}value").
>>>>>>
>>>>>> Secondly, you can map internal Solr fields to aliases using this
>> syntax
>>>>> in
>>>>>> the fl parameter: "display_name:real_solr_name". So when the request
>>>>> comes
>>>>>> in from your app, first you'll map from the requested field alias
>> names
>>>>> to
>>>>>> internal Solr names (while enforcing the whitelist), and then in the
>> fl
>>>>>> parameter supply the aliases you want sent in the response.
>>>>>>
>>>>>>
>>>>>> k/r,
>>>>>> Scott
>>>>>>
>>>>>> On Wed, Oct 28, 2015 at 6:58 PM, Douglas McGilvray <dm...@weemondo.com>
>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> First I’d like to say the nested facets and the json facet api in
>>>>>>> particular have made my world much better, I thank everyone
>> involved,
>>>>> you
>>>>>>> are all awesome.
>>>>>>>
>>>>>>> In my implementation has much of the solr query building working on
>>> the
>>>>>>> browser, solr is behind a php server which acts as “proxy” and
>>> doorman,
>>>>>>> filtering at the document level according to user role and supplying
>>>>> some
>>>>>>> sensible maximums …
>>>>>>>
>>>>>>> However we now wish to filter just one or two potentially sensitive
>>>>> fields
>>>>>>> in one document type according to user role (as determined in the
>> php
>>>>>>> proxy). Duplicating documents (or cores) seems like overkill for
>> just
>>>>> two
>>>>>>> fields in one document type .. I wondered if it would be feasible
>> (in
>>>>> the
>>>>>>> interests of preventing malicious activity) to filter the query
>> itself
>>>>>>> whether it be parameters (fl, facet.fields, terms, etc) … or even
>> deny
>>>>> any
>>>>>>> request in which fieldname occurs …
>>>>>>>
>>>>>>> Is there someway someone might obscure a fieldname in a request?
>>>>>>>
>>>>>>> Kind Regards & thanks in davacne,
>>>>>>> Douglas
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Scott Stults | Founder & Solutions Architect | OpenSource
>> Connections,
>>>>> LLC
>>>>>> | 434.409.2780
>>>>>> http://www.opensourceconnections.com
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> --------------------------
>>>>
>>>> Benedetti Alessandro
>>>> Visiting card : http://about.me/alessandro_benedetti
>>>>
>>>> "Tyger, tyger burning bright
>>>> In the forests of the night,
>>>> What immortal hand or eye
>>>> Could frame thy fearful symmetry?"
>>>>
>>>> William Blake - Songs of Experience -1794 England
>>>
>>>
>>
>>
>> --
>> Scott Stults | Founder & Solutions Architect | OpenSource Connections, LLC
>> | 434.409.2780
>> http://www.opensourceconnections.com
>>
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
Re: Securing field level access permission by filtering the query itself
Posted by Alessandro Benedetti <ab...@apache.org>.
Be careful to the suggester as well. You don't want to show suggestions
coming from sensitive fields.
Cheers
On 5 November 2015 at 15:28, Scott Stults <sstults@opensourceconnections.com
> wrote:
> Good to hear! Depending on how far you want to take it, you can then scan
> the initial request coming in from the client (and the final response) for
> raw Solr fields -- that shouldn't happen. I've used mod_security as a
> general-purpose application firewall and would recommend it.
>
> k/r,
> Scott
>
> On Wed, Nov 4, 2015 at 1:40 PM, Douglas McGilvray <dm...@weemondo.com> wrote:
>
> >
> > Thanks Alessandro, I had overlooked the highlighting component.
> >
> > I will also add a reminder to exclude these fields from spellcheck
> fields,
> > (or maintain different spellcheck fields for different roles).
> >
> > @Scott - Once I started planning my code the penny finally dropped
> > regarding your point about aliasing the fields - it removes the need for
> > calculating which fields to request in the app itself.
> >
> > Regards,
> > D
> >
> >
> > > On 4 Nov 2015, at 14:53, Alessandro Benedetti <ab...@apache.org>
> > wrote:
> > >
> > > Of course it depends of all the query parameter you use and you process
> > in
> > > the response.
> > > The list you wrote should be ok if you use only those components.
> > >
> > > For example if you use highlight, it's not ok and you need to take care
> > of
> > > the highlighted fields as well.
> > >
> > > Cheers
> > >
> > > On 30 October 2015 at 14:51, Douglas McGilvray <dm...@weemondo.com>
> wrote:
> > >
> > >>
> > >> Scott thanks for the reply. I like the idea of mapping all the
> > fieldnames
> > >> internally, adding security through obscurity. My question therefore
> > would
> > >> be what is the definitive list of query parameters that one must
> filter
> > to
> > >> ensure a particular field is not exposed in the query response? Am I
> > >> missing in the following?
> > >>
> > >> fl
> > >> facect.field
> > >> facet.pivot
> > >> json.facet
> > >> terms.fl
> > >>
> > >>
> > >> kr
> > >> Douglas
> > >>
> > >>
> > >>> On 30 Oct 2015, at 07:37, Scott Stults <
> > >> sstults@opensourceconnections.com> wrote:
> > >>>
> > >>> Douglas,
> > >>>
> > >>> Managing a per-user-group whitelist of fields outside of Solr seems
> the
> > >>> best approach. When the query comes in you can then filter out any
> > fields
> > >>> not contained in the whitelist before you send the request to Solr.
> The
> > >>> easy part will be to do that on URL parameters like fl. Depending on
> > how
> > >>> your app generates the actual query string, you may want to also scan
> > >> that
> > >>> for fielded query clauses (eg "badfield:value") and localParams (eg
> > >>> "{!dismax qf=badfield}value").
> > >>>
> > >>> Secondly, you can map internal Solr fields to aliases using this
> syntax
> > >> in
> > >>> the fl parameter: "display_name:real_solr_name". So when the request
> > >> comes
> > >>> in from your app, first you'll map from the requested field alias
> names
> > >> to
> > >>> internal Solr names (while enforcing the whitelist), and then in the
> fl
> > >>> parameter supply the aliases you want sent in the response.
> > >>>
> > >>>
> > >>> k/r,
> > >>> Scott
> > >>>
> > >>> On Wed, Oct 28, 2015 at 6:58 PM, Douglas McGilvray <dm...@weemondo.com>
> > >> wrote:
> > >>>
> > >>>> Hi all,
> > >>>>
> > >>>> First I’d like to say the nested facets and the json facet api in
> > >>>> particular have made my world much better, I thank everyone
> involved,
> > >> you
> > >>>> are all awesome.
> > >>>>
> > >>>> In my implementation has much of the solr query building working on
> > the
> > >>>> browser, solr is behind a php server which acts as “proxy” and
> > doorman,
> > >>>> filtering at the document level according to user role and supplying
> > >> some
> > >>>> sensible maximums …
> > >>>>
> > >>>> However we now wish to filter just one or two potentially sensitive
> > >> fields
> > >>>> in one document type according to user role (as determined in the
> php
> > >>>> proxy). Duplicating documents (or cores) seems like overkill for
> just
> > >> two
> > >>>> fields in one document type .. I wondered if it would be feasible
> (in
> > >> the
> > >>>> interests of preventing malicious activity) to filter the query
> itself
> > >>>> whether it be parameters (fl, facet.fields, terms, etc) … or even
> deny
> > >> any
> > >>>> request in which fieldname occurs …
> > >>>>
> > >>>> Is there someway someone might obscure a fieldname in a request?
> > >>>>
> > >>>> Kind Regards & thanks in davacne,
> > >>>> Douglas
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Scott Stults | Founder & Solutions Architect | OpenSource
> Connections,
> > >> LLC
> > >>> | 434.409.2780
> > >>> http://www.opensourceconnections.com
> > >>
> > >>
> > >
> > >
> > > --
> > > --------------------------
> > >
> > > Benedetti Alessandro
> > > Visiting card : http://about.me/alessandro_benedetti
> > >
> > > "Tyger, tyger burning bright
> > > In the forests of the night,
> > > What immortal hand or eye
> > > Could frame thy fearful symmetry?"
> > >
> > > William Blake - Songs of Experience -1794 England
> >
> >
>
>
> --
> Scott Stults | Founder & Solutions Architect | OpenSource Connections, LLC
> | 434.409.2780
> http://www.opensourceconnections.com
>
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Securing field level access permission by filtering the query itself
Posted by Scott Stults <ss...@opensourceconnections.com>.
Good to hear! Depending on how far you want to take it, you can then scan
the initial request coming in from the client (and the final response) for
raw Solr fields -- that shouldn't happen. I've used mod_security as a
general-purpose application firewall and would recommend it.
k/r,
Scott
On Wed, Nov 4, 2015 at 1:40 PM, Douglas McGilvray <dm...@weemondo.com> wrote:
>
> Thanks Alessandro, I had overlooked the highlighting component.
>
> I will also add a reminder to exclude these fields from spellcheck fields,
> (or maintain different spellcheck fields for different roles).
>
> @Scott - Once I started planning my code the penny finally dropped
> regarding your point about aliasing the fields - it removes the need for
> calculating which fields to request in the app itself.
>
> Regards,
> D
>
>
> > On 4 Nov 2015, at 14:53, Alessandro Benedetti <ab...@apache.org>
> wrote:
> >
> > Of course it depends of all the query parameter you use and you process
> in
> > the response.
> > The list you wrote should be ok if you use only those components.
> >
> > For example if you use highlight, it's not ok and you need to take care
> of
> > the highlighted fields as well.
> >
> > Cheers
> >
> > On 30 October 2015 at 14:51, Douglas McGilvray <dm...@weemondo.com> wrote:
> >
> >>
> >> Scott thanks for the reply. I like the idea of mapping all the
> fieldnames
> >> internally, adding security through obscurity. My question therefore
> would
> >> be what is the definitive list of query parameters that one must filter
> to
> >> ensure a particular field is not exposed in the query response? Am I
> >> missing in the following?
> >>
> >> fl
> >> facect.field
> >> facet.pivot
> >> json.facet
> >> terms.fl
> >>
> >>
> >> kr
> >> Douglas
> >>
> >>
> >>> On 30 Oct 2015, at 07:37, Scott Stults <
> >> sstults@opensourceconnections.com> wrote:
> >>>
> >>> Douglas,
> >>>
> >>> Managing a per-user-group whitelist of fields outside of Solr seems the
> >>> best approach. When the query comes in you can then filter out any
> fields
> >>> not contained in the whitelist before you send the request to Solr. The
> >>> easy part will be to do that on URL parameters like fl. Depending on
> how
> >>> your app generates the actual query string, you may want to also scan
> >> that
> >>> for fielded query clauses (eg "badfield:value") and localParams (eg
> >>> "{!dismax qf=badfield}value").
> >>>
> >>> Secondly, you can map internal Solr fields to aliases using this syntax
> >> in
> >>> the fl parameter: "display_name:real_solr_name". So when the request
> >> comes
> >>> in from your app, first you'll map from the requested field alias names
> >> to
> >>> internal Solr names (while enforcing the whitelist), and then in the fl
> >>> parameter supply the aliases you want sent in the response.
> >>>
> >>>
> >>> k/r,
> >>> Scott
> >>>
> >>> On Wed, Oct 28, 2015 at 6:58 PM, Douglas McGilvray <dm...@weemondo.com>
> >> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> First I’d like to say the nested facets and the json facet api in
> >>>> particular have made my world much better, I thank everyone involved,
> >> you
> >>>> are all awesome.
> >>>>
> >>>> In my implementation has much of the solr query building working on
> the
> >>>> browser, solr is behind a php server which acts as “proxy” and
> doorman,
> >>>> filtering at the document level according to user role and supplying
> >> some
> >>>> sensible maximums …
> >>>>
> >>>> However we now wish to filter just one or two potentially sensitive
> >> fields
> >>>> in one document type according to user role (as determined in the php
> >>>> proxy). Duplicating documents (or cores) seems like overkill for just
> >> two
> >>>> fields in one document type .. I wondered if it would be feasible (in
> >> the
> >>>> interests of preventing malicious activity) to filter the query itself
> >>>> whether it be parameters (fl, facet.fields, terms, etc) … or even deny
> >> any
> >>>> request in which fieldname occurs …
> >>>>
> >>>> Is there someway someone might obscure a fieldname in a request?
> >>>>
> >>>> Kind Regards & thanks in davacne,
> >>>> Douglas
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Scott Stults | Founder & Solutions Architect | OpenSource Connections,
> >> LLC
> >>> | 434.409.2780
> >>> http://www.opensourceconnections.com
> >>
> >>
> >
> >
> > --
> > --------------------------
> >
> > Benedetti Alessandro
> > Visiting card : http://about.me/alessandro_benedetti
> >
> > "Tyger, tyger burning bright
> > In the forests of the night,
> > What immortal hand or eye
> > Could frame thy fearful symmetry?"
> >
> > William Blake - Songs of Experience -1794 England
>
>
--
Scott Stults | Founder & Solutions Architect | OpenSource Connections, LLC
| 434.409.2780
http://www.opensourceconnections.com
Re: Securing field level access permission by filtering the query itself
Posted by Douglas McGilvray <dm...@weemondo.com>.
Thanks Alessandro, I had overlooked the highlighting component.
I will also add a reminder to exclude these fields from spellcheck fields, (or maintain different spellcheck fields for different roles).
@Scott - Once I started planning my code the penny finally dropped regarding your point about aliasing the fields - it removes the need for calculating which fields to request in the app itself.
Regards,
D
> On 4 Nov 2015, at 14:53, Alessandro Benedetti <ab...@apache.org> wrote:
>
> Of course it depends of all the query parameter you use and you process in
> the response.
> The list you wrote should be ok if you use only those components.
>
> For example if you use highlight, it's not ok and you need to take care of
> the highlighted fields as well.
>
> Cheers
>
> On 30 October 2015 at 14:51, Douglas McGilvray <dm...@weemondo.com> wrote:
>
>>
>> Scott thanks for the reply. I like the idea of mapping all the fieldnames
>> internally, adding security through obscurity. My question therefore would
>> be what is the definitive list of query parameters that one must filter to
>> ensure a particular field is not exposed in the query response? Am I
>> missing in the following?
>>
>> fl
>> facect.field
>> facet.pivot
>> json.facet
>> terms.fl
>>
>>
>> kr
>> Douglas
>>
>>
>>> On 30 Oct 2015, at 07:37, Scott Stults <
>> sstults@opensourceconnections.com> wrote:
>>>
>>> Douglas,
>>>
>>> Managing a per-user-group whitelist of fields outside of Solr seems the
>>> best approach. When the query comes in you can then filter out any fields
>>> not contained in the whitelist before you send the request to Solr. The
>>> easy part will be to do that on URL parameters like fl. Depending on how
>>> your app generates the actual query string, you may want to also scan
>> that
>>> for fielded query clauses (eg "badfield:value") and localParams (eg
>>> "{!dismax qf=badfield}value").
>>>
>>> Secondly, you can map internal Solr fields to aliases using this syntax
>> in
>>> the fl parameter: "display_name:real_solr_name". So when the request
>> comes
>>> in from your app, first you'll map from the requested field alias names
>> to
>>> internal Solr names (while enforcing the whitelist), and then in the fl
>>> parameter supply the aliases you want sent in the response.
>>>
>>>
>>> k/r,
>>> Scott
>>>
>>> On Wed, Oct 28, 2015 at 6:58 PM, Douglas McGilvray <dm...@weemondo.com>
>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> First I’d like to say the nested facets and the json facet api in
>>>> particular have made my world much better, I thank everyone involved,
>> you
>>>> are all awesome.
>>>>
>>>> In my implementation has much of the solr query building working on the
>>>> browser, solr is behind a php server which acts as “proxy” and doorman,
>>>> filtering at the document level according to user role and supplying
>> some
>>>> sensible maximums …
>>>>
>>>> However we now wish to filter just one or two potentially sensitive
>> fields
>>>> in one document type according to user role (as determined in the php
>>>> proxy). Duplicating documents (or cores) seems like overkill for just
>> two
>>>> fields in one document type .. I wondered if it would be feasible (in
>> the
>>>> interests of preventing malicious activity) to filter the query itself
>>>> whether it be parameters (fl, facet.fields, terms, etc) … or even deny
>> any
>>>> request in which fieldname occurs …
>>>>
>>>> Is there someway someone might obscure a fieldname in a request?
>>>>
>>>> Kind Regards & thanks in davacne,
>>>> Douglas
>>>
>>>
>>>
>>>
>>> --
>>> Scott Stults | Founder & Solutions Architect | OpenSource Connections,
>> LLC
>>> | 434.409.2780
>>> http://www.opensourceconnections.com
>>
>>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
Re: Securing field level access permission by filtering the query itself
Posted by Alessandro Benedetti <ab...@apache.org>.
Of course it depends of all the query parameter you use and you process in
the response.
The list you wrote should be ok if you use only those components.
For example if you use highlight, it's not ok and you need to take care of
the highlighted fields as well.
Cheers
On 30 October 2015 at 14:51, Douglas McGilvray <dm...@weemondo.com> wrote:
>
> Scott thanks for the reply. I like the idea of mapping all the fieldnames
> internally, adding security through obscurity. My question therefore would
> be what is the definitive list of query parameters that one must filter to
> ensure a particular field is not exposed in the query response? Am I
> missing in the following?
>
> fl
> facect.field
> facet.pivot
> json.facet
> terms.fl
>
>
> kr
> Douglas
>
>
> > On 30 Oct 2015, at 07:37, Scott Stults <
> sstults@opensourceconnections.com> wrote:
> >
> > Douglas,
> >
> > Managing a per-user-group whitelist of fields outside of Solr seems the
> > best approach. When the query comes in you can then filter out any fields
> > not contained in the whitelist before you send the request to Solr. The
> > easy part will be to do that on URL parameters like fl. Depending on how
> > your app generates the actual query string, you may want to also scan
> that
> > for fielded query clauses (eg "badfield:value") and localParams (eg
> > "{!dismax qf=badfield}value").
> >
> > Secondly, you can map internal Solr fields to aliases using this syntax
> in
> > the fl parameter: "display_name:real_solr_name". So when the request
> comes
> > in from your app, first you'll map from the requested field alias names
> to
> > internal Solr names (while enforcing the whitelist), and then in the fl
> > parameter supply the aliases you want sent in the response.
> >
> >
> > k/r,
> > Scott
> >
> > On Wed, Oct 28, 2015 at 6:58 PM, Douglas McGilvray <dm...@weemondo.com>
> wrote:
> >
> >> Hi all,
> >>
> >> First I’d like to say the nested facets and the json facet api in
> >> particular have made my world much better, I thank everyone involved,
> you
> >> are all awesome.
> >>
> >> In my implementation has much of the solr query building working on the
> >> browser, solr is behind a php server which acts as “proxy” and doorman,
> >> filtering at the document level according to user role and supplying
> some
> >> sensible maximums …
> >>
> >> However we now wish to filter just one or two potentially sensitive
> fields
> >> in one document type according to user role (as determined in the php
> >> proxy). Duplicating documents (or cores) seems like overkill for just
> two
> >> fields in one document type .. I wondered if it would be feasible (in
> the
> >> interests of preventing malicious activity) to filter the query itself
> >> whether it be parameters (fl, facet.fields, terms, etc) … or even deny
> any
> >> request in which fieldname occurs …
> >>
> >> Is there someway someone might obscure a fieldname in a request?
> >>
> >> Kind Regards & thanks in davacne,
> >> Douglas
> >
> >
> >
> >
> > --
> > Scott Stults | Founder & Solutions Architect | OpenSource Connections,
> LLC
> > | 434.409.2780
> > http://www.opensourceconnections.com
>
>
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Securing field level access permission by filtering the query itself
Posted by Douglas McGilvray <dm...@weemondo.com>.
Scott thanks for the reply. I like the idea of mapping all the fieldnames internally, adding security through obscurity. My question therefore would be what is the definitive list of query parameters that one must filter to ensure a particular field is not exposed in the query response? Am I missing in the following?
fl
facect.field
facet.pivot
json.facet
terms.fl
kr
Douglas
> On 30 Oct 2015, at 07:37, Scott Stults <ss...@opensourceconnections.com> wrote:
>
> Douglas,
>
> Managing a per-user-group whitelist of fields outside of Solr seems the
> best approach. When the query comes in you can then filter out any fields
> not contained in the whitelist before you send the request to Solr. The
> easy part will be to do that on URL parameters like fl. Depending on how
> your app generates the actual query string, you may want to also scan that
> for fielded query clauses (eg "badfield:value") and localParams (eg
> "{!dismax qf=badfield}value").
>
> Secondly, you can map internal Solr fields to aliases using this syntax in
> the fl parameter: "display_name:real_solr_name". So when the request comes
> in from your app, first you'll map from the requested field alias names to
> internal Solr names (while enforcing the whitelist), and then in the fl
> parameter supply the aliases you want sent in the response.
>
>
> k/r,
> Scott
>
> On Wed, Oct 28, 2015 at 6:58 PM, Douglas McGilvray <dm...@weemondo.com> wrote:
>
>> Hi all,
>>
>> First I’d like to say the nested facets and the json facet api in
>> particular have made my world much better, I thank everyone involved, you
>> are all awesome.
>>
>> In my implementation has much of the solr query building working on the
>> browser, solr is behind a php server which acts as “proxy” and doorman,
>> filtering at the document level according to user role and supplying some
>> sensible maximums …
>>
>> However we now wish to filter just one or two potentially sensitive fields
>> in one document type according to user role (as determined in the php
>> proxy). Duplicating documents (or cores) seems like overkill for just two
>> fields in one document type .. I wondered if it would be feasible (in the
>> interests of preventing malicious activity) to filter the query itself
>> whether it be parameters (fl, facet.fields, terms, etc) … or even deny any
>> request in which fieldname occurs …
>>
>> Is there someway someone might obscure a fieldname in a request?
>>
>> Kind Regards & thanks in davacne,
>> Douglas
>
>
>
>
> --
> Scott Stults | Founder & Solutions Architect | OpenSource Connections, LLC
> | 434.409.2780
> http://www.opensourceconnections.com
Re: Securing field level access permission by filtering the query itself
Posted by Scott Stults <ss...@opensourceconnections.com>.
Douglas,
Managing a per-user-group whitelist of fields outside of Solr seems the
best approach. When the query comes in you can then filter out any fields
not contained in the whitelist before you send the request to Solr. The
easy part will be to do that on URL parameters like fl. Depending on how
your app generates the actual query string, you may want to also scan that
for fielded query clauses (eg "badfield:value") and localParams (eg
"{!dismax qf=badfield}value").
Secondly, you can map internal Solr fields to aliases using this syntax in
the fl parameter: "display_name:real_solr_name". So when the request comes
in from your app, first you'll map from the requested field alias names to
internal Solr names (while enforcing the whitelist), and then in the fl
parameter supply the aliases you want sent in the response.
k/r,
Scott
On Wed, Oct 28, 2015 at 6:58 PM, Douglas McGilvray <dm...@weemondo.com> wrote:
> Hi all,
>
> First I’d like to say the nested facets and the json facet api in
> particular have made my world much better, I thank everyone involved, you
> are all awesome.
>
> In my implementation has much of the solr query building working on the
> browser, solr is behind a php server which acts as “proxy” and doorman,
> filtering at the document level according to user role and supplying some
> sensible maximums …
>
> However we now wish to filter just one or two potentially sensitive fields
> in one document type according to user role (as determined in the php
> proxy). Duplicating documents (or cores) seems like overkill for just two
> fields in one document type .. I wondered if it would be feasible (in the
> interests of preventing malicious activity) to filter the query itself
> whether it be parameters (fl, facet.fields, terms, etc) … or even deny any
> request in which fieldname occurs …
>
> Is there someway someone might obscure a fieldname in a request?
>
> Kind Regards & thanks in davacne,
> Douglas
--
Scott Stults | Founder & Solutions Architect | OpenSource Connections, LLC
| 434.409.2780
http://www.opensourceconnections.com