Enforced filter functions

[Dale Harvey <da...@arandomurl.com>]

So I was trying to implement the ability for logged in users to subscribe
to the changes feed for updates to their own documents (its currently admin
only), its a simple patch but its not very clean (mostly because the we
dont want to have the changes feed read the full document)

A way that I could implement it, that seems a lot more globally applicable
and applies cleaner, is to allow the ddoc author to enforce filters to
happen on non admin access to _changes and replication (on any database)

This should negatively affect anything else, filters are already used and
supported in both places, I have also seen it asked for regularly in the
context of replication, ddoc authors could specify exactly what users are
allowed to replicate, the enforced filter would override any users filter

As far as I can tell the only problem would be how to have the author
specify the enforcement, apart from that it should apply cleanly,
introduces a fairly large amount of new functionality and adds very little
overhead

Ideas?

Jan mentioned this could be used more widely for updates / shows / lists,
that is starting to sound more complicated and does start introducing
problems, I would prefer to look at the simple case of filters for changes
/ replication for now, but if there was an even more global solution that
applied cleanly, I would be happy to

Re: Enforced filter functions

[Jan Lehnardt <ja...@apache.org>]

Another way to solve the particular feature would be "per-doc changes":

    GET /db/doc/_changes

I can see tons of use-cases for that already :)

Cheers
Jan
-- 

On Mar 13, 2012, at 21:17 , Dale Harvey wrote:

> Yup as Jan said, even the 'only run on non admin access' can live inside
> the filter (there may be performance reasons to want to avoid that though)
> 
> Robert has expressed some concerns on irc about the implementation of
> various security related features in an adhoc way, I do agree, the entire
> reason I proposed this was to avoid a special case for subscribing to the
> users table.
> 
> I think with some eye to the future this can be implemented in a way that
> builds towards a reasonably complete set of security functionality, I dont
> think it needs to be another edge case feature, I will try to come up with
> a proposal that keeps in mind new things that couch will likely want to
> implement
> 
> Any things to keep in mind, or other proposals would be nice
> 
> On 13 March 2012 19:36, Jan Lehnardt <ja...@apache.org> wrote:
> 
>> 
>> On Mar 13, 2012, at 20:32 , Randall Leeds wrote:
>> 
>>> Do filters already have access to the user context? My first thought is
>>> that this would allow such a filter function to be written (one that
>> works
>>> for all users, but filters based on user).
>> 
>> Yes.
>> 
>> The problem with your suggestion is that the client can optionally specify
>> the filter, or not, thus bypassing it.
>> 
>> Cheers
>> Jan
>> --
>> 
>>> 
>>> On Tue, Mar 13, 2012 at 08:33, Dale Harvey <da...@arandomurl.com> wrote:
>>> 
>>>> On 13 March 2012 15:30, Dale Harvey <da...@arandomurl.com> wrote:
>>>> 
>>>>> So I was trying to implement the ability for logged in users to
>> subscribe
>>>>> to the changes feed for updates to their own documents (its currently
>>>> admin
>>>>> only), its a simple patch but its not very clean (mostly because the we
>>>>> dont want to have the changes feed read the full document)
>>>>> 
>>>>> A way that I could implement it, that seems a lot more globally
>>>> applicable
>>>>> and applies cleaner, is to allow the ddoc author to enforce filters to
>>>>> happen on non admin access to _changes and replication (on any
>> database)
>>>>> 
>>>>> This should negatively affect anything else, filters are already used
>> and
>>>>> supported in both places, I have also seen it asked for regularly in
>> the
>>>>> context of replication, ddoc authors could specify exactly what users
>> are
>>>>> allowed to replicate, the enforced filter would override any users
>> filter
>>>>> 
>>>>> 
>>>> Obviously I meant "should not negatively affect" :)
>>>> 
>>>> 
>>>>> As far as I can tell the only problem would be how to have the author
>>>>> specify the enforcement, apart from that it should apply cleanly,
>>>>> introduces a fairly large amount of new functionality and adds very
>>>> little
>>>>> overhead
>>>>> 
>>>>> Ideas?
>>>>> 
>>>>> Jan mentioned this could be used more widely for updates / shows /
>> lists,
>>>>> that is starting to sound more complicated and does start introducing
>>>>> problems, I would prefer to look at the simple case of filters for
>>>> changes
>>>>> / replication for now, but if there was an even more global solution
>> that
>>>>> applied cleanly, I would be happy to
>>>>> 
>>>>> 
>>>>> 
>>>> 
>> 
>> 

Re: Enforced filter functions

[Dale Harvey <da...@arandomurl.com>]

Yup as Jan said, even the 'only run on non admin access' can live inside
the filter (there may be performance reasons to want to avoid that though)

Robert has expressed some concerns on irc about the implementation of
various security related features in an adhoc way, I do agree, the entire
reason I proposed this was to avoid a special case for subscribing to the
users table.

I think with some eye to the future this can be implemented in a way that
builds towards a reasonably complete set of security functionality, I dont
think it needs to be another edge case feature, I will try to come up with
a proposal that keeps in mind new things that couch will likely want to
implement

Any things to keep in mind, or other proposals would be nice

On 13 March 2012 19:36, Jan Lehnardt <ja...@apache.org> wrote:

>
> On Mar 13, 2012, at 20:32 , Randall Leeds wrote:
>
> > Do filters already have access to the user context? My first thought is
> > that this would allow such a filter function to be written (one that
> works
> > for all users, but filters based on user).
>
> Yes.
>
> The problem with your suggestion is that the client can optionally specify
> the filter, or not, thus bypassing it.
>
> Cheers
> Jan
> --
>
> >
> > On Tue, Mar 13, 2012 at 08:33, Dale Harvey <da...@arandomurl.com> wrote:
> >
> >> On 13 March 2012 15:30, Dale Harvey <da...@arandomurl.com> wrote:
> >>
> >>> So I was trying to implement the ability for logged in users to
> subscribe
> >>> to the changes feed for updates to their own documents (its currently
> >> admin
> >>> only), its a simple patch but its not very clean (mostly because the we
> >>> dont want to have the changes feed read the full document)
> >>>
> >>> A way that I could implement it, that seems a lot more globally
> >> applicable
> >>> and applies cleaner, is to allow the ddoc author to enforce filters to
> >>> happen on non admin access to _changes and replication (on any
> database)
> >>>
> >>> This should negatively affect anything else, filters are already used
> and
> >>> supported in both places, I have also seen it asked for regularly in
> the
> >>> context of replication, ddoc authors could specify exactly what users
> are
> >>> allowed to replicate, the enforced filter would override any users
> filter
> >>>
> >>>
> >> Obviously I meant "should not negatively affect" :)
> >>
> >>
> >>> As far as I can tell the only problem would be how to have the author
> >>> specify the enforcement, apart from that it should apply cleanly,
> >>> introduces a fairly large amount of new functionality and adds very
> >> little
> >>> overhead
> >>>
> >>> Ideas?
> >>>
> >>> Jan mentioned this could be used more widely for updates / shows /
> lists,
> >>> that is starting to sound more complicated and does start introducing
> >>> problems, I would prefer to look at the simple case of filters for
> >> changes
> >>> / replication for now, but if there was an even more global solution
> that
> >>> applied cleanly, I would be happy to
> >>>
> >>>
> >>>
> >>
>
>

Re: Enforced filter functions

[Jan Lehnardt <ja...@apache.org>]

On Mar 13, 2012, at 20:32 , Randall Leeds wrote:

> Do filters already have access to the user context? My first thought is
> that this would allow such a filter function to be written (one that works
> for all users, but filters based on user).

Yes.

The problem with your suggestion is that the client can optionally specify
the filter, or not, thus bypassing it.

Cheers
Jan
-- 

> 
> On Tue, Mar 13, 2012 at 08:33, Dale Harvey <da...@arandomurl.com> wrote:
> 
>> On 13 March 2012 15:30, Dale Harvey <da...@arandomurl.com> wrote:
>> 
>>> So I was trying to implement the ability for logged in users to subscribe
>>> to the changes feed for updates to their own documents (its currently
>> admin
>>> only), its a simple patch but its not very clean (mostly because the we
>>> dont want to have the changes feed read the full document)
>>> 
>>> A way that I could implement it, that seems a lot more globally
>> applicable
>>> and applies cleaner, is to allow the ddoc author to enforce filters to
>>> happen on non admin access to _changes and replication (on any database)
>>> 
>>> This should negatively affect anything else, filters are already used and
>>> supported in both places, I have also seen it asked for regularly in the
>>> context of replication, ddoc authors could specify exactly what users are
>>> allowed to replicate, the enforced filter would override any users filter
>>> 
>>> 
>> Obviously I meant "should not negatively affect" :)
>> 
>> 
>>> As far as I can tell the only problem would be how to have the author
>>> specify the enforcement, apart from that it should apply cleanly,
>>> introduces a fairly large amount of new functionality and adds very
>> little
>>> overhead
>>> 
>>> Ideas?
>>> 
>>> Jan mentioned this could be used more widely for updates / shows / lists,
>>> that is starting to sound more complicated and does start introducing
>>> problems, I would prefer to look at the simple case of filters for
>> changes
>>> / replication for now, but if there was an even more global solution that
>>> applied cleanly, I would be happy to
>>> 
>>> 
>>> 
>> 

Re: Enforced filter functions

[Randall Leeds <ra...@gmail.com>]

Do filters already have access to the user context? My first thought is
that this would allow such a filter function to be written (one that works
for all users, but filters based on user).

On Tue, Mar 13, 2012 at 08:33, Dale Harvey <da...@arandomurl.com> wrote:

> On 13 March 2012 15:30, Dale Harvey <da...@arandomurl.com> wrote:
>
> > So I was trying to implement the ability for logged in users to subscribe
> > to the changes feed for updates to their own documents (its currently
> admin
> > only), its a simple patch but its not very clean (mostly because the we
> > dont want to have the changes feed read the full document)
> >
> > A way that I could implement it, that seems a lot more globally
> applicable
> > and applies cleaner, is to allow the ddoc author to enforce filters to
> > happen on non admin access to _changes and replication (on any database)
> >
> > This should negatively affect anything else, filters are already used and
> > supported in both places, I have also seen it asked for regularly in the
> > context of replication, ddoc authors could specify exactly what users are
> > allowed to replicate, the enforced filter would override any users filter
> >
> >
> Obviously I meant "should not negatively affect" :)
>
>
> > As far as I can tell the only problem would be how to have the author
> > specify the enforcement, apart from that it should apply cleanly,
> > introduces a fairly large amount of new functionality and adds very
> little
> > overhead
> >
> > Ideas?
> >
> > Jan mentioned this could be used more widely for updates / shows / lists,
> > that is starting to sound more complicated and does start introducing
> > problems, I would prefer to look at the simple case of filters for
> changes
> > / replication for now, but if there was an even more global solution that
> > applied cleanly, I would be happy to
> >
> >
> >
>

Re: Enforced filter functions

[Robert Newson <rn...@apache.org>]

Summarizing some things I said on IRC:

I'm more than a little nervous about any more steps that appear to
give fine-grained read access control within a database. We have
repeatedly rejected the idea when it's been raised by community
members citing the numerous problems that come with it (data leaking
via views, for example) so we risk introducing even more confusion
with a volte face. If we pursue this, we must get it right.

Today there's at least one case where we currently have mandatory code
execution, validate_doc_update, that is causing issues. Because it
cannot be bypassed, even by an admin, BigCouch, which strives to make
N copies of every document, can be hindered. This example illustrates
some part of the danger that 'enforced' code execution brings. Dale
does state that the proposed filters would affect non-admins but we
should be be conscious of how different couchdb looks and behaves to
users vs admins.

I agree completely that we need to add features incrementally rather
than attempt some giant feat within a release. What I haven't yet
heard is a coherent plan for where we'll end up. As a committer, I
share responsibility in helping to define and deliver that plan.

B.


On 13 March 2012 15:33, Dale Harvey <da...@arandomurl.com> wrote:
> On 13 March 2012 15:30, Dale Harvey <da...@arandomurl.com> wrote:
>
>> So I was trying to implement the ability for logged in users to subscribe
>> to the changes feed for updates to their own documents (its currently admin
>> only), its a simple patch but its not very clean (mostly because the we
>> dont want to have the changes feed read the full document)
>>
>> A way that I could implement it, that seems a lot more globally applicable
>> and applies cleaner, is to allow the ddoc author to enforce filters to
>> happen on non admin access to _changes and replication (on any database)
>>
>> This should negatively affect anything else, filters are already used and
>> supported in both places, I have also seen it asked for regularly in the
>> context of replication, ddoc authors could specify exactly what users are
>> allowed to replicate, the enforced filter would override any users filter
>>
>>
> Obviously I meant "should not negatively affect" :)
>
>
>> As far as I can tell the only problem would be how to have the author
>> specify the enforcement, apart from that it should apply cleanly,
>> introduces a fairly large amount of new functionality and adds very little
>> overhead
>>
>> Ideas?
>>
>> Jan mentioned this could be used more widely for updates / shows / lists,
>> that is starting to sound more complicated and does start introducing
>> problems, I would prefer to look at the simple case of filters for changes
>> / replication for now, but if there was an even more global solution that
>> applied cleanly, I would be happy to
>>
>>
>>

Re: Enforced filter functions

[Dale Harvey <da...@arandomurl.com>]

On 13 March 2012 15:30, Dale Harvey <da...@arandomurl.com> wrote:

> So I was trying to implement the ability for logged in users to subscribe
> to the changes feed for updates to their own documents (its currently admin
> only), its a simple patch but its not very clean (mostly because the we
> dont want to have the changes feed read the full document)
>
> A way that I could implement it, that seems a lot more globally applicable
> and applies cleaner, is to allow the ddoc author to enforce filters to
> happen on non admin access to _changes and replication (on any database)
>
> This should negatively affect anything else, filters are already used and
> supported in both places, I have also seen it asked for regularly in the
> context of replication, ddoc authors could specify exactly what users are
> allowed to replicate, the enforced filter would override any users filter
>
>
Obviously I meant "should not negatively affect" :)


> As far as I can tell the only problem would be how to have the author
> specify the enforcement, apart from that it should apply cleanly,
> introduces a fairly large amount of new functionality and adds very little
> overhead
>
> Ideas?
>
> Jan mentioned this could be used more widely for updates / shows / lists,
> that is starting to sound more complicated and does start introducing
> problems, I would prefer to look at the simple case of filters for changes
> / replication for now, but if there was an even more global solution that
> applied cleanly, I would be happy to
>
>
>