You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Jan Lehnardt <ja...@apache.org> on 2020/07/26 18:28:20 UTC
Per Doc Access
Hey all,
I’m happy to present the first PR worth sharing for introducing per-doc-access control to the 3.x codebase.
https://github.com/apache/couchdb/pull/3038
There are few odds and ends left to do, but this is in good enough shape to get wider review on approach and implementation so far.
My hope would be to include this in a future 3.2.0 release before embarking on reimplementing this for 4.x, which should be considerably simpler.
The PR and linked resources have most of the information relevant to this.
Please review, test and critique heavily, and let me know any questions you might have.
This concludes a couple of weeks worth of effort spread across multiple years. It all started with the developer summit in Boston and Adam’s initial presentation of this design. I hope this makes it justice.
Best
Jan
—
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/
Re: Per Doc Access
Posted by Alessio 'Blaster' Biancalana <do...@gmail.com>.
I think I'm not qualified enough to write down a technical commentary, so
the only thing I can do is congrat with Jan :-D
Alessio
Il ven 21 ago 2020, 20:08 Joan Touzet <wo...@apache.org> ha scritto:
> Comments, in no particular order:
>
> * I like that it's opt-in on a per-database level to create and maintain
> the additional indexes.
>
> * I like that this is an MVP for the feature, one that will get more
> advanced over time.
>
> * I guess we are putting off using maps (vs. records) until 4.x at the
> earlier?
>
> * There's a whole lot of feedback on the RFC from the IBM core team that
> needs to get addressed before that can be merged. Most of it is
> structural, such as Garren's comments, but there are some questions
> from Mike Rhodes to which I haven't seen Jan reply yet. I don't know
> if the PR addresses those or not.
>
> * Obviously Jan needs help on point 9, which I'll start investigating
> later today (after mid-day errands)
>
> * If this is intended to replace db-per-user, we should immediately
> file the deprecation notice on that and prepare to remove it entirely
> in 4.x.
>
> Great to see this move forward!
>
> -Joan "goin' to the bank like an adult" Touzet
>
> On 21/08/2020 08:20, Jan Lehnardt wrote:
> > Hi all, I‘d like to once again solicit feedback from the core team about
> my PR for per doc access control.
> >
> > I know we all have a lot to do, but it’d be great to get some pointers
> on this, so I can gauge how much work it‘ll be to take over the finish line.
> >
> > If it helps any, I‘d be happy to set up a video call to walk folks
> through the main parts.
> >
> > I understand that a lot of Cloudant folks are focused on 4.x, but when
> we last talked, we deemed this feature important enough for 3.x, so I built
> that first. The experience from building this suggests to me that’s 4.x
> port should be fairly straightforward, and that that port should even make
> it easy to add the much desired addition of group sharing.
> >
> > I’m equally happy to take silence as approval, in which case all I ask
> for is a thumbs up, at which point, I‘ll plow through the remaining todos
> and get this out asap.
> >
> > Best
> > Jan
> > —
> >
> >> On 3. Aug 2020, at 17:29, Jan Lehnardt <ja...@apache.org> wrote:
> >>
> >> *bump* Hey all, it’d be great to get at least some cursory feedback on
> this.
> >>
> >> Best
> >> Jan
> >> —
> >>
> >>>>> On 26. Jul 2020, at 20:28, Jan Lehnardt <ja...@apache.org> wrote:
> >>> Hey all,
> >>> I’m happy to present the first PR worth sharing for introducing
> per-doc-access control to the 3.x codebase.
> >>> https://github.com/apache/couchdb/pull/3038
> >>> There are few odds and ends left to do, but this is in good enough
> shape to get wider review on approach and implementation so far.
> >>> My hope would be to include this in a future 3.2.0 release before
> embarking on reimplementing this for 4.x, which should be considerably
> simpler.
> >>> The PR and linked resources have most of the information relevant to
> this.
> >>> Please review, test and critique heavily, and let me know any
> questions you might have.
> >>> This concludes a couple of weeks worth of effort spread across
> multiple years. It all started with the developer summit in Boston and
> Adam’s initial presentation of this design. I hope this makes it justice.
> >>> Best
> >>> Jan
> >>> —
> >>> Professional Support for Apache CouchDB:
> >>> https://neighbourhood.ie/couchdb-support/
> >
>
Re: Per Doc Access
Posted by Joan Touzet <wo...@apache.org>.
Comments, in no particular order:
* I like that it's opt-in on a per-database level to create and maintain
the additional indexes.
* I like that this is an MVP for the feature, one that will get more
advanced over time.
* I guess we are putting off using maps (vs. records) until 4.x at the
earlier?
* There's a whole lot of feedback on the RFC from the IBM core team that
needs to get addressed before that can be merged. Most of it is
structural, such as Garren's comments, but there are some questions
from Mike Rhodes to which I haven't seen Jan reply yet. I don't know
if the PR addresses those or not.
* Obviously Jan needs help on point 9, which I'll start investigating
later today (after mid-day errands)
* If this is intended to replace db-per-user, we should immediately
file the deprecation notice on that and prepare to remove it entirely
in 4.x.
Great to see this move forward!
-Joan "goin' to the bank like an adult" Touzet
On 21/08/2020 08:20, Jan Lehnardt wrote:
> Hi all, I‘d like to once again solicit feedback from the core team about my PR for per doc access control.
>
> I know we all have a lot to do, but it’d be great to get some pointers on this, so I can gauge how much work it‘ll be to take over the finish line.
>
> If it helps any, I‘d be happy to set up a video call to walk folks through the main parts.
>
> I understand that a lot of Cloudant folks are focused on 4.x, but when we last talked, we deemed this feature important enough for 3.x, so I built that first. The experience from building this suggests to me that’s 4.x port should be fairly straightforward, and that that port should even make it easy to add the much desired addition of group sharing.
>
> I’m equally happy to take silence as approval, in which case all I ask for is a thumbs up, at which point, I‘ll plow through the remaining todos and get this out asap.
>
> Best
> Jan
> —
>
>> On 3. Aug 2020, at 17:29, Jan Lehnardt <ja...@apache.org> wrote:
>>
>> *bump* Hey all, it’d be great to get at least some cursory feedback on this.
>>
>> Best
>> Jan
>> —
>>
>>>>> On 26. Jul 2020, at 20:28, Jan Lehnardt <ja...@apache.org> wrote:
>>> Hey all,
>>> I’m happy to present the first PR worth sharing for introducing per-doc-access control to the 3.x codebase.
>>> https://github.com/apache/couchdb/pull/3038
>>> There are few odds and ends left to do, but this is in good enough shape to get wider review on approach and implementation so far.
>>> My hope would be to include this in a future 3.2.0 release before embarking on reimplementing this for 4.x, which should be considerably simpler.
>>> The PR and linked resources have most of the information relevant to this.
>>> Please review, test and critique heavily, and let me know any questions you might have.
>>> This concludes a couple of weeks worth of effort spread across multiple years. It all started with the developer summit in Boston and Adam’s initial presentation of this design. I hope this makes it justice.
>>> Best
>>> Jan
>>> —
>>> Professional Support for Apache CouchDB:
>>> https://neighbourhood.ie/couchdb-support/
>
Re: Per Doc Access
Posted by Jan Lehnardt <ja...@apache.org>.
Hi all, I‘d like to once again solicit feedback from the core team about my PR for per doc access control.
I know we all have a lot to do, but it’d be great to get some pointers on this, so I can gauge how much work it‘ll be to take over the finish line.
If it helps any, I‘d be happy to set up a video call to walk folks through the main parts.
I understand that a lot of Cloudant folks are focused on 4.x, but when we last talked, we deemed this feature important enough for 3.x, so I built that first. The experience from building this suggests to me that’s 4.x port should be fairly straightforward, and that that port should even make it easy to add the much desired addition of group sharing.
I’m equally happy to take silence as approval, in which case all I ask for is a thumbs up, at which point, I‘ll plow through the remaining todos and get this out asap.
Best
Jan
—
> On 3. Aug 2020, at 17:29, Jan Lehnardt <ja...@apache.org> wrote:
>
> *bump* Hey all, it’d be great to get at least some cursory feedback on this.
>
> Best
> Jan
> —
>
>>>> On 26. Jul 2020, at 20:28, Jan Lehnardt <ja...@apache.org> wrote:
>> Hey all,
>> I’m happy to present the first PR worth sharing for introducing per-doc-access control to the 3.x codebase.
>> https://github.com/apache/couchdb/pull/3038
>> There are few odds and ends left to do, but this is in good enough shape to get wider review on approach and implementation so far.
>> My hope would be to include this in a future 3.2.0 release before embarking on reimplementing this for 4.x, which should be considerably simpler.
>> The PR and linked resources have most of the information relevant to this.
>> Please review, test and critique heavily, and let me know any questions you might have.
>> This concludes a couple of weeks worth of effort spread across multiple years. It all started with the developer summit in Boston and Adam’s initial presentation of this design. I hope this makes it justice.
>> Best
>> Jan
>> —
>> Professional Support for Apache CouchDB:
>> https://neighbourhood.ie/couchdb-support/
Re: Per Doc Access
Posted by Jan Lehnardt <ja...@apache.org>.
> On 4. Aug 2020, at 16:03, Bessenyei Balázs Donát <be...@apache.org> wrote:
>
> On Tue, 4 Aug 2020 at 13:10, Jan Lehnardt <ja...@apache.org> wrote:
>>
>> Ah, there might be a misconception. Per-doc-access databases are not “more secure”
>> than regular databases. They are a trade-off between additional access-control for
>> additional CPU and disk resources. But it’s not a case of having a regular db-as-
>> we-know-and-use-it-today and enabling per-doc-access and now it is more secure,
>> it behaves differently and your app needs to account for that.
>
> I didn't mean it would make the product more secure out-of-the-box. I
> was just referring to the principle of least privilege ([1]) - as in
> people would not be able to create "free for all" databases by
> accident (forgetting to supply the enable flag). Please let me know if
> I misunderstood the feature somehow.
Happy to clarify ;)
I see where you are coming from, but I think the nature of the feature is more:
- I accept the trade-offs for getting advanced access control feature
rather than
- All new databases should be set up this way
The main thrust of this feature is to make the db-per-user pattern obsolete.
If you use CouchDB without db-per-user, then you won’t get much benefits from
per-doc-access.
Best
Jan
—
>
>> I don’t mind adding a global off switch that overrides the on-when-specified case
>> to disable all per-doc-access creations.
>
> Awesome, thank you!
>
>
> Donat
>
> [1] https://en.wikipedia.org/wiki/Principle_of_least_privilege
Re: Per Doc Access
Posted by Bessenyei Balázs Donát <be...@apache.org>.
On Tue, 4 Aug 2020 at 13:10, Jan Lehnardt <ja...@apache.org> wrote:
>
> Ah, there might be a misconception. Per-doc-access databases are not “more secure”
> than regular databases. They are a trade-off between additional access-control for
> additional CPU and disk resources. But it’s not a case of having a regular db-as-
> we-know-and-use-it-today and enabling per-doc-access and now it is more secure,
> it behaves differently and your app needs to account for that.
I didn't mean it would make the product more secure out-of-the-box. I
was just referring to the principle of least privilege ([1]) - as in
people would not be able to create "free for all" databases by
accident (forgetting to supply the enable flag). Please let me know if
I misunderstood the feature somehow.
> I don’t mind adding a global off switch that overrides the on-when-specified case
> to disable all per-doc-access creations.
Awesome, thank you!
Donat
[1] https://en.wikipedia.org/wiki/Principle_of_least_privilege
Re: Per Doc Access
Posted by Jan Lehnardt <ja...@apache.org>.
> On 4. Aug 2020, at 13:01, Bessenyei Balázs Donát <be...@apache.org> wrote:
>
> On Tue, 4 Aug 2020 at 12:34, Jan Lehnardt <ja...@apache.org> wrote:
>>
>> The Erlang tests are already exclusively using the HTTP API. I don’t plan
>> to rewrite those to Elixir, but documentation on how to use this will be
>> written before this is merged.
>
> The documentation before merge sounds like a good idea, thank you for that!
>
>>
>>> - what do you think about making this feature toggleable via an ini option?
>>
>> I could be persuaded to provide a default toggle like we have for `q` as an
>> ini option, but I want to make sure people know this is opt-in behaviour, so
>> I’m on the fence on allowing this to be the default on database creation.
>
> Are you suggesting a three-state flag, such as "always-on" (for the
> security-conscious people and environments),
> "per-db-user-defined-on-create-but-off-when-unspecified" (opt-in, most
> flexible), "always-off" ("compatibility mode")?
> That would be neat.
Ah, there might be a misconception. Per-doc-access databases are not “more secure”
than regular databases. They are a trade-off between additional access-control for
additional CPU and disk resources. But it’s not a case of having a regular db-as-
we-know-and-use-it-today and enabling per-doc-access and now it is more secure,
it behaves differently and your app needs to account for that.
For that reason, I’ve chosen your middle option: off-when-unspecified, on-when-
specified.
I don’t mind adding a global off switch that overrides the on-when-specified case
to disable all per-doc-access creations.
Best
Jan
—
>
>
> Thank you,
>
> Donat
Re: Per Doc Access
Posted by Bessenyei Balázs Donát <be...@apache.org>.
On Tue, 4 Aug 2020 at 12:34, Jan Lehnardt <ja...@apache.org> wrote:
>
> The Erlang tests are already exclusively using the HTTP API. I don’t plan
> to rewrite those to Elixir, but documentation on how to use this will be
> written before this is merged.
The documentation before merge sounds like a good idea, thank you for that!
>
> > - what do you think about making this feature toggleable via an ini option?
>
> I could be persuaded to provide a default toggle like we have for `q` as an
> ini option, but I want to make sure people know this is opt-in behaviour, so
> I’m on the fence on allowing this to be the default on database creation.
Are you suggesting a three-state flag, such as "always-on" (for the
security-conscious people and environments),
"per-db-user-defined-on-create-but-off-when-unspecified" (opt-in, most
flexible), "always-off" ("compatibility mode")?
That would be neat.
Thank you,
Donat
Re: Per Doc Access
Posted by Jan Lehnardt <ja...@apache.org>.
Thank you for the review :)
> On 4. Aug 2020, at 12:25, Bessenyei Balázs Donát <be...@apache.org> wrote:
>
> Hey Jan,
>
> I've skimmed through the PR and the RFC and it looks good to me on a first read!
> I have two questions (requests, maybe) though:
> - do you think adding some "system-level" (elixir, I guess) tests
> would be valuable so that a more formalized specification (and
> verification) of the behavior is available? (They certainly help me
> understand functionality a lot.)
The Erlang tests are already exclusively using the HTTP API. I don’t plan
to rewrite those to Elixir, but documentation on how to use this will be
written before this is merged.
> - what do you think about making this feature toggleable via an ini option?
I could be persuaded to provide a default toggle like we have for `q` as an
ini option, but I want to make sure people know this is opt-in behaviour, so
I’m on the fence on allowing this to be the default on database creation.
> Also, I've noticed there are a couple of TODO comments in the code.
> Are you planning to remove them before merging?
Absolutely :)
Best
Jan
—
> Anyway, I'm (non-binding) +1 on the change!
>
>
> Thank you,
>
> Donat
>
> On Mon, 3 Aug 2020 at 17:29, Jan Lehnardt <ja...@apache.org> wrote:
>>
>> *bump* Hey all, it’d be great to get at least some cursory feedback on this.
>>
>> Best
>> Jan
>> —
>>
>>> On 26. Jul 2020, at 20:28, Jan Lehnardt <ja...@apache.org> wrote:
>>>
>>> Hey all,
>>>
>>> I’m happy to present the first PR worth sharing for introducing per-doc-access control to the 3.x codebase.
>>>
>>> https://github.com/apache/couchdb/pull/3038
>>>
>>> There are few odds and ends left to do, but this is in good enough shape to get wider review on approach and implementation so far.
>>>
>>> My hope would be to include this in a future 3.2.0 release before embarking on reimplementing this for 4.x, which should be considerably simpler.
>>>
>>> The PR and linked resources have most of the information relevant to this.
>>>
>>> Please review, test and critique heavily, and let me know any questions you might have.
>>>
>>> This concludes a couple of weeks worth of effort spread across multiple years. It all started with the developer summit in Boston and Adam’s initial presentation of this design. I hope this makes it justice.
>>>
>>> Best
>>> Jan
>>> —
>>> Professional Support for Apache CouchDB:
>>> https://neighbourhood.ie/couchdb-support/
>>>
>>
Re: Per Doc Access
Posted by Bessenyei Balázs Donát <be...@apache.org>.
Hey Jan,
I've skimmed through the PR and the RFC and it looks good to me on a first read!
I have two questions (requests, maybe) though:
- do you think adding some "system-level" (elixir, I guess) tests
would be valuable so that a more formalized specification (and
verification) of the behavior is available? (They certainly help me
understand functionality a lot.)
- what do you think about making this feature toggleable via an ini option?
Also, I've noticed there are a couple of TODO comments in the code.
Are you planning to remove them before merging?
Anyway, I'm (non-binding) +1 on the change!
Thank you,
Donat
On Mon, 3 Aug 2020 at 17:29, Jan Lehnardt <ja...@apache.org> wrote:
>
> *bump* Hey all, it’d be great to get at least some cursory feedback on this.
>
> Best
> Jan
> —
>
> > On 26. Jul 2020, at 20:28, Jan Lehnardt <ja...@apache.org> wrote:
> >
> > Hey all,
> >
> > I’m happy to present the first PR worth sharing for introducing per-doc-access control to the 3.x codebase.
> >
> > https://github.com/apache/couchdb/pull/3038
> >
> > There are few odds and ends left to do, but this is in good enough shape to get wider review on approach and implementation so far.
> >
> > My hope would be to include this in a future 3.2.0 release before embarking on reimplementing this for 4.x, which should be considerably simpler.
> >
> > The PR and linked resources have most of the information relevant to this.
> >
> > Please review, test and critique heavily, and let me know any questions you might have.
> >
> > This concludes a couple of weeks worth of effort spread across multiple years. It all started with the developer summit in Boston and Adam’s initial presentation of this design. I hope this makes it justice.
> >
> > Best
> > Jan
> > —
> > Professional Support for Apache CouchDB:
> > https://neighbourhood.ie/couchdb-support/
> >
>
Re: Per Doc Access
Posted by Jan Lehnardt <ja...@apache.org>.
*bump* Hey all, it’d be great to get at least some cursory feedback on this.
Best
Jan
—
> On 26. Jul 2020, at 20:28, Jan Lehnardt <ja...@apache.org> wrote:
>
> Hey all,
>
> I’m happy to present the first PR worth sharing for introducing per-doc-access control to the 3.x codebase.
>
> https://github.com/apache/couchdb/pull/3038
>
> There are few odds and ends left to do, but this is in good enough shape to get wider review on approach and implementation so far.
>
> My hope would be to include this in a future 3.2.0 release before embarking on reimplementing this for 4.x, which should be considerably simpler.
>
> The PR and linked resources have most of the information relevant to this.
>
> Please review, test and critique heavily, and let me know any questions you might have.
>
> This concludes a couple of weeks worth of effort spread across multiple years. It all started with the developer summit in Boston and Adam’s initial presentation of this design. I hope this makes it justice.
>
> Best
> Jan
> —
> Professional Support for Apache CouchDB:
> https://neighbourhood.ie/couchdb-support/
>