0.11 Release / Feature Freeze for 1.0

[Chris Anderson <jc...@apache.org>]

Devs,

I propose that we start the gears turning for a 0.11 release. We'd
talked about Feb 1 as a feature-freeze deadline for 1.0 -- we'd like
to spend the time between now and 1.0 getting feedback about
reliability and squashing bugs. I'm ready to consider what we have in
CouchDB trunk a release candidate.

The one new feature I'd be happy to see during the freeze is the
replication configuration database. I don't see there being much
chance of destabilizing the replicator by keeping it's state in a
database.

If anyone has any other new feature they'd like to commit to trunk
before 1.0, please let us know. I think it is the project's interest
that we don't add more API in the coming weeks. When the replicator
work is done we might even wait until 1.1 to ship it, that's up to the
community to decide.

Chris


-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Re: 0.11 Release / Feature Freeze for 1.0

[Brian Candler <B....@pobox.com>]

I see the readeracl branch was recently merged into trunk, and I've just
been testing it again.

My concern is that the design is flawed, and that if this goes into 0.11
then we are stuck with it forever; so it's better to address these sooner
rather than later.

I do see logic in keeping the admin/reader authorizations for a database
within the database itself. The problems are:

(1) _all_dbs currently shows everything - even those databases you don't
have access to. This leads to the following issues:

* Users won't want other users to be able to see the names of their
  databases, for privacy reasons. (Imagine what would happen if github
  revealed the names of all the private repos on it, even if you couldn't
  access the contents)

* I don't want users to know many databases I have on my box, for commercial
  reasons. (Ditto for github).

* I don't want users to have to page through loads of databases they can't
  access

* Futon constantly pops up errors about "Database information could not be
  retrieved" (although that one's easily fixable)

I believe that in its current form, _all_dbs simply won't scale to millions
of databases on a box if you want to limit it to accessible dbs only.

(2) _readers is a single monolithic object. I believe that it won't scale to
millions of users having access to the same database.

(3) _readers has no concurrency control. One admin making an ACL change in
futon (say) will silently overwrite changes made around the same time by
another admin. This will get worse the more frequently users are added and
removed.

For me, those are serious problems. I sketched a design for an alternative
approach, using the _user db to hold the authorizations in terms of
database-specific roles.  Unfortunately I didn't have time to contribute an
implementation of this.  If there's a chance this alternative approach would
be used then I will try to steal the time from somewhere.  The ideas behind
it weren't explicitly rejected, but neither were they acknowledged as a good
approach.

If the current design stays, then I think there will be sticking-plaster
solutions forever; e.g. proxies to fake out _all_dbs and ACL changes,
mapping them to a 'real' database behind.

Right, I've said all that. Now I have a few further observations from the
current implementation.

(4) An "admin" is not a "reader", and this is clearly intentional from
comments in the code. However, someone who has an "admin" role without
"reader" role is unable to perform ACL changes, which for me defeats the
whole purpose of the "admin" role.

Example: user "brianadmin" is in "_admins" on database "briantest", but not
in "_readers":

$ curl http://admin:admin@127.0.0.1:5984/briantest/_admins
{"names":["brianadmin"],"roles":[]}
$ curl http://admin:admin@127.0.0.1:5984/briantest/_readers
{"names":["brian"],"roles":[]}

But when "brianadmin" tries to update an ACL, here's what happens:

$ curl http://brianadmin:brianadmin@127.0.0.1:5984/briantest/_readers
{"error":"unauthorized","reason":"You are not authorized to access this db."}
$ curl -X PUT -d '{"names":["foo","brian"],"roles":[]}'
http://brianadmin:brianadmin@127.0.0.1:5984/briantest/_readers
{"error":"unauthorized","reason":"You are not authorized to access this db."}

Even if this were fixed so that a db admin had access to _readers and
_admins resources (and design docs), I think that in practice a database
administrator would be expected to have access to the database she is
administering. In that case, adding the same user to both "admin" and
"reader" roles simply involves duplicating data, as well as having to
remember to remove that user from two places when she leaves. That
introduces more scope for error.

So I'd propose that the relaxed approach is that a database "admin" should
inherit "reader" rights. Isn't that true for a server-level admin anyway?

(5) Non-admin readers can view the entire _readers, _admins and _security
resources.  I think this is quite a severe privacy concern, but it is easily
fixed.

(6) Databases are created world-readable by default, which means a race to
get the _readers set before someone else starts inserting documents. I think
a PUT /dbname option to set a non-empty readers list would be a good idea
(and a corresponding checkbox in futon)

(7) Couchdb accepts nonsense _readers documents, e.g.

$ curl -X PUT -d '{"names":{"foo":"bar"},"roles":456}'
http://admin:admin@127.0.0.1:5984/briantest/_readers
{"ok":true}

The effect is to reset the _readers document to its permit-all default, thus
opening up the database to the world.

$ curl http://127.0.0.1:5984/briantest/_readers
{"names":[],"roles":[]}

(8) Point (7) is arguably a simple bug which can be fixed, but I'd prefer
for couchdb to be fail-safe; that is, an empty ACL means nobody has access.

One way to achieve this would be for two new roles, "_anon" and "_user",
granted to all unauthenticated and authenticated users respectively. Then a
fully public database would have roles:["_anon","_user"], and this would
be added to a new database unless you ask otherwise (see point (6)).

(9) The _users db itself is world-readable (showing not only who your users
are, but their password hashes). Highly undesirable.

You can set a _readers ACL on it, but it has consequences:
* users can't sign up for new accounts
* users can't change their own passwords
This forces such things through a privileged external interface. Actually
I'm fine with that, because I want to validate signups anyway, but others
might not be.

(10) I don't think you can replicate _readers, _admins and _security, unless
(a) you are doing rsync filesystem-level replication, or (b) you explicitly
GET and PUT these resources from one DB to another. This is arguably a
feature not a bug.

(11) Trivial problem of the day: _security resources which are not objects
give an erlang-flavoured error.

$ curl -X PUT -d '["foo","bar"]' http://brianadmin:brianadmin@127.0.0.1:5984/briantest/_security
{"error":"unknown_error","reason":"function_clause"}

Sorry for the long post, and if I really am barking up the wrong tree here,
please tell me so.

Regards,

Brian.

Re: 0.11 Release / Feature Freeze for 1.0

[Chris Anderson <jc...@apache.org>]

On Mon, Feb 1, 2010 at 11:31 PM, Dirkjan Ochtman <dj...@gmail.com> wrote:
> On Tue, Feb 2, 2010 at 06:27, Chris Anderson <jc...@apache.org> wrote:
>> I propose that we start the gears turning for a 0.11 release. We'd
>> talked about Feb 1 as a feature-freeze deadline for 1.0 -- we'd like
>> to spend the time between now and 1.0 getting feedback about
>> reliability and squashing bugs. I'm ready to consider what we have in
>> CouchDB trunk a release candidate.
>>
>> The one new feature I'd be happy to see during the freeze is the
>> replication configuration database. I don't see there being much
>> chance of destabilizing the replicator by keeping it's state in a
>> database.
>>
>> If anyone has any other new feature they'd like to commit to trunk
>> before 1.0, please let us know. I think it is the project's interest
>> that we don't add more API in the coming weeks. When the replicator
>> work is done we might even wait until 1.1 to ship it, that's up to the
>> community to decide.
>
> Pretty short notice for 0.11. I'd hoped to get my view options work
> for Futon in.


Sorry about that -- it was mentioned on the "Roadmap" thread from a
few months ago. I felt like I should have written a mail earlier but
I've been busy writing patches.

If your view options are close I'm fine waiting.

The point of the freeze is especially to lock in the API and low-level
storage details. Clean patches to Futon shouldn't pose a problem
during the run up to 1.0, at least in my opinion. So if you aren't
quite ready with the Futon you can do it in the weeks before 1.0

FYI ideally we'd release 1.0 in early March, which should be enough
time for the 0.11 dust to settle.

Chris

>
> Cheers,
>
> Dirkjan
>



-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Re: 0.11 Release / Feature Freeze for 1.0

[Dirkjan Ochtman <dj...@gmail.com>]

On Tue, Feb 2, 2010 at 06:27, Chris Anderson <jc...@apache.org> wrote:
> I propose that we start the gears turning for a 0.11 release. We'd
> talked about Feb 1 as a feature-freeze deadline for 1.0 -- we'd like
> to spend the time between now and 1.0 getting feedback about
> reliability and squashing bugs. I'm ready to consider what we have in
> CouchDB trunk a release candidate.
>
> The one new feature I'd be happy to see during the freeze is the
> replication configuration database. I don't see there being much
> chance of destabilizing the replicator by keeping it's state in a
> database.
>
> If anyone has any other new feature they'd like to commit to trunk
> before 1.0, please let us know. I think it is the project's interest
> that we don't add more API in the coming weeks. When the replicator
> work is done we might even wait until 1.1 to ship it, that's up to the
> community to decide.

Pretty short notice for 0.11. I'd hoped to get my view options work
for Futon in.

Cheers,

Dirkjan