You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Filipe Manana (JIRA)" <ji...@apache.org> on 2010/02/19 19:40:30 UTC
[jira] Created: (COUCHDB-661) _all_dbs should list only the DBs
accessible to the user
_all_dbs should list only the DBs accessible to the user
--------------------------------------------------------
Key: COUCHDB-661
URL: https://issues.apache.org/jira/browse/COUCHDB-661
Project: CouchDB
Issue Type: Bug
Components: HTTP Interface
Affects Versions: 0.11
Environment: trunk / 0.11
Reporter: Filipe Manana
Fix For: 0.11
Attachments: couchdb-_all_dbs-auth.patch
As discussed in the auth roadmap mail, sent by Chris to @dev, the _all_dbs URI should only list the DBs that are accessible to the user.
The following patch is a naive solution. It doesn't scale for CouchDB servers with millions of DBs. Regarding this scaling detail, I'll discuss soon in the @dev mailing list some ideas.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-661) _all_dbs should list only the DBs
accessible to the user
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836462#action_12836462 ]
Filipe Manana commented on COUCHDB-661:
---------------------------------------
Yes Brian, I do share the same vision.
The idea, which I pretend to discuss soon in the @dev mailing list, is to use a view which will map user names and roles to lists of dbs. This view would exist in the design doc of a special db named "_dbs" (or whatever).
I will send a mail asap to the dev mailing list presenting a partial solution and pointing out some technical issues, as well as to collect some feeback about them.
cheers
> _all_dbs should list only the DBs accessible to the user
> --------------------------------------------------------
>
> Key: COUCHDB-661
> URL: https://issues.apache.org/jira/browse/COUCHDB-661
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 0.11
> Environment: trunk / 0.11
> Reporter: Filipe Manana
> Fix For: 0.11
>
> Attachments: couchdb-_all_dbs-auth-2.patch, couchdb-_all_dbs-auth.patch
>
>
> As discussed in the auth roadmap mail, sent by Chris to @dev, the _all_dbs URI should only list the DBs that are accessible to the user.
> The following patch is a naive solution. It doesn't scale for CouchDB servers with millions of DBs. Regarding this scaling detail, I'll discuss soon in the @dev mailing list some ideas.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-661) _all_dbs should list only the DBs
accessible to the user
Posted by "Noah Slater (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Noah Slater updated COUCHDB-661:
--------------------------------
Fix Version/s: (was: 0.11)
0.12
> _all_dbs should list only the DBs accessible to the user
> --------------------------------------------------------
>
> Key: COUCHDB-661
> URL: https://issues.apache.org/jira/browse/COUCHDB-661
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 0.11
> Environment: trunk / 0.11
> Reporter: Filipe Manana
> Fix For: 0.12
>
> Attachments: couchdb-_all_dbs-auth-2.patch, couchdb-_all_dbs-auth.patch
>
>
> As discussed in the auth roadmap mail, sent by Chris to @dev, the _all_dbs URI should only list the DBs that are accessible to the user.
> The following patch is a naive solution. It doesn't scale for CouchDB servers with millions of DBs. Regarding this scaling detail, I'll discuss soon in the @dev mailing list some ideas.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-661) _all_dbs should list only the DBs
accessible to the user
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana updated COUCHDB-661:
----------------------------------
Attachment: couchdb-_all_dbs-auth-2.patch
Ops, forgot to include the adaptations necessary to test/etap/070-couch-db.t
> _all_dbs should list only the DBs accessible to the user
> --------------------------------------------------------
>
> Key: COUCHDB-661
> URL: https://issues.apache.org/jira/browse/COUCHDB-661
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 0.11
> Environment: trunk / 0.11
> Reporter: Filipe Manana
> Fix For: 0.11
>
> Attachments: couchdb-_all_dbs-auth-2.patch, couchdb-_all_dbs-auth.patch
>
>
> As discussed in the auth roadmap mail, sent by Chris to @dev, the _all_dbs URI should only list the DBs that are accessible to the user.
> The following patch is a naive solution. It doesn't scale for CouchDB servers with millions of DBs. Regarding this scaling detail, I'll discuss soon in the @dev mailing list some ideas.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-661) _all_dbs should list only the DBs
accessible to the user
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Joseph Davis updated COUCHDB-661:
--------------------------------------
Skill Level: Regular Contributors Level (Easy to Medium)
> _all_dbs should list only the DBs accessible to the user
> --------------------------------------------------------
>
> Key: COUCHDB-661
> URL: https://issues.apache.org/jira/browse/COUCHDB-661
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 0.11
> Environment: trunk / 0.11
> Reporter: Filipe Manana
> Fix For: 0.12
>
> Attachments: couchdb-_all_dbs-auth-2.patch, couchdb-_all_dbs-auth.patch
>
>
> As discussed in the auth roadmap mail, sent by Chris to @dev, the _all_dbs URI should only list the DBs that are accessible to the user.
> The following patch is a naive solution. It doesn't scale for CouchDB servers with millions of DBs. Regarding this scaling detail, I'll discuss soon in the @dev mailing list some ideas.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-661) _all_dbs should list only the DBs
accessible to the user
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836408#action_12836408 ]
Filipe Manana commented on COUCHDB-661:
---------------------------------------
So, testing this with with a server having 1000 DBs, each one with a size of about 1Mb and 100 docs, the response time for _all_dbs is about 0.5s on my system (Ubuntu 9.10, SATA hd 7200rpm)
The DB was populated with the tool at http://github.com/fdmanana/seatoncouch using the following doc template:
{
"_id": "doc#{doc_id_counter}",
"name": "#{random_string(100)}",
"address": "#{random_string(200)}",
"age": #{random_int(1, 100)},
"children": #{random_int(0, 10)},
"bio": "#{random_string(10000)}"
}
Each DB has the security doc:
{
"admins": {
"names": ["joe"],
"roles": ["test_admin", "superuser"]
},
"readers": {
"names": ["fdmanana"],
"roles": []
}
}
$ time ./seatoncouch.rb --doc-tpl fdmanana_doc.tpl --security-doc security_doc.json --dbs 1000 --docs 100
[INFO] Created DB named testdb1
[INFO] Created doc at /testdb1/doc1
[INFO] Created doc at /testdb1/doc2
# etc... takes more than 1 hour
Measuring the time:
$ time curl http://localhost:5984/_all_dbs
[
"testdb2",
"testdb485",
"testdb497",
# etc...
real 0m0.498s
user 0m0.000s
sys 0m0.010s
Increasing the number of DBs to 7500:
$ time ./seatoncouch.rb --doc-tpl fdmanana_doc.tpl --security-doc security_doc.json --dbs 6500 --docs 100 --db-start-id 1001
...
$ time curl http://localhost:5984/_all_dbs 2> /dev/null | wc -l
7502
real 0m3.763s
user 0m0.010s
sys 0m0.090s
$ time curl http://localhost:5984/_all_dbs 2> /dev/null | wc -l
7502
real 0m3.804s
user 0m0.020s
sys 0m0.060s
$ time curl http://localhost:5984/_all_dbs 2> /dev/null | wc -l
7502
real 0m3.714s
user 0m0.020s
sys 0m0.100s
> _all_dbs should list only the DBs accessible to the user
> --------------------------------------------------------
>
> Key: COUCHDB-661
> URL: https://issues.apache.org/jira/browse/COUCHDB-661
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 0.11
> Environment: trunk / 0.11
> Reporter: Filipe Manana
> Fix For: 0.11
>
> Attachments: couchdb-_all_dbs-auth-2.patch, couchdb-_all_dbs-auth.patch
>
>
> As discussed in the auth roadmap mail, sent by Chris to @dev, the _all_dbs URI should only list the DBs that are accessible to the user.
> The following patch is a naive solution. It doesn't scale for CouchDB servers with millions of DBs. Regarding this scaling detail, I'll discuss soon in the @dev mailing list some ideas.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-661) _all_dbs should list only the DBs
accessible to the user
Posted by "Brian Candler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836454#action_12836454 ]
Brian Candler commented on COUCHDB-661:
---------------------------------------
Looks like a serious DoS to me, with "only" 7500 databases.
If _all_dbs won't scale, then I think it should be for admins only (ideally with startkey/limit like _all_docs for efficient pagination, but that's a different issue)
Or perhaps it should be possible to replace _all_dbs with a view in a 'real' database for non-admins.
e.g. occasionally you could copy all the _security objects into another database, and generate a view with keys like
emit(['name',name],db)
emit(['role',role],db)
for efficient querying.
(IMHO this is another reason why _security objects should be real docs: so that you can follow a _changes feed on them)
> _all_dbs should list only the DBs accessible to the user
> --------------------------------------------------------
>
> Key: COUCHDB-661
> URL: https://issues.apache.org/jira/browse/COUCHDB-661
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 0.11
> Environment: trunk / 0.11
> Reporter: Filipe Manana
> Fix For: 0.11
>
> Attachments: couchdb-_all_dbs-auth-2.patch, couchdb-_all_dbs-auth.patch
>
>
> As discussed in the auth roadmap mail, sent by Chris to @dev, the _all_dbs URI should only list the DBs that are accessible to the user.
> The following patch is a naive solution. It doesn't scale for CouchDB servers with millions of DBs. Regarding this scaling detail, I'll discuss soon in the @dev mailing list some ideas.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-661) _all_dbs should list only the DBs
accessible to the user
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana updated COUCHDB-661:
----------------------------------
Attachment: couchdb-_all_dbs-auth.patch
> _all_dbs should list only the DBs accessible to the user
> --------------------------------------------------------
>
> Key: COUCHDB-661
> URL: https://issues.apache.org/jira/browse/COUCHDB-661
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 0.11
> Environment: trunk / 0.11
> Reporter: Filipe Manana
> Fix For: 0.11
>
> Attachments: couchdb-_all_dbs-auth.patch
>
>
> As discussed in the auth roadmap mail, sent by Chris to @dev, the _all_dbs URI should only list the DBs that are accessible to the user.
> The following patch is a naive solution. It doesn't scale for CouchDB servers with millions of DBs. Regarding this scaling detail, I'll discuss soon in the @dev mailing list some ideas.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.