You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2021/05/16 10:56:38 UTC
[GitHub] [couchdb] flimzy opened a new issue #3565: CouchDB unescaping URL paths incorrectly
flimzy opened a new issue #3565:
URL: https://github.com/apache/couchdb/issues/3565
[NOTE]: # ( ^^ Provide a general summary of the issue in the title above. ^^ )
## Description
CouchDB appears to be applying URL unescaping logic intended for query parameters to the path. This leads to problems with `+` signs.
Example (with Couch 3.1.1, but I've observed the same with 2.3 and 3.0):
```shell
$ export COUCH_USER=admin
$ export COUCH_PASSWORD=abc123
$ docker run --name couch31 -p 6005:5984/tcp -d --rm -e COUCHDB_USER -e COUCHDB_PASSWORD apache/couchdb:3.1.1
aa9d21fa02bce29857179cf170b5e756f6a3439cc592369a6830842110ff89af
$ curl -X PUT http://admin:abc123@localhost:6005/foo
{"ok":true}
$ curl -X PUT http://admin:abc123@localhost:6005/foo/foo+bar -H 'Content-Type: application/json' -d '{}'
{"ok":true,"id":"foo bar","rev":"1-967a00dff5e02add41819138abb3284d"}
$ curl http://admin:abc123@localhost:6005/foo/_all_docs
{"total_rows":1,"offset":0,"rows":[
{"id":"foo bar","key":"foo bar","value":{"rev":"1-967a00dff5e02add41819138abb3284d"}}
]}
```
As you can see, creating a document with a `PUT` to `/foo/foo+bar` creates a document with `_id` of `foo bar`. But according to [RFC 3986, 2.2](https://datatracker.ietf.org/doc/html/rfc3986#section-2.2), the `+` character has no special meaning in the path segment of a URL, and thus should not be converted to a space (as it would if found in in a query paremter).
> gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@"
>
> A subset of the reserved characters (gen-delims) is used as delimiters of the generic URI components described in Section 3.
And the preamble to section 3:
> The generic URI syntax consists of a hierarchical sequence of components referred to as the scheme, authority, path, query, and fragment.
>
> URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]
>
> hier-part = "//" authority path-abempty
> / path-absolute
> / path-rootless
> / path-empty
## Steps to Reproduce
See above
## Expected Behaviour
I would expect the document to be created with id of `foo+bar`
## Your Environment
[TIP]: # ( Include as many relevant details about your environment as possible. )
[TIP]: # ( You can paste the output of curl http://YOUR-COUCHDB:5984/ here. )
* CouchDB version used: 3.1.1, 3.0.0, 2.3.0
* Browser name and version: curl 7.64.0
* Operating system and version: Debian Linux 10.9
## Additional commentary
Changing this may well be considered a breaking change, since anyone depending on `+` being converted to a space. Maybe it's safe enough to change with the next major release of 4.0. Until then, perhaps a documentation update is in order (or maybe this behavior is already documented, but I haven't found it)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] flimzy closed issue #3565: CouchDB unescaping URL paths incorrectly
Posted by GitBox <gi...@apache.org>.
flimzy closed issue #3565:
URL: https://github.com/apache/couchdb/issues/3565
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] nickva edited a comment on issue #3565: CouchDB unescaping URL paths incorrectly
Posted by GitBox <gi...@apache.org>.
nickva edited a comment on issue #3565:
URL: https://github.com/apache/couchdb/issues/3565#issuecomment-841866009
I think it's coming from mochiweb were it calls [unquote]( https://github.com/mochi/mochiweb/blob/070594e4d66163d662ac7e3bfb75dadcc922dd7c/src/mochiweb_request.erl#L214) on the path but it actually uses the [unquote_plus](https://github.com/mochi/mochiweb/blob/070594e4d66163d662ac7e3bfb75dadcc922dd7c/src/mochiweb_util.erl#L256-L257) decoding logic where a `+` would be turned into a space.
Recently we encountered a somewhat similar issue with userinfo decoding/decoding in replicator in https://github.com/apache/couchdb/pull/3546. Where, according to the RFC, it turns out `+` is a valid userinfo character as well but we would decode it into a space now if we just the use the decode logic in mochiweb.
Agree, not sure what a good way to fix this is in a backwards compatible way. Could be it's a 4.0 feature or we just document that `+` will always be decoded to a space and if you want a `+` in doc id it would have to be url encoded.
For how other libraries handle the issue I found this one pretty well done:
https://boltons.readthedocs.io/en/latest/_modules/boltons/urlutils.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] nickva commented on issue #3565: CouchDB unescaping URL paths incorrectly
Posted by GitBox <gi...@apache.org>.
nickva commented on issue #3565:
URL: https://github.com/apache/couchdb/issues/3565#issuecomment-841866009
I think it's coming from mochiweb were it calls [unquote]( https://github.com/mochi/mochiweb/blob/070594e4d66163d662ac7e3bfb75dadcc922dd7c/src/mochiweb_request.erl#L214) on the path but it actually uses the [unquote_plus](https://github.com/mochi/mochiweb/blob/070594e4d66163d662ac7e3bfb75dadcc922dd7c/src/mochiweb_util.erl#L256-L257) decoding logic where a `+` would be turned into a space.
Recently we encountered a somewhat similar issue with userinfo decoding/decoding in replicator in https://github.com/apache/couchdb/pull/3546. Where, according to the RFC, it turns out `+` is a valid userinfo character as well but we would decode it into a space now if we just the use the decode logic in mochiweb.
Agree, not sure what a good way to fix this is in a backwards compatible way. Could be it's a 4.0 feature or we just document that `+` will always be decoded to a space and if you want a `+` in doc id it would have to be url encoded.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] bessbd commented on issue #3565: CouchDB unescaping URL paths incorrectly
Posted by GitBox <gi...@apache.org>.
bessbd commented on issue #3565:
URL: https://github.com/apache/couchdb/issues/3565#issuecomment-844025759
I think this is a duplicate of https://github.com/apache/couchdb/issues/2235
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] nickva edited a comment on issue #3565: CouchDB unescaping URL paths incorrectly
Posted by GitBox <gi...@apache.org>.
nickva edited a comment on issue #3565:
URL: https://github.com/apache/couchdb/issues/3565#issuecomment-841866009
I think it's coming from mochiweb were it calls [unquote]( https://github.com/mochi/mochiweb/blob/070594e4d66163d662ac7e3bfb75dadcc922dd7c/src/mochiweb_request.erl#L214) on the path but it actually uses the [unquote_plus](https://github.com/mochi/mochiweb/blob/070594e4d66163d662ac7e3bfb75dadcc922dd7c/src/mochiweb_util.erl#L256-L257) decoding logic where a `+` would be turned into a space.
Recently we encountered a somewhat similar issue with userinfo decoding/decoding in replicator in https://github.com/apache/couchdb/pull/3546. Where, according to the RFC, it turns out `+` is a valid userinfo character as well but we would decode it into a space now if we just the use the decode logic in mochiweb.
Agree, not sure what a good way to fix this is in a backwards compatible way. Could be it's a 4.0 feature or we just document that `+` will always be decoded to a space and if you want a `+` in doc id it would have to be url encoded.
For how other libraries handle the issue I found this one pretty well done:
https://boltons.readthedocs.io/en/latest/_modules/boltons/urlutils.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] bessbd commented on issue #3565: CouchDB unescaping URL paths incorrectly
Posted by GitBox <gi...@apache.org>.
bessbd commented on issue #3565:
URL: https://github.com/apache/couchdb/issues/3565#issuecomment-844161884
Can we close this, @flimzy ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] nickva commented on issue #3565: CouchDB unescaping URL paths incorrectly
Posted by GitBox <gi...@apache.org>.
nickva commented on issue #3565:
URL: https://github.com/apache/couchdb/issues/3565#issuecomment-841866009
I think it's coming from mochiweb were it calls [unquote]( https://github.com/mochi/mochiweb/blob/070594e4d66163d662ac7e3bfb75dadcc922dd7c/src/mochiweb_request.erl#L214) on the path but it actually uses the [unquote_plus](https://github.com/mochi/mochiweb/blob/070594e4d66163d662ac7e3bfb75dadcc922dd7c/src/mochiweb_util.erl#L256-L257) decoding logic where a `+` would be turned into a space.
Recently we encountered a somewhat similar issue with userinfo decoding/decoding in replicator in https://github.com/apache/couchdb/pull/3546. Where, according to the RFC, it turns out `+` is a valid userinfo character as well but we would decode it into a space now if we just the use the decode logic in mochiweb.
Agree, not sure what a good way to fix this is in a backwards compatible way. Could be it's a 4.0 feature or we just document that `+` will always be decoded to a space and if you want a `+` in doc id it would have to be url encoded.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] bessbd commented on issue #3565: CouchDB unescaping URL paths incorrectly
Posted by GitBox <gi...@apache.org>.
bessbd commented on issue #3565:
URL: https://github.com/apache/couchdb/issues/3565#issuecomment-844026225
Maybe it's time to fix this :D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] flimzy commented on issue #3565: CouchDB unescaping URL paths incorrectly
Posted by GitBox <gi...@apache.org>.
flimzy commented on issue #3565:
URL: https://github.com/apache/couchdb/issues/3565#issuecomment-844039944
@bessbd I think you're right. On both counts :grin:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org