[jira] [Commented] (COUCHDB-1425) Emitting UTF-8 chars >= 0xD800 in JS map stops design doc from indexing

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/COUCHDB-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13592224#comment-13592224 ] 

ASF subversion and git services commented on COUCHDB-1425:
----------------------------------------------------------

Commit be4e41ff27a9c5ac270e24dcf2b3fca26a938149 in branch refs/heads/1425-fix-graceful-surrogate-handling from [~janl]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb.git;h=be4e41f ]

Handle invalid UTF-8 byte sequences gracefully by replacing them with '?'

CouchDB's Erlang JSON parser allows storing of invalid UTF-8 byte sequences.
The Query Server inside CouchDB fails upon necountering these byte sequences.
The view process fails for the current batch of document updates. The result
is that the view is invariably broken. Only removing the document in question
solves this otherwise, but finding that is hard as the `log()` inside the
Query Server dies with the invalid byte sequence because our protocol is
synchronous and map results an `log()` messages generated therein are
submitted together.

This patch replaces invalid bytes with the the '?' (0x3f) byte.

Closes COUCHDB-1425.

Patch by Sam Rijs <re...@awesan.de>

Eventually, this should be fixed at the HTTP level, so that no documents
with invalid byte sequences can be written to CouchDB. The jiffy encoder
we'll get with BigCouch will do that for us. This is a fix for the releases
until then.

                
> Emitting UTF-8 chars >= 0xD800 in JS map stops design doc from indexing
> -----------------------------------------------------------------------
>
>                 Key: COUCHDB-1425
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1425
>             Project: CouchDB
>          Issue Type: Bug
>          Components: JavaScript View Server
>    Affects Versions: 1.1.1
>         Environment: Mac OS 10.6.8, but not sure that matters.
>            Reporter: Jim Klo
>         Attachments: utf8.c.diff
>
>
> Was trying determine UTF-8 Char collation, using the following Gist: https://gist.github.com/1904807
> It turns out that once the view gets to the document that would emit "\uD800", the view server times out and stops indexing that design document.
> This seems like a bug, since I can 'store' a document with UTF-8 chars >= 0xD800, but one cannot emit a key with that char in the string.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira