You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Matthew Gregg (JIRA)" <ji...@apache.org> on 2009/04/21 01:25:47 UTC

[jira] Created: (COUCHDB-327) Replication of document results in invalid_json error

Replication of document results in invalid_json error
-----------------------------------------------------

                 Key: COUCHDB-327
                 URL: https://issues.apache.org/jira/browse/COUCHDB-327
             Project: CouchDB
          Issue Type: Bug
    Affects Versions: 0.9
         Environment: Ubuntu 8.10
            Reporter: Matthew Gregg


While replicating a document containing UTf8 characters replication fails with invalid_json errors in the receiving couch instance.
The test below will cause the replication error.

couchTests.utf8_replication = function(debug) {
  if (debug) debugger;
  var host = CouchDB.host;
  var texts = [];
  var dbPairs = [
    {source:"http://"+host+"/test_suite_db_a",
      target:"http://"+host+"/test_suite_db_b"}
  ]
  var dbA = new CouchDB("test_suite_db_a");
  var dbB = new CouchDB("test_suite_db_b");
  var numDocs = 10;
  var xhr;
  for (var testPair = 0; testPair < dbPairs.length; testPair++) {
    var A = dbPairs[testPair].source
    var B = dbPairs[testPair].target

    dbA.deleteDb();
    dbA.createDb();
    dbB.deleteDb();
    dbB.createDb();

    text = "􀂀 Fakain: 􀂀 Faktor produksi yang digunakan hanya tenaga kerja saja. 􀂀 Kualitas ba";
    dbA.save({_id:"1", text:text})

    var result = CouchDB.replicate(A, B);

  }
};


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (COUCHDB-327) Replication of document results in invalid_json error

Posted by "Matthew Gregg (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/COUCHDB-327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701019#action_12701019 ] 

Matthew Gregg commented on COUCHDB-327:
---------------------------------------

Applying this patch http://code.google.com/p/mochiweb/issues/detail?id=32 solve this issue for me.

> Replication of document results in invalid_json error
> -----------------------------------------------------
>
>                 Key: COUCHDB-327
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-327
>             Project: CouchDB
>          Issue Type: Bug
>    Affects Versions: 0.9
>         Environment: Ubuntu 8.10
>            Reporter: Matthew Gregg
>
> While replicating a document containing UTf8 characters replication fails with invalid_json errors in the receiving couch instance.
> The test below will cause the replication error.
> couchTests.utf8_replication = function(debug) {
>   if (debug) debugger;
>   var host = CouchDB.host;
>   var texts = [];
>   var dbPairs = [
>     {source:"http://"+host+"/test_suite_db_a",
>       target:"http://"+host+"/test_suite_db_b"}
>   ]
>   var dbA = new CouchDB("test_suite_db_a");
>   var dbB = new CouchDB("test_suite_db_b");
>   var numDocs = 10;
>   var xhr;
>   for (var testPair = 0; testPair < dbPairs.length; testPair++) {
>     var A = dbPairs[testPair].source
>     var B = dbPairs[testPair].target
>     dbA.deleteDb();
>     dbA.createDb();
>     dbB.deleteDb();
>     dbB.createDb();
>     text = "􀂀 Fakain: 􀂀 Faktor produksi yang digunakan hanya tenaga kerja saja. 􀂀 Kualitas ba";
>     dbA.save({_id:"1", text:text})
>     var result = CouchDB.replicate(A, B);
>   }
> };

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (COUCHDB-327) Replication of document results in invalid_json error

Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/COUCHDB-327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adam Kocoloski resolved COUCHDB-327.
------------------------------------

    Resolution: Fixed

patch applied to trunk (r 782643) and 0.9.x (r 782645)

> Replication of document results in invalid_json error
> -----------------------------------------------------
>
>                 Key: COUCHDB-327
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-327
>             Project: CouchDB
>          Issue Type: Bug
>    Affects Versions: 0.9
>         Environment: Ubuntu 8.10
>            Reporter: Matthew Gregg
>            Assignee: Adam Kocoloski
>
> While replicating a document containing UTf8 characters replication fails with invalid_json errors in the receiving couch instance.
> The test below will cause the replication error.
> couchTests.utf8_replication = function(debug) {
>   if (debug) debugger;
>   var host = CouchDB.host;
>   var texts = [];
>   var dbPairs = [
>     {source:"http://"+host+"/test_suite_db_a",
>       target:"http://"+host+"/test_suite_db_b"}
>   ]
>   var dbA = new CouchDB("test_suite_db_a");
>   var dbB = new CouchDB("test_suite_db_b");
>   var numDocs = 10;
>   var xhr;
>   for (var testPair = 0; testPair < dbPairs.length; testPair++) {
>     var A = dbPairs[testPair].source
>     var B = dbPairs[testPair].target
>     dbA.deleteDb();
>     dbA.createDb();
>     dbB.deleteDb();
>     dbB.createDb();
>     text = "􀂀 Fakain: 􀂀 Faktor produksi yang digunakan hanya tenaga kerja saja. 􀂀 Kualitas ba";
>     dbA.save({_id:"1", text:text})
>     var result = CouchDB.replicate(A, B);
>   }
> };

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (COUCHDB-327) Replication of document results in invalid_json error

Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/COUCHDB-327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adam Kocoloski updated COUCHDB-327:
-----------------------------------


I also cannot get that particular utf8_replication test to fail, but I agree that there is a problem here.  I believe the root of the problem is mochijson2's inability to decode unicode characters outside the Basic Multilingual Plane that are transmitted escaped as UTF-16 surrogate pairs.  There's a couchdb and a  mochiweb issue for this:

https://issues.apache.org/jira/browse/COUCHDB-333
http://code.google.com/p/mochiweb/issues/detail?id=35

Replication breaks on documents containing raw UTF-8 corresponding to Unicode outside the BMP, because the mochijson2 encoder transmits the characters as UTF-16 surrogate pairs, but the decoder on the decoder on the other end can't handle that.

Matthew was able to work around the problem by making the encoder transmit raw unicode instead of escaped.  I thought we were discussing switching to that mode of operation in another ticket, but I can't find it at the moment.

I submitted a patch for this to mochiweb (in addition to one that was already available), but we can also apply it in our repo now if so desired.


> Replication of document results in invalid_json error
> -----------------------------------------------------
>
>                 Key: COUCHDB-327
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-327
>             Project: CouchDB
>          Issue Type: Bug
>    Affects Versions: 0.9
>         Environment: Ubuntu 8.10
>            Reporter: Matthew Gregg
>
> While replicating a document containing UTf8 characters replication fails with invalid_json errors in the receiving couch instance.
> The test below will cause the replication error.
> couchTests.utf8_replication = function(debug) {
>   if (debug) debugger;
>   var host = CouchDB.host;
>   var texts = [];
>   var dbPairs = [
>     {source:"http://"+host+"/test_suite_db_a",
>       target:"http://"+host+"/test_suite_db_b"}
>   ]
>   var dbA = new CouchDB("test_suite_db_a");
>   var dbB = new CouchDB("test_suite_db_b");
>   var numDocs = 10;
>   var xhr;
>   for (var testPair = 0; testPair < dbPairs.length; testPair++) {
>     var A = dbPairs[testPair].source
>     var B = dbPairs[testPair].target
>     dbA.deleteDb();
>     dbA.createDb();
>     dbB.deleteDb();
>     dbB.createDb();
>     text = "􀂀 Fakain: 􀂀 Faktor produksi yang digunakan hanya tenaga kerja saja. 􀂀 Kualitas ba";
>     dbA.save({_id:"1", text:text})
>     var result = CouchDB.replicate(A, B);
>   }
> };

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Closed: (COUCHDB-327) Replication of document results in invalid_json error

Posted by "Jan Lehnardt (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/COUCHDB-327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jan Lehnardt closed COUCHDB-327.
--------------------------------

    Resolution: Cannot Reproduce

Can't reproduce. Can you add more information about the dependencie's versions and your operating system?

> Replication of document results in invalid_json error
> -----------------------------------------------------
>
>                 Key: COUCHDB-327
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-327
>             Project: CouchDB
>          Issue Type: Bug
>    Affects Versions: 0.9
>         Environment: Ubuntu 8.10
>            Reporter: Matthew Gregg
>
> While replicating a document containing UTf8 characters replication fails with invalid_json errors in the receiving couch instance.
> The test below will cause the replication error.
> couchTests.utf8_replication = function(debug) {
>   if (debug) debugger;
>   var host = CouchDB.host;
>   var texts = [];
>   var dbPairs = [
>     {source:"http://"+host+"/test_suite_db_a",
>       target:"http://"+host+"/test_suite_db_b"}
>   ]
>   var dbA = new CouchDB("test_suite_db_a");
>   var dbB = new CouchDB("test_suite_db_b");
>   var numDocs = 10;
>   var xhr;
>   for (var testPair = 0; testPair < dbPairs.length; testPair++) {
>     var A = dbPairs[testPair].source
>     var B = dbPairs[testPair].target
>     dbA.deleteDb();
>     dbA.createDb();
>     dbB.deleteDb();
>     dbB.createDb();
>     text = "􀂀 Fakain: 􀂀 Faktor produksi yang digunakan hanya tenaga kerja saja. 􀂀 Kualitas ba";
>     dbA.save({_id:"1", text:text})
>     var result = CouchDB.replicate(A, B);
>   }
> };

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (COUCHDB-327) Replication of document results in invalid_json error

Posted by "Matthew Gregg (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/COUCHDB-327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701011#action_12701011 ] 

Matthew Gregg commented on COUCHDB-327:
---------------------------------------

The following dependencies are installed, all via Ubuntu 8.10 packages.
Erlang:  12.b.5
ICU:  3.8.1
openssl: 0.9.8g
spidermonkey(libmozjs0d):  1.8.1
libcurl: 7.18.2


> Replication of document results in invalid_json error
> -----------------------------------------------------
>
>                 Key: COUCHDB-327
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-327
>             Project: CouchDB
>          Issue Type: Bug
>    Affects Versions: 0.9
>         Environment: Ubuntu 8.10
>            Reporter: Matthew Gregg
>
> While replicating a document containing UTf8 characters replication fails with invalid_json errors in the receiving couch instance.
> The test below will cause the replication error.
> couchTests.utf8_replication = function(debug) {
>   if (debug) debugger;
>   var host = CouchDB.host;
>   var texts = [];
>   var dbPairs = [
>     {source:"http://"+host+"/test_suite_db_a",
>       target:"http://"+host+"/test_suite_db_b"}
>   ]
>   var dbA = new CouchDB("test_suite_db_a");
>   var dbB = new CouchDB("test_suite_db_b");
>   var numDocs = 10;
>   var xhr;
>   for (var testPair = 0; testPair < dbPairs.length; testPair++) {
>     var A = dbPairs[testPair].source
>     var B = dbPairs[testPair].target
>     dbA.deleteDb();
>     dbA.createDb();
>     dbB.deleteDb();
>     dbB.createDb();
>     text = "􀂀 Fakain: 􀂀 Faktor produksi yang digunakan hanya tenaga kerja saja. 􀂀 Kualitas ba";
>     dbA.save({_id:"1", text:text})
>     var result = CouchDB.replicate(A, B);
>   }
> };

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Reopened: (COUCHDB-327) Replication of document results in invalid_json error

Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/COUCHDB-327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adam Kocoloski reopened COUCHDB-327:
------------------------------------

      Assignee: Adam Kocoloski

> Replication of document results in invalid_json error
> -----------------------------------------------------
>
>                 Key: COUCHDB-327
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-327
>             Project: CouchDB
>          Issue Type: Bug
>    Affects Versions: 0.9
>         Environment: Ubuntu 8.10
>            Reporter: Matthew Gregg
>            Assignee: Adam Kocoloski
>
> While replicating a document containing UTf8 characters replication fails with invalid_json errors in the receiving couch instance.
> The test below will cause the replication error.
> couchTests.utf8_replication = function(debug) {
>   if (debug) debugger;
>   var host = CouchDB.host;
>   var texts = [];
>   var dbPairs = [
>     {source:"http://"+host+"/test_suite_db_a",
>       target:"http://"+host+"/test_suite_db_b"}
>   ]
>   var dbA = new CouchDB("test_suite_db_a");
>   var dbB = new CouchDB("test_suite_db_b");
>   var numDocs = 10;
>   var xhr;
>   for (var testPair = 0; testPair < dbPairs.length; testPair++) {
>     var A = dbPairs[testPair].source
>     var B = dbPairs[testPair].target
>     dbA.deleteDb();
>     dbA.createDb();
>     dbB.deleteDb();
>     dbB.createDb();
>     text = "􀂀 Fakain: 􀂀 Faktor produksi yang digunakan hanya tenaga kerja saja. 􀂀 Kualitas ba";
>     dbA.save({_id:"1", text:text})
>     var result = CouchDB.replicate(A, B);
>   }
> };

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.