You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Landon Clark <la...@lclark.net> on 2010/08/20 01:39:05 UTC
invalid UTF-8 JSON
I am using a tool called Chef to manage our servers. Chef uses
CouchDB as a backend storage engine. Chef captures about 200KB of
data about the configuration of each server. Occasionally, CouchDB
returns an error (using both 1.0.1 on OSX 10.6 and .10.0 on Ubuntu
10.4):
{"error":"bad_request","reason":"invalid UTF-8 JSON"}
As far as I can tell, the JSON is valid. I have loaded it using the
Ruby JSON parser and a JSONlint validator.
Can someone give me a hint about how I should go about debugging this?
I tried enabling debugging in CouchDB but that did not give me any
more information that I can decipher.
Attached in the JSON and I am trying to load it using this command:
curl -X PUT -H "content-type: application/json"
http://127.0.0.1:5984/test/doc -d @error.log
Any help would be much appreciated.
Thanks,
Landon
Re: invalid UTF-8 JSON
Posted by "Mark J. Reed" <ma...@gmail.com>.
On Thu, Aug 19, 2010 at 7:47 PM, Tyler Gillies <tj...@gmail.com> wrote:
> Have you ever used puppet? http://www.puppetlabs.com/puppet/introduction/
What does that have to do with anything? This is a CouchDB list; Chef
was brought up because it uses CouchDB as its data store. Puppet
doesn't, and so is off-topic. Please, let's not get into the whole
Chef vs. Puppet thing.
--
Mark J. Reed <ma...@gmail.com>
Re: invalid UTF-8 JSON
Posted by Tyler Gillies <tj...@gmail.com>.
Have you ever used puppet? http://www.puppetlabs.com/puppet/introduction/
<http://www.puppetlabs.com/puppet/introduction/>chef alternative
On Thu, Aug 19, 2010 at 4:45 PM, Randall Leeds <ra...@gmail.com>wrote:
> If you recently updated couch be sure you don't have an old version of
> the mochiweb libraries around anywhere.
> See for example:
> http://permalink.gmane.org/gmane.comp.db.couchdb.user/7780
>
> On Thu, Aug 19, 2010 at 16:39, Landon Clark <la...@lclark.net> wrote:
> > I am using a tool called Chef to manage our servers. Chef uses
> > CouchDB as a backend storage engine. Chef captures about 200KB of
> > data about the configuration of each server. Occasionally, CouchDB
> > returns an error (using both 1.0.1 on OSX 10.6 and .10.0 on Ubuntu
> > 10.4):
> > {"error":"bad_request","reason":"invalid UTF-8 JSON"}
> >
> > As far as I can tell, the JSON is valid. I have loaded it using the
> > Ruby JSON parser and a JSONlint validator.
> >
> > Can someone give me a hint about how I should go about debugging this?
> > I tried enabling debugging in CouchDB but that did not give me any
> > more information that I can decipher.
> >
> > Attached in the JSON and I am trying to load it using this command:
> > curl -X PUT -H "content-type: application/json"
> > http://127.0.0.1:5984/test/doc -d @error.log
> >
> > Any help would be much appreciated.
> >
> > Thanks,
> > Landon
> >
>
--
PdxBrain <http://www.pdxbrain.com> - My Home
Re: invalid UTF-8 JSON
Posted by Michael Genereux <mg...@gmail.com>.
Landon, I had a nightmare a while back and posted the scenario and
solution on here. I wasn't dealing with Chef but Rails was very happy
exporting invalid JSON from a character set perspective. Here it is
again and hope it helps!
--- Problem ---
I'm exporting a portion of a massive MySQL database that I'm thinking
is better suited to a CouchDB database. The MySQL database supports a
Rails application and the MySQL server is set to UTF8 for everything.
I'm using to_json in Rails that appears to convert the records just
fine to JSON. I get about 200 records converted and imported into
CouchDB and then the process dies with "Invalid UTF-8 JSON". One of
my fields in the offending record has the word "fête". The JSON
produced by Rails doesn't convert this UTF-8 character to the JSON \u0000
notation. I don't think it should have to but maybe I'm not
understanding the standard.
--- Solution ---
The original importer of the data took ISO-8599-1 data and jammed it
into a UTF-8 field in the database. The character that I was having
problems with was being auto translated when viewed in phpMyAdmin by
the web browser using a kind of ASCII/IOS-8859-1/Windows-1252 fallback
on non UTF-8 characters. So the character that looked fine was actual
not UTF-8. I could cut and paste the converted Unicode character from
phpMyAdmin right into Futon and the JSON was valid. The solution
within Rails was after I converted the Rails object to JSON, I ran
"json_data = Iconv.conv( 'utf-8', 'iso-8859-1', json_data )" to clean
out bad characters. Worked like a charm!
On Thu, Aug 19, 2010 at 8:31 PM, Landon Clark <la...@lclark.net> wrote:
> Response inline
>
> On Thu, Aug 19, 2010 at 4:45 PM, Randall Leeds <ra...@gmail.com> wrote:
>> If you recently updated couch be sure you don't have an old version of
>> the mochiweb libraries around anywhere.
>> See for example: http://permalink.gmane.org/gmane.comp.db.couchdb.user/7780
>>
>
> Good hint. I double checked this and the only version that I have is:
> ...../lib/couchdb/erlang/lib/mochiweb-r113
>
> It appears that's an ok version right?
>
> Thanks,
> Landon
>
Re: invalid UTF-8 JSON
Posted by Landon Clark <la...@lclark.net>.
Response inline
On Thu, Aug 19, 2010 at 4:45 PM, Randall Leeds <ra...@gmail.com> wrote:
> If you recently updated couch be sure you don't have an old version of
> the mochiweb libraries around anywhere.
> See for example: http://permalink.gmane.org/gmane.comp.db.couchdb.user/7780
>
Good hint. I double checked this and the only version that I have is:
...../lib/couchdb/erlang/lib/mochiweb-r113
It appears that's an ok version right?
Thanks,
Landon
Re: invalid UTF-8 JSON
Posted by Randall Leeds <ra...@gmail.com>.
If you recently updated couch be sure you don't have an old version of
the mochiweb libraries around anywhere.
See for example: http://permalink.gmane.org/gmane.comp.db.couchdb.user/7780
On Thu, Aug 19, 2010 at 16:39, Landon Clark <la...@lclark.net> wrote:
> I am using a tool called Chef to manage our servers. Chef uses
> CouchDB as a backend storage engine. Chef captures about 200KB of
> data about the configuration of each server. Occasionally, CouchDB
> returns an error (using both 1.0.1 on OSX 10.6 and .10.0 on Ubuntu
> 10.4):
> {"error":"bad_request","reason":"invalid UTF-8 JSON"}
>
> As far as I can tell, the JSON is valid. I have loaded it using the
> Ruby JSON parser and a JSONlint validator.
>
> Can someone give me a hint about how I should go about debugging this?
> I tried enabling debugging in CouchDB but that did not give me any
> more information that I can decipher.
>
> Attached in the JSON and I am trying to load it using this command:
> curl -X PUT -H "content-type: application/json"
> http://127.0.0.1:5984/test/doc -d @error.log
>
> Any help would be much appreciated.
>
> Thanks,
> Landon
>