You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Paulo Carvalho <pj...@gmail.com> on 2012/04/26 16:37:41 UTC
Couchdb encoding configuration
Hello,
I have a json file, which I think is valid, with the following content:
{"docs":[{"comm":"Taille éoyenne pour les structures -- Échantillons :
116-121"}]}
To insert it on my database, I use Curl:
> curl -X PUT -d @xpto2.json http://127.0.0.1:5984/mybd/doc_id
I get the following error on the couchDB log:
"lexical error: invalid bytes in UTF8 string."
I think something is wrong with the encoding configuration of coucDB.
However, I cannot find how and where it can be solved. I am working with
the windows version of couchDB.
Could anyone help me with this issue?
Thank you
Best regards
Re: Couchdb encoding configuration
Posted by Paulo Carvalho <pj...@gmail.com>.
Yes, that was the problem.
I used a java class to create my file in UTF8 and it worked.
Thank you both.
On Fri, Apr 27, 2012 at 8:46 AM, Robert Newson <rn...@apache.org> wrote:
> Paulo,
>
> What encoding is your file stored in? My suggestion applies only if
> it's already in UTF-8 encoding (-d would mangle that but --data-binary
> or -T would not). I think it's clear that the file is not stored in
> UTF-8 encoding (I'm guessing iso-8859-1 btw) so you'll need to convert
> it as dch shows.
>
> B.
>
> On 27 April 2012 07:33, Paulo Carvalho <pj...@gmail.com> wrote:
> > Hello
> >
> > Thanks for the answers.
> >
> > As suggested, I executed the following command:
> > curl -X PUT -T xpto2.json http://127.0.0.1:5984/mybd/doc_id
> >
> > But the result was the same:
> > {"error":"bad_request","reason":"invalid_json"}
> >
> > And in couchDB log:
> > <<"{\"docs\":[{\"comm\":\"Taille éoyenne pour les structures --
> > Échantillons : 116-121\"}]}">>}
> >
> > If you have another suggestion, it is very welcome.
> >
> > Thank you
> >
> > Regards
> >
> >
> > On Thu, Apr 26, 2012 at 8:40 PM, Dave Cottlehuber <da...@muse.net.nz>
> wrote:
> >
> >> > On Thu, Apr 26, 2012 at 4:37 PM, Paulo Carvalho <pjcarvalho@gmail.com
> >> >wrote:
> >> >
> >> >> Hello,
> >> >>
> >> >> I have a json file, which I think is valid, with the following
> content:
> >> >>
> >> >> {"docs":[{"comm":"Taille éoyenne pour les structures -- Échantillons
> :
> >> >> 116-121"}]}
> >>
> >> Prior to transferring to couch, can one validate this?
> >>
> >> - as utf-8
> >> - as json
> >>
> >> Given a file utf8wtf.json, how about:
> >>
> >> iconv -f UTF-8 utf8wtf.json
> >>
> >> or with perl installed I can use this and check json as well:
> >>
> >> json_xs -t null -v < utf8wtf.json
> >>
> >> Any other ideas that might be more cross platform?
> >>
> >> A+
> >> Dave
> >>
> >
> >
> >
> > --
> > Paulo Carvalho
> > 1 rue du Chateau
> > 57710 Aumetz
> > France
> > http://forum-informatico.forumeiros.com/index.htm
> > http://ummundoecologico.blogspot.com
>
--
Paulo Carvalho
1 rue du Chateau
57710 Aumetz
France
http://forum-informatico.forumeiros.com/index.htm
http://ummundoecologico.blogspot.com
Re: Couchdb encoding configuration
Posted by Robert Newson <rn...@apache.org>.
Paulo,
What encoding is your file stored in? My suggestion applies only if
it's already in UTF-8 encoding (-d would mangle that but --data-binary
or -T would not). I think it's clear that the file is not stored in
UTF-8 encoding (I'm guessing iso-8859-1 btw) so you'll need to convert
it as dch shows.
B.
On 27 April 2012 07:33, Paulo Carvalho <pj...@gmail.com> wrote:
> Hello
>
> Thanks for the answers.
>
> As suggested, I executed the following command:
> curl -X PUT -T xpto2.json http://127.0.0.1:5984/mybd/doc_id
>
> But the result was the same:
> {"error":"bad_request","reason":"invalid_json"}
>
> And in couchDB log:
> <<"{\"docs\":[{\"comm\":\"Taille éoyenne pour les structures --
> Échantillons : 116-121\"}]}">>}
>
> If you have another suggestion, it is very welcome.
>
> Thank you
>
> Regards
>
>
> On Thu, Apr 26, 2012 at 8:40 PM, Dave Cottlehuber <da...@muse.net.nz> wrote:
>
>> > On Thu, Apr 26, 2012 at 4:37 PM, Paulo Carvalho <pjcarvalho@gmail.com
>> >wrote:
>> >
>> >> Hello,
>> >>
>> >> I have a json file, which I think is valid, with the following content:
>> >>
>> >> {"docs":[{"comm":"Taille éoyenne pour les structures -- Échantillons :
>> >> 116-121"}]}
>>
>> Prior to transferring to couch, can one validate this?
>>
>> - as utf-8
>> - as json
>>
>> Given a file utf8wtf.json, how about:
>>
>> iconv -f UTF-8 utf8wtf.json
>>
>> or with perl installed I can use this and check json as well:
>>
>> json_xs -t null -v < utf8wtf.json
>>
>> Any other ideas that might be more cross platform?
>>
>> A+
>> Dave
>>
>
>
>
> --
> Paulo Carvalho
> 1 rue du Chateau
> 57710 Aumetz
> France
> http://forum-informatico.forumeiros.com/index.htm
> http://ummundoecologico.blogspot.com
Re: Couchdb encoding configuration
Posted by Paulo Carvalho <pj...@gmail.com>.
Hello
Thanks for the answers.
As suggested, I executed the following command:
curl -X PUT -T xpto2.json http://127.0.0.1:5984/mybd/doc_id
But the result was the same:
{"error":"bad_request","reason":"invalid_json"}
And in couchDB log:
<<"{\"docs\":[{\"comm\":\"Taille éoyenne pour les structures --
Échantillons : 116-121\"}]}">>}
If you have another suggestion, it is very welcome.
Thank you
Regards
On Thu, Apr 26, 2012 at 8:40 PM, Dave Cottlehuber <da...@muse.net.nz> wrote:
> > On Thu, Apr 26, 2012 at 4:37 PM, Paulo Carvalho <pjcarvalho@gmail.com
> >wrote:
> >
> >> Hello,
> >>
> >> I have a json file, which I think is valid, with the following content:
> >>
> >> {"docs":[{"comm":"Taille éoyenne pour les structures -- Échantillons :
> >> 116-121"}]}
>
> Prior to transferring to couch, can one validate this?
>
> - as utf-8
> - as json
>
> Given a file utf8wtf.json, how about:
>
> iconv -f UTF-8 utf8wtf.json
>
> or with perl installed I can use this and check json as well:
>
> json_xs -t null -v < utf8wtf.json
>
> Any other ideas that might be more cross platform?
>
> A+
> Dave
>
--
Paulo Carvalho
1 rue du Chateau
57710 Aumetz
France
http://forum-informatico.forumeiros.com/index.htm
http://ummundoecologico.blogspot.com
Re: Couchdb encoding configuration
Posted by Dave Cottlehuber <da...@muse.net.nz>.
> On Thu, Apr 26, 2012 at 4:37 PM, Paulo Carvalho <pj...@gmail.com>wrote:
>
>> Hello,
>>
>> I have a json file, which I think is valid, with the following content:
>>
>> {"docs":[{"comm":"Taille éoyenne pour les structures -- Échantillons :
>> 116-121"}]}
Prior to transferring to couch, can one validate this?
- as utf-8
- as json
Given a file utf8wtf.json, how about:
iconv -f UTF-8 utf8wtf.json
or with perl installed I can use this and check json as well:
json_xs -t null -v < utf8wtf.json
Any other ideas that might be more cross platform?
A+
Dave
Re: Couchdb encoding configuration
Posted by CGS <cg...@gmail.com>.
Hi,
I think chars like "é" and "É" need to be encoded before being added in the
string.
CGS
On Thu, Apr 26, 2012 at 4:37 PM, Paulo Carvalho <pj...@gmail.com>wrote:
> Hello,
>
> I have a json file, which I think is valid, with the following content:
>
> {"docs":[{"comm":"Taille éoyenne pour les structures -- Échantillons :
> 116-121"}]}
>
> To insert it on my database, I use Curl:
>
> > curl -X PUT -d @xpto2.json http://127.0.0.1:5984/mybd/doc_id
>
>
> I get the following error on the couchDB log:
>
> "lexical error: invalid bytes in UTF8 string."
>
>
>
> I think something is wrong with the encoding configuration of coucDB.
> However, I cannot find how and where it can be solved. I am working with
> the windows version of couchDB.
>
> Could anyone help me with this issue?
>
> Thank you
>
> Best regards
>
Re: Couchdb encoding configuration
Posted by Robert Newson <rn...@apache.org>.
-d is the data mangling option. Use --data-binary or, better, -T <filename>
B.
On 26 April 2012 15:37, Paulo Carvalho <pj...@gmail.com> wrote:
> Hello,
>
> I have a json file, which I think is valid, with the following content:
>
> {"docs":[{"comm":"Taille éoyenne pour les structures -- Échantillons :
> 116-121"}]}
>
> To insert it on my database, I use Curl:
>
>> curl -X PUT -d @xpto2.json http://127.0.0.1:5984/mybd/doc_id
>
>
> I get the following error on the couchDB log:
>
> "lexical error: invalid bytes in UTF8 string."
>
>
>
> I think something is wrong with the encoding configuration of coucDB.
> However, I cannot find how and where it can be solved. I am working with
> the windows version of couchDB.
>
> Could anyone help me with this issue?
>
> Thank you
>
> Best regards