You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Sam Overend (JIRA)" <ji...@apache.org> on 2012/09/25 13:29:07 UTC
[jira] [Created] (AVRO-1172) Avro C++ Json Decoder: Double cannot
be decoded
Sam Overend created AVRO-1172:
---------------------------------
Summary: Avro C++ Json Decoder: Double cannot be decoded
Key: AVRO-1172
URL: https://issues.apache.org/jira/browse/AVRO-1172
Project: Avro
Issue Type: Bug
Components: c++
Affects Versions: 1.7.1
Environment: Built under msys and gcc-4.6.1 on a Windows7/64 bit machine.
Reporter: Sam Overend
Short version: Looks like the C++ version of AVRO-1099.
Long version: When a non-decimal double is read from a json file, the parser treats it as an long, not a double, and therefore throws an exception.
Two possible solutions: (1) The decoder should be able to convert longs to doubles or acknowledge that a long is a type of double.
(2) The encoder should always output a double with a decimal point.
Example code is included below. Output is:
(1.01, 2.13)
terminate called after throwing an instance of 'avro::Exception'
what(): Incorrect token in the stream. Expected: Double, found Integer
After running complex.json is: {"re":1,"im":2.13
#include <iostream>
#include <fstream>
using namespace std;
#include "cpx.hh"
#include "avro/Compiler.hh"
#include "avro/Encoder.hh"
#include "avro/Decoder.hh"
avro::ValidSchema load(const char* filename)
{
std::ifstream ifs(filename);
avro::ValidSchema result;
avro::compileJsonSchema(ifs, result);
return result;
}
void OutTest()
{
avro::ValidSchema cpxSchema = load("cpx_schema.json");
std::auto_ptr<avro::OutputStream> out = avro::fileOutputStream("complex.json",1);
avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
e->init(*out);
c::cpx c1;
c1.re = 1.01;
c1.im = 2.13;
avro::encode(*e, c1);
out->flush();
}
void OutTest2()
{
avro::ValidSchema cpxSchema = load("cpx_schema.json");
std::auto_ptr<avro::OutputStream> out = avro::fileOutputStream("complex.json",1);
avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
e->init(*out);
c::cpx c1;
c1.re = 1.0;
c1.im = 2.13;
avro::encode(*e, c1);
out->flush();
}
void InTest()
{
avro::ValidSchema cpxSchema = load("cpx_schema.json");
std::auto_ptr<avro::InputStream> in = avro::fileInputStream("complex.json",1);
avro::DecoderPtr d = avro::jsonDecoder(cpxSchema);
d->init(*in);
c::cpx c2;
avro::decode(*d, c2);
std::cout << '(' << c2.re << ", " << c2.im << ')' << std::endl;
}
int main()
{
OutTest();
InTest();
OutTest2();
InTest();
return 0;
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1172) Avro C++ Json Decoder: Double cannot
be decoded
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-1172:
--------------------------------------
Attachment: AVRO-1172.patch
Good catch.
This patch fixes the problem.
In the sample code, instead of flushing the stream, if the encoder is flushes, the terminating close-braces would get written.
> Avro C++ Json Decoder: Double cannot be decoded
> -----------------------------------------------
>
> Key: AVRO-1172
> URL: https://issues.apache.org/jira/browse/AVRO-1172
> Project: Avro
> Issue Type: Bug
> Components: c++
> Affects Versions: 1.7.1
> Environment: Built under msys and gcc-4.6.1 on a Windows7/64 bit machine.
> Reporter: Sam Overend
> Labels: patch
> Attachments: AVRO-1172.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Short version: Looks like the C++ version of AVRO-1099.
> Long version: When a non-decimal double is read from a json file, the parser treats it as an long, not a double, and therefore throws an exception.
> Two possible solutions: (1) The decoder should be able to convert longs to doubles or acknowledge that a long is a type of double.
> (2) The encoder should always output a double with a decimal point.
> Example code is included below. Output is:
> (1.01, 2.13)
> terminate called after throwing an instance of 'avro::Exception'
> what(): Incorrect token in the stream. Expected: Double, found Integer
> After running complex.json is: {"re":1,"im":2.13
> #include <iostream>
> #include <fstream>
> using namespace std;
> #include "cpx.hh"
> #include "avro/Compiler.hh"
> #include "avro/Encoder.hh"
> #include "avro/Decoder.hh"
> avro::ValidSchema load(const char* filename)
> {
> std::ifstream ifs(filename);
> avro::ValidSchema result;
> avro::compileJsonSchema(ifs, result);
> return result;
> }
> void OutTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr<avro::OutputStream> out = avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.01;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void OutTest2()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr<avro::OutputStream> out = avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.0;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void InTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr<avro::InputStream> in = avro::fileInputStream("complex.json",1);
> avro::DecoderPtr d = avro::jsonDecoder(cpxSchema);
> d->init(*in);
> c::cpx c2;
> avro::decode(*d, c2);
> std::cout << '(' << c2.re << ", " << c2.im << ')' << std::endl;
> }
> int main()
> {
> OutTest();
> InTest();
> OutTest2();
> InTest();
> return 0;
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1172) Avro C++ Json Decoder: Double cannot
be decoded
Posted by "Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thiruvalluvan M. G. updated AVRO-1172:
--------------------------------------
Status: Patch Available (was: Open)
> Avro C++ Json Decoder: Double cannot be decoded
> -----------------------------------------------
>
> Key: AVRO-1172
> URL: https://issues.apache.org/jira/browse/AVRO-1172
> Project: Avro
> Issue Type: Bug
> Components: c++
> Affects Versions: 1.7.1
> Environment: Built under msys and gcc-4.6.1 on a Windows7/64 bit machine.
> Reporter: Sam Overend
> Labels: patch
> Attachments: AVRO-1172.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Short version: Looks like the C++ version of AVRO-1099.
> Long version: When a non-decimal double is read from a json file, the parser treats it as an long, not a double, and therefore throws an exception.
> Two possible solutions: (1) The decoder should be able to convert longs to doubles or acknowledge that a long is a type of double.
> (2) The encoder should always output a double with a decimal point.
> Example code is included below. Output is:
> (1.01, 2.13)
> terminate called after throwing an instance of 'avro::Exception'
> what(): Incorrect token in the stream. Expected: Double, found Integer
> After running complex.json is: {"re":1,"im":2.13
> #include <iostream>
> #include <fstream>
> using namespace std;
> #include "cpx.hh"
> #include "avro/Compiler.hh"
> #include "avro/Encoder.hh"
> #include "avro/Decoder.hh"
> avro::ValidSchema load(const char* filename)
> {
> std::ifstream ifs(filename);
> avro::ValidSchema result;
> avro::compileJsonSchema(ifs, result);
> return result;
> }
> void OutTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr<avro::OutputStream> out = avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.01;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void OutTest2()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr<avro::OutputStream> out = avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.0;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void InTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr<avro::InputStream> in = avro::fileInputStream("complex.json",1);
> avro::DecoderPtr d = avro::jsonDecoder(cpxSchema);
> d->init(*in);
> c::cpx c2;
> avro::decode(*d, c2);
> std::cout << '(' << c2.re << ", " << c2.im << ')' << std::endl;
> }
> int main()
> {
> OutTest();
> InTest();
> OutTest2();
> InTest();
> return 0;
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira