correct way of saving data to file in C++, 1.8.2 api

[scarlehoff <ju...@gmail.com>]

Hi,
I'm trying to modify the first example in the c++ tutorial:
http://avro.apache.org/docs/1.8.1/api/cpp/html/index.html (note: docs for
c++,1.8.2 are not working for me)
I'm trying to write to a file. For this I modify the example from 

    std::auto_ptr<avro::OutputStream> *out = avro::memoryOutputStream();*
    avro::EncoderPtr e = avro::binaryEncoder();
    e->init(*out);
    c::cpx c1;
    c1.re = 1.0;
    c1.im = 2.13;
    avro::encode(*e, c1);

to
     std::auto_ptr<avro::OutputStream> *outFile =
avro::fileOutputStream("filetest.db");*
     avro::EncoderPtr e = avro::binaryEncoder();
     e->init(*outFile);
     c::cpx c1;
     c1.re = 1.0;
     c1.im = 2.13;
     avro::encode(*e, c1);
*     outFile->flush();*

I get a file "filetest.db" which I can then read using C++.
However, I cannot read the resulting file using python because the following
code fails:
    from avro.datafile import DataFileReader
    from avro.io import DatumReader
    reader = DataFileReader(open("filetest.db", "rb"), DatumReader())
 UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8f in position 2:
invalid start byte

My guess is my C++ code is wrong, but I'm can't find any (recent)
documentation on what is the correct way of saving things to a file when
using C++.
Thanks,
s 



--
View this message in context: http://apache-avro.679487.n3.nabble.com/correct-way-of-saving-data-to-file-in-C-1-8-2-api-tp4037666.html
Sent from the Avro - Users mailing list archive at Nabble.com.

Re: correct way of saving data to file in C++, 1.8.2 api

[Tim Upthegrove <ti...@gmail.com>]

I have not tested this, but it is possible that you all want the
avro::DataFileWriter<T> templated class.  I believe that class will ensure
the appropriate header information is in the Avro data file, and it has a
write method that takes in data of type T and serializes it, appending that
to the end of the file.  I am not sure what the avro::fileOutputStream
does, but perhaps it does not include the header information that is
expected in the Avro data file format.  The python class referenced in the
original email definitely does expect the header to be in place.  As I
mentioned before, I have not tested this, but I figured I'd throw this out
as a thread to pull on in case anyone was still stick.

Good luck,
Tim Upthegrove

On Tue, Oct 31, 2017 at 5:33 PM, iampete <pe...@excite.com> wrote:

> Did you ever work this out?  I am having the same problem.
> Thanks
>
>
>
>
> --
> Sent from: http://apache-avro.679487.n3.nabble.com/Avro-Users-f679479.html
>

Re: correct way of saving data to file in C++, 1.8.2 api

[iampete <pe...@excite.com>]

Did you ever work this out?  I am having the same problem.
Thanks




--
Sent from: http://apache-avro.679487.n3.nabble.com/Avro-Users-f679479.html