c++ DataFileWriter not doing validation

["SCHENK, Jarrad" <ja...@baesystems.com>]

Hi List,

I'm working with the c++ bindings to try to write data to avro files.

Much of the documentation assumes that the types to be written (and the code to write the data) are generated using avrogencpp.

In my case I have an existing set of type/struct hierarchies that I'm trying to write so I don't want to use the output of avrogencpp directly. Instead I am producing code that is very similar to what avrogencpp outputs but is adapted to suit my types.

What I'm finding is that the c++ DataFileWriter does no validation between the schema that I provide and the datums that get written. As such any discrepancy between the schema and the datums that are written causes the file to be corrupted and essentially unreadable.

I see that there is a ValidatingEncoder class that can be used when serialising to a memorystream (as per the Getting Started docs) but there doesn't appear to be any method for using this encoder with the DataFileWriter.

Am I missing something? Is there a preferred way to make the writer do validation?

Thanks

Jarrad
"Warning:
The information contained in this email and any attached files is
confidential to BAE Systems Australia. If you are not the intended
recipient, any use, disclosure or copying of this email or any
attachments is expressly prohibited.  If you have received this email
in error, please notify us immediately. VIRUS: Every care has been
taken to ensure this email and its attachments are virus free,
however, any loss or damage incurred in using this email is not the
sender's responsibility.  It is your responsibility to ensure virus
checks are completed before installing any data sent in this email to
your computer."

RE: c++ DataFileWriter not doing validation

["SCHENK, Jarrad" <ja...@baesystems.com>]

> From: SCHENK, Jarrad 
> Sent: Tuesday, 21 May 2013 12:52 PM
> To: user@avro.apache.org
> Subject: c++ DataFileWriter not doing validation 
>
> Hi List, 
>
> I'm working with the c++ bindings to try to write data to avro files. 
>
> Much of the documentation assumes that the types to be written (and the code to write the data) are generated using avrogencpp.
>
> In my case I have an existing set of type/struct hierarchies that I'm trying to write so I don't want to use the output of avrogencpp  >directly. Instead I am producing code that is very similar to what avrogencpp outputs but is adapted to suit my types.
>
> What I'm finding is that the c++ DataFileWriter does no validation between the schema that I provide and the datums that get written. As such any discrepancy between the schema and the datums that are written causes the file to be corrupted and essentially unreadable.
>
> I see that there is a ValidatingEncoder class that can be used when serialising to a memorystream (as per the Getting Started docs) but there doesn't appear to be any method for using this encoder with the DataFileWriter.
>
> Am I missing something? Is there a preferred way to make the writer do validation?
>
> Thanks
>
> Jarrad

I never got any responses to this so I assume either no-one knows, or no-one actively uses the C++ bindings anymore?

My basic question boils down to can the DataFileWriter be made to do validation against the schema?

Any help much appreciated.

Jarrad
"Warning:
The information contained in this email and any attached files is
confidential to BAE Systems Australia. If you are not the intended
recipient, any use, disclosure or copying of this email or any
attachments is expressly prohibited.  If you have received this email
in error, please notify us immediately. VIRUS: Every care has been
taken to ensure this email and its attachments are virus free,
however, any loss or damage incurred in using this email is not the
sender's responsibility.  It is your responsibility to ensure virus
checks are completed before installing any data sent in this email to
your computer."