Field mapping with the Avro SerDe

[Pedro Figueiredo <pf...@89clouds.com>]

Hi,

I'm working with some Avro data where a member of a struct has unfortunately been named 'data', which is a reserved keyword. It's therefore impossible, to my knowledge, to do a select on this.

Is there a way to remap fields, like the JSON SerDe[0] does? Or is there a workaround for this (other than changing the Avro schema upstream) ? I'm running Hive 0.8.1 in CDH3, with the haivvreo[1] SerDe..

[0]: https://github.com/rathboma/Hive-JSON-Serde-1
[1]: https://github.com/jghoman/haivvreo

Thank you in advance,

Pedro

Re: Field mapping with the Avro SerDe

[Pedro Figueiredo <pf...@89clouds.com>]

On 31 Oct 2012, at 15:19, Dean Wampler <de...@thinkbiganalytics.com> wrote:

> I had no idea if the following hack would work or not, but it's used in the Scala language for similar purposes:
> 
> Put the word data in left-hand ("back") quotes, e.g.,  `data`. For example
> 
> create table toss (`data` string);
> 

And indeed like in SQL. So SELECT mystruct.`data` works perfectly. And even mystruct.`data`["foo"].

Thank you so much!

Re: Field mapping with the Avro SerDe

[Dean Wampler <de...@thinkbiganalytics.com>]

I had no idea if the following hack would work or not, but it's used in the
Scala language for similar purposes:

Put the word data in left-hand ("back") quotes, e.g.,  `data`. For example

create table toss (`data` string);

It works!!

dean

On Wed, Oct 31, 2012 at 9:59 AM, Pedro Figueiredo <pf...@89clouds.com> wrote:

> Hi,
>
> I'm working with some Avro data where a member of a struct has
> unfortunately been named 'data', which is a reserved keyword. It's
> therefore impossible, to my knowledge, to do a select on this.
>
> Is there a way to remap fields, like the JSON SerDe[0] does? Or is there a
> workaround for this (other than changing the Avro schema upstream) ? I'm
> running Hive 0.8.1 in CDH3, with the haivvreo[1] SerDe..
>
> [0]: https://github.com/rathboma/Hive-JSON-Serde-1
> [1]: https://github.com/jghoman/haivvreo
>
> Thank you in advance,
>
> Pedro




-- 
*Dean Wampler, Ph.D.*
thinkbiganalytics.com
+1-312-339-1330