Structs in Hive

["kulkarni.swarnim@gmail.com" <ku...@gmail.com>]

Hello,

I kind of have a pretty basic question here. I am trying to read structs
stored in HBase to be read by Hive. In what format should these structs be
written so that they can be read?

For instance, if my query has the following struct:

s struct<a: STRING, b: STRING>

How should I be writing my data in HBase so that when read, it fits into
this struct? In other words, can I create my own class 'MyStruct' which is
something like:

class MyStruct{
   string a;
   string b;
}

to create the struct bytes and read them using hive with the struct defined
above? I hope I made my question clear. I will be glad to provide any
clarifications.

Thanks,

-- 
Swarnim

Re: Structs in Hive

[Edward Capriolo <ed...@gmail.com>]

If you are writing a GenericUDF or serde and want to return struct
types there are object inspectors to build structs. The Java type
return is an Object[]. Hive expects that if the struct has 5 fields
the object array will have a length of 5.

On Mon, Jul 23, 2012 at 1:34 PM,  <ku...@gmail.com> wrote:
> Cool. Thanks :)
>
> Also was just curious what do people generally use to write struct data in hive tables? I see that there is a STRUCT function defined that takes parameters and creates structs off them. Can we use a custom class as well?
>
> Thanks again.
>
> Sent from my iPhone
>
> On Jul 23, 2012, at 12:05 PM, Edward Capriolo <ed...@gmail.com> wrote:
>
>> in your case hbase has a custom serde, the Deserializer interface is
>> what turns the value from the input format into something that hive
>> can understand. HBase support uses the user specified table property
>> columns.mapping as information for what it should parse out of the
>> hbase result.
>>
>> On Mon, Jul 23, 2012 at 12:27 PM, kulkarni.swarnim@gmail.com
>> <ku...@gmail.com> wrote:
>>> Hello,
>>>
>>> I kind of have a pretty basic question here. I am trying to read structs
>>> stored in HBase to be read by Hive. In what format should these structs be
>>> written so that they can be read?
>>>
>>> For instance, if my query has the following struct:
>>>
>>> s struct<a: STRING, b: STRING>
>>>
>>> How should I be writing my data in HBase so that when read, it fits into
>>> this struct? In other words, can I create my own class 'MyStruct' which is
>>> something like:
>>>
>>> class MyStruct{
>>>   string a;
>>>   string b;
>>> }
>>>
>>> to create the struct bytes and read them using hive with the struct defined
>>> above? I hope I made my question clear. I will be glad to provide any
>>> clarifications.
>>>
>>> Thanks,
>>>
>>> --
>>> Swarnim

Re: Structs in Hive

[ku...@gmail.com]

Cool. Thanks :)

Also was just curious what do people generally use to write struct data in hive tables? I see that there is a STRUCT function defined that takes parameters and creates structs off them. Can we use a custom class as well?

Thanks again.

Sent from my iPhone

On Jul 23, 2012, at 12:05 PM, Edward Capriolo <ed...@gmail.com> wrote:

> in your case hbase has a custom serde, the Deserializer interface is
> what turns the value from the input format into something that hive
> can understand. HBase support uses the user specified table property
> columns.mapping as information for what it should parse out of the
> hbase result.
> 
> On Mon, Jul 23, 2012 at 12:27 PM, kulkarni.swarnim@gmail.com
> <ku...@gmail.com> wrote:
>> Hello,
>> 
>> I kind of have a pretty basic question here. I am trying to read structs
>> stored in HBase to be read by Hive. In what format should these structs be
>> written so that they can be read?
>> 
>> For instance, if my query has the following struct:
>> 
>> s struct<a: STRING, b: STRING>
>> 
>> How should I be writing my data in HBase so that when read, it fits into
>> this struct? In other words, can I create my own class 'MyStruct' which is
>> something like:
>> 
>> class MyStruct{
>>   string a;
>>   string b;
>> }
>> 
>> to create the struct bytes and read them using hive with the struct defined
>> above? I hope I made my question clear. I will be glad to provide any
>> clarifications.
>> 
>> Thanks,
>> 
>> --
>> Swarnim

Re: Structs in Hive

[Edward Capriolo <ed...@gmail.com>]

in your case hbase has a custom serde, the Deserializer interface is
what turns the value from the input format into something that hive
can understand. HBase support uses the user specified table property
columns.mapping as information for what it should parse out of the
hbase result.

On Mon, Jul 23, 2012 at 12:27 PM, kulkarni.swarnim@gmail.com
<ku...@gmail.com> wrote:
> Hello,
>
> I kind of have a pretty basic question here. I am trying to read structs
> stored in HBase to be read by Hive. In what format should these structs be
> written so that they can be read?
>
> For instance, if my query has the following struct:
>
> s struct<a: STRING, b: STRING>
>
> How should I be writing my data in HBase so that when read, it fits into
> this struct? In other words, can I create my own class 'MyStruct' which is
> something like:
>
> class MyStruct{
>    string a;
>    string b;
> }
>
> to create the struct bytes and read them using hive with the struct defined
> above? I hope I made my question clear. I will be glad to provide any
> clarifications.
>
> Thanks,
>
> --
> Swarnim