You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Abhishek Agarwal <ab...@gmail.com> on 2014/07/25 13:46:59 UTC
Re: Using Parquet and Thrift in Hive
+ Re-sending as delivery of earlier mail failed.
On Fri, Jul 25, 2014 at 5:14 PM, Abhishek Agarwal <ab...@gmail.com>
wrote:
> Hi All,
> Is it possible to create a table with Parquet as storage mechanism, with
> schema being supplied from thrift IDL rather than metastore?
>
> Something like below,
>
>
> hive> CREATE EXTERNAL TABLE <Table Name> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer'
> WITH serdeproperties (
>
>
> "serialization.class"="<IDL Class>",
> "serialization.format"="org.apache.thrift.protocol.TBinaryProtocol"
> ) STORED AS
>
>
> INPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' OUTPUTFORMAT
>
>
> 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat';
>
>
>
> I want the schema to be generated through the ThriftSerDe and also use
> Parquet as storage. Would I need to write custom wrappers around the serde
> and input/output format?
>
>
> --
> Regards,
> Abhishek Agarwal
>
>
--
Regards,
Abhishek Agarwal
Re: [parquet-dev] Re: Using Parquet and Thrift in Hive
Posted by Brock Noland <br...@cloudera.com>.
This is not possible today. Can you file an enhancement and describe
the motivation?
Also Parquet has moved to apache: http://parquet.incubator.apache.org/
All questions and discussions should now be sent to
dev@parquet.incubator.apache.org please subscribe by emailing
dev-subscribe@parquet.incubator.apache.org
On Fri, Jul 25, 2014 at 4:46 AM, Abhishek Agarwal <ab...@gmail.com> wrote:
> + Re-sending as delivery of earlier mail failed.
>
>
> On Fri, Jul 25, 2014 at 5:14 PM, Abhishek Agarwal <ab...@gmail.com>
> wrote:
>>
>> Hi All,
>> Is it possible to create a table with Parquet as storage mechanism, with
>> schema being supplied from thrift IDL rather than metastore?
>>
>> Something like below,
>>
>>
>> hive> CREATE EXTERNAL TABLE <Table Name> ROW FORMAT SERDE
>> 'org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer'
>> WITH serdeproperties (
>>
>> "serialization.class"="<IDL Class>",
>> "serialization.format"="org.apache.thrift.protocol.TBinaryProtocol"
>> ) STORED AS
>>
>> INPUTFORMAT
>> 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' OUTPUTFORMAT
>>
>> 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat';
>>
>>
>> I want the schema to be generated through the ThriftSerDe and also use
>> Parquet as storage. Would I need to write custom wrappers around the serde
>> and input/output format?
>>
>>
>>
>> --
>> Regards,
>> Abhishek Agarwal
>>
>
>
>
> --
> Regards,
> Abhishek Agarwal
>
> --
> All questions and discussions should now be sent to
> dev@parquet.incubator.apache.org
> please subscribe by emailing dev-subscribe@parquet.incubator.apache.org
> ---
> You received this message because you are subscribed to the Google Groups
> "Parquet" group.
> To post to this group, send email to parquet-dev@googlegroups.com.