You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Manoj Murumkar <ma...@gmail.com> on 2017/03/24 06:14:48 UTC

ORC plugin for Kafka HDFS connector

Hi,

I am developing a connector to support ORC data type in HDFS connector.
Everything is in place except for hive integration. Specifically, in the
SchemaFileReader implementation. It wants to extract Avro schema from ORC
record. However, I am unable to get record name from ORC record in order to
build the Avro schema. Has anyone implemented this anywhere?

This is what I am using for testing (bold information is missing in ORC):

value.schema='{"type":"record",*"name":"orcrecord"*
,"fields":[{"name":"name","type":"string"},{"name":"age","type":"int"}]}'

Thanks,

Manoj

Re: ORC plugin for Kafka HDFS connector

Posted by Manoj Murumkar <ma...@gmail.com>.

>> It wants to extract Avro schema from ORC record.

Should say: It wants to extract connect schema from ORC record.

On Thu, Mar 23, 2017 at 11:14 PM, Manoj Murumkar <ma...@gmail.com>
wrote:

> Hi,
>
> I am developing a connector to support ORC data type in HDFS connector.
> Everything is in place except for hive integration. Specifically, in the
> SchemaFileReader implementation. It wants to extract Avro schema from ORC
> record. However, I am unable to get record name from ORC record in order to
> build the Avro schema. Has anyone implemented this anywhere?
>
> This is what I am using for testing (bold information is missing in ORC):
>
> value.schema='{"type":"record",*"name":"orcrecord"*,"fields":[
> {"name":"name","type":"string"},{"name":"age","type":"int"}]}'
>
> Thanks,
>
> Manoj
>