You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by Doug Daniels <dd...@mortardata.com> on 2011/09/23 06:27:22 UTC

protobuf without LZO in elephant-bird

Is it possible to use elephant-bird to load protobuf-encoded data that isn't compressed with LZO?

Thanks,
Doug

Re: protobuf without LZO in elephant-bird

Posted by Raghu Angadi <an...@gmail.com>.

What format is the file in?

You can use ProtobufBytesToTuple() udf to convert protobuf bytes to a PIG
tuple.

e.g.:

define
ToTuple com.twitter.elephantbird.pig.piggybank.ProtobufBytesToTuple('com.example.protobufs.Table.User');

users = foreach binary generate ToTuple(user_bytes);
describe users;

Raghu.

On Thu, Sep 22, 2011 at 9:27 PM, Doug Daniels <dd...@mortardata.com>wrote:

> Is it possible to use elephant-bird to load protobuf-encoded data that
> isn't compressed with LZO?
>
> Thanks,
> Doug
>