You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Saptarshi Guha <sa...@gmail.com> on 2013/09/09 20:14:52 UTC
Question for ORCFileFormat
Hello,
Are there any examples of writing using ORC as aFileOutputFormat (and then
as a FileInputFormat) in MapReduce jobs? I was looking at the source of
ORCFIleInput/OutputFormat but couldn't quite grok how to compose the
ORCSerdes.
Cheers
Saptarshi
Re: Question for ORCFileFormat
Posted by Thejas Nair <th...@hortonworks.com>.
If you have hive metastore setup, using hcatalog is easy, you just
need the jars and hive-site.xml directory in the classpath. Then you
can use the hcat input/output formats in your map-reduce program -
http://hive.apache.org/docs/hcat_r0.5.0/inputoutput.html
On Wed, Sep 11, 2013 at 4:35 PM, Saptarshi Guha
<sa...@gmail.com> wrote:
> Hi,
> Thanks, but assuming i can't use HCatalog, or integrating it is difficult,
> is there an example of using
> ORC as an outputformat in a mapreduce job?
>
> Regards
> Saptarshi
>
>
>
> On Wed, Sep 11, 2013 at 1:36 PM, Owen O'Malley <om...@apache.org> wrote:
>>
>> The easiest way to use it is to use HCatalog, which enables you to read or
>> write ORC files from MapReduce or Pig.
>>
>> -- Owen
>>
>>
>> On Mon, Sep 9, 2013 at 11:14 AM, Saptarshi Guha <sa...@gmail.com>
>> wrote:
>>>
>>> Hello,
>>>
>>> Are there any examples of writing using ORC as aFileOutputFormat (and
>>> then as a FileInputFormat) in MapReduce jobs? I was looking at the source of
>>> ORCFIleInput/OutputFormat but couldn't quite grok how to compose the
>>> ORCSerdes.
>>>
>>> Cheers
>>> Saptarshi
>>>
>>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Question for ORCFileFormat
Posted by Saptarshi Guha <sa...@gmail.com>.
Hi,
Thanks, but assuming i can't use HCatalog, or integrating it is difficult,
is there an example of using
ORC as an outputformat in a mapreduce job?
Regards
Saptarshi
On Wed, Sep 11, 2013 at 1:36 PM, Owen O'Malley <om...@apache.org> wrote:
> The easiest way to use it is to use HCatalog, which enables you to read or
> write ORC files from MapReduce or Pig.
>
> -- Owen
>
>
> On Mon, Sep 9, 2013 at 11:14 AM, Saptarshi Guha <sa...@gmail.com>wrote:
>
>> Hello,
>>
>> Are there any examples of writing using ORC as aFileOutputFormat (and
>> then as a FileInputFormat) in MapReduce jobs? I was looking at the source
>> of ORCFIleInput/OutputFormat but couldn't quite grok how to compose the
>> ORCSerdes.
>>
>> Cheers
>> Saptarshi
>>
>>
>
Re: Question for ORCFileFormat
Posted by Owen O'Malley <om...@apache.org>.
The easiest way to use it is to use HCatalog, which enables you to read or
write ORC files from MapReduce or Pig.
-- Owen
On Mon, Sep 9, 2013 at 11:14 AM, Saptarshi Guha <sa...@gmail.com>wrote:
> Hello,
>
> Are there any examples of writing using ORC as aFileOutputFormat (and then
> as a FileInputFormat) in MapReduce jobs? I was looking at the source of
> ORCFIleInput/OutputFormat but couldn't quite grok how to compose the
> ORCSerdes.
>
> Cheers
> Saptarshi
>
>