You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Rob Verkuylen <ro...@verkuylen.net> on 2012/02/09 15:39:27 UTC
Flume avro file extension
I ran into problems with post processing Flume avrodata files with Pig's
AvroStorage loader. The loader will only read files with a .avro extension.
There's no section in the official AVRO spec, but the general consensus
seems to be to use the ".avro" extension for writing avro data files. See
AVRO-153 <https://issues.apache.org/jira/browse/AVRO-153> and Cloudera on
AVRO Usage <https://ccp.cloudera.com/display/CDHDOC/Avro+Usage>
Don't see it in Flume's Jira, but I can't be the first to raise this issue
either. Any thoughts?
Re: Flume avro file extension
Posted by Brock Noland <br...@cloudera.com>.
Missed that during my search, 980 is marked as duplicate/resolved.
On Mon, Feb 20, 2012 at 12:41 PM, Robby <ro...@gmail.com> wrote:
> That would become the dupe of 965: https://issues.apache.org/jira/browse/FLUME-965
>
> On 20 feb. 2012, at 15:18, Brock Noland <br...@cloudera.com> wrote:
>
>> https://issues.apache.org/jira/browse/FLUME-980
>>
>> On Mon, Feb 20, 2012 at 12:49 AM, Eric Sammer <es...@cloudera.com> wrote:
>>> Makes sense to me. Can I convince one of you guys to file a JIRA so we don't
>>> lose this one?
>>>
>>> Thanks.
>>>
>>>
>>> On Thu, Feb 9, 2012 at 2:10 PM, Tom White <to...@cloudera.com> wrote:
>>>>
>>>> +1 to making Flume Avro data files have the .avro extension.
>>>>
>>>> The Avro MapReduce classes expect the .avro extension too.
>>>>
>>>> Cheers,
>>>> Tom
>>>>
>>>> On Thu, Feb 9, 2012 at 6:39 AM, Rob Verkuylen <ro...@verkuylen.net> wrote:
>>>>> I ran into problems with post processing Flume avrodata files with Pig's
>>>>> AvroStorage loader. The loader will only read files with a .avro
>>>>> extension.
>>>>>
>>>>> There's no section in the official AVRO spec, but the general consensus
>>>>> seems to be to use the ".avro" extension for writing avro data files.
>>>>> See
>>>>> AVRO-153 and Cloudera on AVRO Usage
>>>>>
>>>>> Don't see it in Flume's Jira, but I can't be the first to raise this
>>>>> issue
>>>>> either. Any thoughts?
>>>>>
>>>
>>>
>>>
>>>
>>> --
>>> Eric Sammer
>>> twitter: esammer
>>> data: www.cloudera.com
>>
>>
>>
>> --
>> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Re: Flume avro file extension
Posted by Robby <ro...@gmail.com>.
That would become the dupe of 965: https://issues.apache.org/jira/browse/FLUME-965
On 20 feb. 2012, at 15:18, Brock Noland <br...@cloudera.com> wrote:
> https://issues.apache.org/jira/browse/FLUME-980
>
> On Mon, Feb 20, 2012 at 12:49 AM, Eric Sammer <es...@cloudera.com> wrote:
>> Makes sense to me. Can I convince one of you guys to file a JIRA so we don't
>> lose this one?
>>
>> Thanks.
>>
>>
>> On Thu, Feb 9, 2012 at 2:10 PM, Tom White <to...@cloudera.com> wrote:
>>>
>>> +1 to making Flume Avro data files have the .avro extension.
>>>
>>> The Avro MapReduce classes expect the .avro extension too.
>>>
>>> Cheers,
>>> Tom
>>>
>>> On Thu, Feb 9, 2012 at 6:39 AM, Rob Verkuylen <ro...@verkuylen.net> wrote:
>>>> I ran into problems with post processing Flume avrodata files with Pig's
>>>> AvroStorage loader. The loader will only read files with a .avro
>>>> extension.
>>>>
>>>> There's no section in the official AVRO spec, but the general consensus
>>>> seems to be to use the ".avro" extension for writing avro data files.
>>>> See
>>>> AVRO-153 and Cloudera on AVRO Usage
>>>>
>>>> Don't see it in Flume's Jira, but I can't be the first to raise this
>>>> issue
>>>> either. Any thoughts?
>>>>
>>
>>
>>
>>
>> --
>> Eric Sammer
>> twitter: esammer
>> data: www.cloudera.com
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Re: Flume avro file extension
Posted by Brock Noland <br...@cloudera.com>.
https://issues.apache.org/jira/browse/FLUME-980
On Mon, Feb 20, 2012 at 12:49 AM, Eric Sammer <es...@cloudera.com> wrote:
> Makes sense to me. Can I convince one of you guys to file a JIRA so we don't
> lose this one?
>
> Thanks.
>
>
> On Thu, Feb 9, 2012 at 2:10 PM, Tom White <to...@cloudera.com> wrote:
>>
>> +1 to making Flume Avro data files have the .avro extension.
>>
>> The Avro MapReduce classes expect the .avro extension too.
>>
>> Cheers,
>> Tom
>>
>> On Thu, Feb 9, 2012 at 6:39 AM, Rob Verkuylen <ro...@verkuylen.net> wrote:
>> > I ran into problems with post processing Flume avrodata files with Pig's
>> > AvroStorage loader. The loader will only read files with a .avro
>> > extension.
>> >
>> > There's no section in the official AVRO spec, but the general consensus
>> > seems to be to use the ".avro" extension for writing avro data files.
>> > See
>> > AVRO-153 and Cloudera on AVRO Usage
>> >
>> > Don't see it in Flume's Jira, but I can't be the first to raise this
>> > issue
>> > either. Any thoughts?
>> >
>
>
>
>
> --
> Eric Sammer
> twitter: esammer
> data: www.cloudera.com
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Re: Flume avro file extension
Posted by Eric Sammer <es...@cloudera.com>.
Makes sense to me. Can I convince one of you guys to file a JIRA so we
don't lose this one?
Thanks.
On Thu, Feb 9, 2012 at 2:10 PM, Tom White <to...@cloudera.com> wrote:
> +1 to making Flume Avro data files have the .avro extension.
>
> The Avro MapReduce classes expect the .avro extension too.
>
> Cheers,
> Tom
>
> On Thu, Feb 9, 2012 at 6:39 AM, Rob Verkuylen <ro...@verkuylen.net> wrote:
> > I ran into problems with post processing Flume avrodata files with Pig's
> > AvroStorage loader. The loader will only read files with a .avro
> extension.
> >
> > There's no section in the official AVRO spec, but the general consensus
> > seems to be to use the ".avro" extension for writing avro data files. See
> > AVRO-153 and Cloudera on AVRO Usage
> >
> > Don't see it in Flume's Jira, but I can't be the first to raise this
> issue
> > either. Any thoughts?
> >
>
--
Eric Sammer
twitter: esammer
data: www.cloudera.com
Re: Flume avro file extension
Posted by Tom White <to...@cloudera.com>.
+1 to making Flume Avro data files have the .avro extension.
The Avro MapReduce classes expect the .avro extension too.
Cheers,
Tom
On Thu, Feb 9, 2012 at 6:39 AM, Rob Verkuylen <ro...@verkuylen.net> wrote:
> I ran into problems with post processing Flume avrodata files with Pig's
> AvroStorage loader. The loader will only read files with a .avro extension.
>
> There's no section in the official AVRO spec, but the general consensus
> seems to be to use the ".avro" extension for writing avro data files. See
> AVRO-153 and Cloudera on AVRO Usage
>
> Don't see it in Flume's Jira, but I can't be the first to raise this issue
> either. Any thoughts?
>