You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by hc busy <hc...@gmail.com> on 2010/05/09 09:03:04 UTC
WARNING: Files starting with periods are skipped
This might be well known already, but I just got kicked in the behind by a
temporary file that I generate and load that starts it's file name with a
period. Apparently PigStorage will not load any files starting with a
period... Sigh, probly a feature, but took a while to figure out.
It's actually kinda funny that I initially thought cogroup was broken before
checking out the Storage as the culprit...
-- busy
Re: WARNING: Files starting with periods are skipped
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
FYI, also files and directories that start with an underscore.
On Sun, May 9, 2010 at 12:32 AM, Jeff Zhang <zj...@gmail.com> wrote:
> Pig will filter file start with ".", and this is also the default
> behavior of hadoop's InputFormat. If you use the next release Pig 0.7
> which integrate Hadoop's InputFormat, you can override the InputFormat
> to include files start with "."
>
>
>
> On Sun, May 9, 2010 at 12:03 AM, hc busy <hc...@gmail.com> wrote:
>> This might be well known already, but I just got kicked in the behind by a
>> temporary file that I generate and load that starts it's file name with a
>> period. Apparently PigStorage will not load any files starting with a
>> period... Sigh, probly a feature, but took a while to figure out.
>>
>>
>> It's actually kinda funny that I initially thought cogroup was broken before
>> checking out the Storage as the culprit...
>>
>>
>>
>> -- busy
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
Re: WARNING: Files starting with periods are skipped
Posted by Jeff Zhang <zj...@gmail.com>.
Pig will filter file start with ".", and this is also the default
behavior of hadoop's InputFormat. If you use the next release Pig 0.7
which integrate Hadoop's InputFormat, you can override the InputFormat
to include files start with "."
On Sun, May 9, 2010 at 12:03 AM, hc busy <hc...@gmail.com> wrote:
> This might be well known already, but I just got kicked in the behind by a
> temporary file that I generate and load that starts it's file name with a
> period. Apparently PigStorage will not load any files starting with a
> period... Sigh, probly a feature, but took a while to figure out.
>
>
> It's actually kinda funny that I initially thought cogroup was broken before
> checking out the Storage as the culprit...
>
>
>
> -- busy
>
--
Best Regards
Jeff Zhang