Import File Filter on AvroStorage

["Kettenbach, Thomas" <ke...@adtech.com>]

Hi list,

pig uses the AvroStorage to import Avro binary files. This provides the 
functionality of
importing all Avro data files recursively in one directory. All Avro 
data files need to
be of the same schema.

I was wondering if there can be options given to the AvroStorage class 
from within the
pig script in order to filter the list of files beeing imported.

Is there some feature like this already beeing asked for? Can you point 
me in the right direction?

Thanks,
-- 
Thomas Kettenbach | Backend Development
ADTECH | AOL Advertising |
thomas.kettenbach@adtech.com | www.adtech.com <http://www.adtech.com>

Re: Import File Filter on AvroStorage

[Bill Graham <bi...@gmail.com>]

There's a partial patch here to support path globing:
https://issues.apache.org/jira/browse/PIG-2492


On Tue, Feb 7, 2012 at 7:39 AM, Kettenbach, Thomas <ke...@adtech.com>wrote:

> Hi list,
>
> pig uses the AvroStorage to import Avro binary files. This provides the
> functionality of
> importing all Avro data files recursively in one directory. All Avro data
> files need to
> be of the same schema.
>
> I was wondering if there can be options given to the AvroStorage class
> from within the
> pig script in order to filter the list of files beeing imported.
>
> Is there some feature like this already beeing asked for? Can you point me
> in the right direction?
>
> Thanks,
> --
> Thomas Kettenbach | Backend Development
> ADTECH | AOL Advertising |
> thomas.kettenbach@adtech.com | www.adtech.com <http://www.adtech.com>
>
>
>


-- 
*Note that I'm no longer using my Yahoo! email address. Please email me at
billgraham@gmail.com going forward.*