You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by meghana narasimhan <me...@gmail.com> on 2012/12/14 01:55:12 UTC

Loading data in different timezone

Hi All,

So we have data in S3 partitioned by hour in UTC :

2012/10/11/00
2012/10/11/01
....

2012/10/12/00
2012/10/12/01

We need to now load data in Pacific time so we need to load for 2012/10/12
data from

2012/10/11/08
2012/10/11/09
....

2012/10/12/00
.....
2012/10/12/07

We also need to load data for an entire month in Pacific.

So far what we have planned to do, is to extend TextLoader (we want to use
TextLoader). Send in the date or the month and then return a list of dates
to be loaded.

Are there any better ideas out there?

Thanks,
Meg