You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@chukwa.apache.org by Corbin Hoenes <co...@tynt.com> on 2010/04/08 00:49:13 UTC
Re: reprocess *InError
Thanks Eric.
I am going to have to redemux a bunch of data since our custom processor has changed a bit and we parse data and store into the fields map for pig to read easily.
I think I can do this simply by copying from /chukwa/finalArchives/20100309
then renaming to .done.
Once I do that does the data I reprocess overwrite the existing data in /chukwa/repos or do I need to remove that data as well?
Once it's completed does it put the data into the finalArchives/20100309 dir or put into finalArchives/[TODAY'S DATE]? So if I ever needed to reprocess again I'd loose the ability to reprocess by date?
On Mar 12, 2010, at 2:07 PM, Eric Yang wrote:
> It's for debug. You should be able to reprocess original data by copying
> archive for that time range back to sink directory and rename to .done file.
>
> Regards,
> Eric
>
> On 3/12/10 12:05 PM, "Corbin Hoenes" <co...@tynt.com> wrote:
>
>> Is there are way to rerun the data stored InError through the demux process
>> again? Or is that bucket more for debugging and investigation?
>>
>
Re: reprocess *InError
Posted by Eric Yang <ey...@yahoo-inc.com>.
Final archive is base on today's date. Hence, use cp or mv to .done wisely.
:)
Regards,
Eric
On 4/7/10 3:49 PM, "Corbin Hoenes" <co...@tynt.com> wrote:
> Thanks Eric.
>
> I am going to have to redemux a bunch of data since our custom processor has
> changed a bit and we parse data and store into the fields map for pig to read
> easily.
>
> I think I can do this simply by copying from /chukwa/finalArchives/20100309
> then renaming to .done.
>
> Once I do that does the data I reprocess overwrite the existing data in
> /chukwa/repos or do I need to remove that data as well?
>
> Once it's completed does it put the data into the finalArchives/20100309 dir
> or put into finalArchives/[TODAY'S DATE]? So if I ever needed to reprocess
> again I'd loose the ability to reprocess by date?
>
>
> On Mar 12, 2010, at 2:07 PM, Eric Yang wrote:
>
>> It's for debug. You should be able to reprocess original data by copying
>> archive for that time range back to sink directory and rename to .done file.
>>
>> Regards,
>> Eric
>>
>> On 3/12/10 12:05 PM, "Corbin Hoenes" <co...@tynt.com> wrote:
>>
>>> Is there are way to rerun the data stored InError through the demux process
>>> again? Or is that bucket more for debugging and investigation?
>>>
>>
>