Re: [jira] Commented: (CHUKWA-4) Collectors don't finish writing .done datasink from last .chukwa datasink when stopped using bin/stop-collectors

[Ahmed Fathalla <af...@gmail.com>]

So, what would be an effective solution, reopen the FileSystem to rename the
file from *.chukwa to *.done?

On Fri, Mar 26, 2010 at 6:38 PM, Jerome Boulon (JIRA) <ji...@apache.org>wrote:

>
>    [
> https://issues.apache.org/jira/browse/CHUKWA-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12850206#action_12850206]
>
> Jerome Boulon commented on CHUKWA-4:
> ------------------------------------
>
> This is because HDFS is closing the fileSystem on kill -term
> (shutdownHook).
> The workaround is to remove the HDFS shurdown hook and call it after the
> collector close but it's just a workaround.
>
> > Collectors don't finish writing .done datasink from last .chukwa datasink
> when stopped using bin/stop-collectors
> >
> ----------------------------------------------------------------------------------------------------------------
> >
> >                 Key: CHUKWA-4
> >                 URL: https://issues.apache.org/jira/browse/CHUKWA-4
> >             Project: Hadoop Chukwa
> >          Issue Type: Bug
> >          Components: data collection
> >         Environment: I am running on our local cluster. This is a linux
> machine that I also run Hadoop cluster from.
> >            Reporter: Andy Konwinski
> >            Priority: Minor
> >
> > When I use start-collectors, it creates the datasink as expected, writes
> to it as per normal, i.e. writes to the .chukwa file, and roll overs work
> fine when it renames the .chukwa file to .done. However, when I use
> bin/stop-collectors to shut down the running collector it leaves a .chukwa
> file in the HDFS file system. Not sure if this is a valid sink or not, but I
> think that the collector should gracefully clean up the datasink and rename
> it .done before exiting.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>


-- 
Ahmed Fathalla