[jira] Resolved: (NUTCH-866) STOP Nutch without breaking the crawled data

["Julien Nioche (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NUTCH-866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Julien Nioche resolved NUTCH-866.
---------------------------------

    Resolution: Not A Problem

> STOP Nutch without breaking the crawled data
> --------------------------------------------
>
>                 Key: NUTCH-866
>                 URL: https://issues.apache.org/jira/browse/NUTCH-866
>             Project: Nutch
>          Issue Type: New Feature
>            Reporter: Pham Tuan Minh
>             Fix For: 2.0
>
>
> How we can stop running nutch instance in local mode and in reducer mode without breaking the crawled data? 
> For example, you push a list of site that take a long time to complete crawl; then you want to stop nutch instance suddenly ...
> - For local mode, I suggest as below
> We create a stop.txt file in specific directory, then for a piece of time, nutch instance will check whether this file existed or not; if existed, nutch instance will stop itself normally
> - For reducer mode, may we use zookeper to keep state of each instance?
> Any other suggestion?
> Thanks,

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.