You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Markus Jelsma (Created) (JIRA)" <ji...@apache.org> on 2011/10/25 01:36:32 UTC
[jira] [Created] (NUTCH-1180) UpdateDB to backup previous CrawlDB
UpdateDB to backup previous CrawlDB
-----------------------------------
Key: NUTCH-1180
URL: https://issues.apache.org/jira/browse/NUTCH-1180
Project: Nutch
Issue Type: Improvement
Reporter: Markus Jelsma
Assignee: Markus Jelsma
Priority: Minor
Fix For: 1.5
Nutch currently replaces an existing CrawlDB with the new CrawlDB. By optionally keeping a previous version on HDFS users can easily revert in case of a mistake without relying on external backup mechanims.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1180) UpdateDB to backup previous CrawlDB
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149002#comment-13149002 ]
Hudson commented on NUTCH-1180:
-------------------------------
Integrated in Nutch-trunk #1660 (See [https://builds.apache.org/job/Nutch-trunk/1660/])
NUTCH-1180 UpdateDB to backup previous CrawlDB
markus : http://svn.apache.org/viewvc/nutch/trunk/viewvc/?view=rev&root=&revision=1200830
Files :
* /nutch/trunk/CHANGES.txt
* /nutch/trunk/conf/nutch-default.xml
* /nutch/trunk/src/java/org/apache/nutch/crawl/CrawlDb.java
> UpdateDB to backup previous CrawlDB
> -----------------------------------
>
> Key: NUTCH-1180
> URL: https://issues.apache.org/jira/browse/NUTCH-1180
> Project: Nutch
> Issue Type: Improvement
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Fix For: 1.5
>
> Attachments: NUTCH-1180-1.5.-1.patch
>
>
> Nutch currently replaces an existing CrawlDB with the new CrawlDB. By optionally keeping a previous version on HDFS users can easily revert in case of a mistake without relying on external backup mechanims.
> This should be enabled by default.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (NUTCH-1180) UpdateDB to backup previous CrawlDB
Posted by "Markus Jelsma (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Markus Jelsma resolved NUTCH-1180.
----------------------------------
Resolution: Fixed
Committed for 1.5 in rev. 1200830.
> UpdateDB to backup previous CrawlDB
> -----------------------------------
>
> Key: NUTCH-1180
> URL: https://issues.apache.org/jira/browse/NUTCH-1180
> Project: Nutch
> Issue Type: Improvement
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Fix For: 1.5
>
> Attachments: NUTCH-1180-1.5.-1.patch
>
>
> Nutch currently replaces an existing CrawlDB with the new CrawlDB. By optionally keeping a previous version on HDFS users can easily revert in case of a mistake without relying on external backup mechanims.
> This should be enabled by default.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1180) UpdateDB to backup previous CrawlDB
Posted by "Markus Jelsma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147715#comment-13147715 ]
Markus Jelsma commented on NUTCH-1180:
--------------------------------------
Config directive:
{code}
<property>
<name>db.preserve.backup</name>
<value>true</value>
<description>If true, updatedb will keep a backup of the previous CrawlDB
version in the old directory. In case of disaster, one can rename old to
current and restore the CrawlDB to its previous state.
</description>
</property>
{code}
Fine? Wrong?
> UpdateDB to backup previous CrawlDB
> -----------------------------------
>
> Key: NUTCH-1180
> URL: https://issues.apache.org/jira/browse/NUTCH-1180
> Project: Nutch
> Issue Type: Improvement
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Fix For: 1.5
>
> Attachments: NUTCH-1180-1.5.-1.patch
>
>
> Nutch currently replaces an existing CrawlDB with the new CrawlDB. By optionally keeping a previous version on HDFS users can easily revert in case of a mistake without relying on external backup mechanims.
> This should be enabled by default.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1180) UpdateDB to backup previous CrawlDB
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149032#comment-13149032 ]
Hudson commented on NUTCH-1180:
-------------------------------
Integrated in Nutch-trunk-ant #77 (See [https://builds.apache.org/job/Nutch-trunk-ant/77/])
NUTCH-1180 UpdateDB to backup previous CrawlDB
markus : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1200830
Files :
* /nutch/trunk/CHANGES.txt
* /nutch/trunk/conf/nutch-default.xml
* /nutch/trunk/src/java/org/apache/nutch/crawl/CrawlDb.java
> UpdateDB to backup previous CrawlDB
> -----------------------------------
>
> Key: NUTCH-1180
> URL: https://issues.apache.org/jira/browse/NUTCH-1180
> Project: Nutch
> Issue Type: Improvement
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Fix For: 1.5
>
> Attachments: NUTCH-1180-1.5.-1.patch
>
>
> Nutch currently replaces an existing CrawlDB with the new CrawlDB. By optionally keeping a previous version on HDFS users can easily revert in case of a mistake without relying on external backup mechanims.
> This should be enabled by default.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1180) UpdateDB to backup previous CrawlDB
Posted by "Markus Jelsma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147641#comment-13147641 ]
Markus Jelsma commented on NUTCH-1180:
--------------------------------------
I'll send this in if there are no objections.
> UpdateDB to backup previous CrawlDB
> -----------------------------------
>
> Key: NUTCH-1180
> URL: https://issues.apache.org/jira/browse/NUTCH-1180
> Project: Nutch
> Issue Type: Improvement
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Fix For: 1.5
>
> Attachments: NUTCH-1180-1.5.-1.patch
>
>
> Nutch currently replaces an existing CrawlDB with the new CrawlDB. By optionally keeping a previous version on HDFS users can easily revert in case of a mistake without relying on external backup mechanims.
> This should be enabled by default.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1180) UpdateDB to backup previous CrawlDB
Posted by "Markus Jelsma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Markus Jelsma updated NUTCH-1180:
---------------------------------
Patch Info: Patch Available
> UpdateDB to backup previous CrawlDB
> -----------------------------------
>
> Key: NUTCH-1180
> URL: https://issues.apache.org/jira/browse/NUTCH-1180
> Project: Nutch
> Issue Type: Improvement
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Fix For: 1.5
>
> Attachments: NUTCH-1180-1.5.-1.patch
>
>
> Nutch currently replaces an existing CrawlDB with the new CrawlDB. By optionally keeping a previous version on HDFS users can easily revert in case of a mistake without relying on external backup mechanims.
> This should be enabled by default.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1180) UpdateDB to backup previous CrawlDB
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148444#comment-13148444 ]
Hudson commented on NUTCH-1180:
-------------------------------
Integrated in nutch-trunk-maven #19 (See [https://builds.apache.org/job/nutch-trunk-maven/19/])
NUTCH-1180 UpdateDB to backup previous CrawlDB
markus : http://svn.apache.org/viewvc/nutch/trunk/viewvc/?view=rev&root=&revision=1200830
Files :
* /nutch/trunk/CHANGES.txt
* /nutch/trunk/conf/nutch-default.xml
* /nutch/trunk/src/java/org/apache/nutch/crawl/CrawlDb.java
> UpdateDB to backup previous CrawlDB
> -----------------------------------
>
> Key: NUTCH-1180
> URL: https://issues.apache.org/jira/browse/NUTCH-1180
> Project: Nutch
> Issue Type: Improvement
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Fix For: 1.5
>
> Attachments: NUTCH-1180-1.5.-1.patch
>
>
> Nutch currently replaces an existing CrawlDB with the new CrawlDB. By optionally keeping a previous version on HDFS users can easily revert in case of a mistake without relying on external backup mechanims.
> This should be enabled by default.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1180) UpdateDB to backup previous CrawlDB
Posted by "Markus Jelsma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Markus Jelsma updated NUTCH-1180:
---------------------------------
Description:
Nutch currently replaces an existing CrawlDB with the new CrawlDB. By optionally keeping a previous version on HDFS users can easily revert in case of a mistake without relying on external backup mechanims.
This should be enabled by default.
was:Nutch currently replaces an existing CrawlDB with the new CrawlDB. By optionally keeping a previous version on HDFS users can easily revert in case of a mistake without relying on external backup mechanims.
> UpdateDB to backup previous CrawlDB
> -----------------------------------
>
> Key: NUTCH-1180
> URL: https://issues.apache.org/jira/browse/NUTCH-1180
> Project: Nutch
> Issue Type: Improvement
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Fix For: 1.5
>
>
> Nutch currently replaces an existing CrawlDB with the new CrawlDB. By optionally keeping a previous version on HDFS users can easily revert in case of a mistake without relying on external backup mechanims.
> This should be enabled by default.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1180) UpdateDB to backup previous CrawlDB
Posted by "Markus Jelsma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Markus Jelsma updated NUTCH-1180:
---------------------------------
Attachment: NUTCH-1180-1.5.-1.patch
Code was already there, only needed to conditionally not delete the old DB. Patch introduces a db.preserve.backup boolean directive which is enabled by default.
> UpdateDB to backup previous CrawlDB
> -----------------------------------
>
> Key: NUTCH-1180
> URL: https://issues.apache.org/jira/browse/NUTCH-1180
> Project: Nutch
> Issue Type: Improvement
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Fix For: 1.5
>
> Attachments: NUTCH-1180-1.5.-1.patch
>
>
> Nutch currently replaces an existing CrawlDB with the new CrawlDB. By optionally keeping a previous version on HDFS users can easily revert in case of a mistake without relying on external backup mechanims.
> This should be enabled by default.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira