You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Markus Jelsma (Created) (JIRA)" <ji...@apache.org> on 2012/04/06 15:09:23 UTC
[jira] [Created] (NUTCH-1330) OutlinkDB to preserve back up
OutlinkDB to preserve back up
-----------------------------
Key: NUTCH-1330
URL: https://issues.apache.org/jira/browse/NUTCH-1330
Project: Nutch
Issue Type: Improvement
Affects Versions: 1.4
Reporter: Markus Jelsma
Assignee: Markus Jelsma
Fix For: 1.6
The webgraph's outlinkDB is the single source for all scoring jobs and GB's that eventually come out. In case of disaster, that didn't happen yet, it should be able to preserve back up just like other DB's. This means users with an existing outlinkdb must move it from a crawl/webgraphdb/outlinks/ to crawl/webgraphdb/outlinks/current/.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (NUTCH-1330) OutlinkDB to preserve back up
Posted by "Markus Jelsma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Markus Jelsma resolved NUTCH-1330.
----------------------------------
Resolution: Fixed
Committed for 1.6 in rev. 1349240.
> OutlinkDB to preserve back up
> -----------------------------
>
> Key: NUTCH-1330
> URL: https://issues.apache.org/jira/browse/NUTCH-1330
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 1.4
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.6
>
> Attachments: NUTCH-1330-1.6-1.patch, NUTCH-1330-1.6-2.patch
>
>
> The webgraph's outlinkDB is the single source for all scoring jobs and GB's that eventually come out. In case of disaster, that didn't happen yet, it should be able to preserve back up just like other DB's. This means users with an existing outlinkdb must move it from a crawl/webgraphdb/outlinks/ to crawl/webgraphdb/outlinks/current/.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1330) OutlinkDB to preserve back up
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295799#comment-13295799 ]
Hudson commented on NUTCH-1330:
-------------------------------
Integrated in Nutch-trunk #1869 (See [https://builds.apache.org/job/Nutch-trunk/1869/])
NUTCH-1330 WebGraph OutlinkDB to preserve back up (Revision 1349240)
Result = SUCCESS
markus : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349240
Files :
* /nutch/trunk/CHANGES.txt
* /nutch/trunk/src/java/org/apache/nutch/scoring/webgraph/WebGraph.java
> OutlinkDB to preserve back up
> -----------------------------
>
> Key: NUTCH-1330
> URL: https://issues.apache.org/jira/browse/NUTCH-1330
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 1.4
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.6
>
> Attachments: NUTCH-1330-1.6-1.patch, NUTCH-1330-1.6-2.patch
>
>
> The webgraph's outlinkDB is the single source for all scoring jobs and GB's that eventually come out. In case of disaster, that didn't happen yet, it should be able to preserve back up just like other DB's. This means users with an existing outlinkdb must move it from a crawl/webgraphdb/outlinks/ to crawl/webgraphdb/outlinks/current/.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1330) OutlinkDB to preserve back up
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13293545#comment-13293545 ]
Hudson commented on NUTCH-1330:
-------------------------------
Integrated in nutch-trunk-maven #310 (See [https://builds.apache.org/job/nutch-trunk-maven/310/])
NUTCH-1330 WebGraph OutlinkDB to preserve back up (Revision 1349240)
Result = SUCCESS
markus :
Files :
* /nutch/trunk/CHANGES.txt
* /nutch/trunk/src/java/org/apache/nutch/scoring/webgraph/WebGraph.java
> OutlinkDB to preserve back up
> -----------------------------
>
> Key: NUTCH-1330
> URL: https://issues.apache.org/jira/browse/NUTCH-1330
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 1.4
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.6
>
> Attachments: NUTCH-1330-1.6-1.patch, NUTCH-1330-1.6-2.patch
>
>
> The webgraph's outlinkDB is the single source for all scoring jobs and GB's that eventually come out. In case of disaster, that didn't happen yet, it should be able to preserve back up just like other DB's. This means users with an existing outlinkdb must move it from a crawl/webgraphdb/outlinks/ to crawl/webgraphdb/outlinks/current/.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1330) OutlinkDB to preserve back up
Posted by "Markus Jelsma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253422#comment-13253422 ]
Markus Jelsma commented on NUTCH-1330:
--------------------------------------
The db.preserve.backup was introduced in NUTCH-1180 and is present in the conf and indeed defaults to true. It will hopefully save some users from disaster.
> OutlinkDB to preserve back up
> -----------------------------
>
> Key: NUTCH-1330
> URL: https://issues.apache.org/jira/browse/NUTCH-1330
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 1.4
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.6
>
> Attachments: NUTCH-1330-1.6-1.patch, NUTCH-1330-1.6-2.patch
>
>
> The webgraph's outlinkDB is the single source for all scoring jobs and GB's that eventually come out. In case of disaster, that didn't happen yet, it should be able to preserve back up just like other DB's. This means users with an existing outlinkdb must move it from a crawl/webgraphdb/outlinks/ to crawl/webgraphdb/outlinks/current/.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1330) OutlinkDB to preserve back up
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253447#comment-13253447 ]
Lewis John McGibbney commented on NUTCH-1330:
---------------------------------------------
Nice 1 Markus. +1
> OutlinkDB to preserve back up
> -----------------------------
>
> Key: NUTCH-1330
> URL: https://issues.apache.org/jira/browse/NUTCH-1330
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 1.4
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.6
>
> Attachments: NUTCH-1330-1.6-1.patch, NUTCH-1330-1.6-2.patch
>
>
> The webgraph's outlinkDB is the single source for all scoring jobs and GB's that eventually come out. In case of disaster, that didn't happen yet, it should be able to preserve back up just like other DB's. This means users with an existing outlinkdb must move it from a crawl/webgraphdb/outlinks/ to crawl/webgraphdb/outlinks/current/.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1330) OutlinkDB to preserve back up
Posted by "Markus Jelsma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13293502#comment-13293502 ]
Markus Jelsma commented on NUTCH-1330:
--------------------------------------
Thanks Lewis!
> OutlinkDB to preserve back up
> -----------------------------
>
> Key: NUTCH-1330
> URL: https://issues.apache.org/jira/browse/NUTCH-1330
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 1.4
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.6
>
> Attachments: NUTCH-1330-1.6-1.patch, NUTCH-1330-1.6-2.patch
>
>
> The webgraph's outlinkDB is the single source for all scoring jobs and GB's that eventually come out. In case of disaster, that didn't happen yet, it should be able to preserve back up just like other DB's. This means users with an existing outlinkdb must move it from a crawl/webgraphdb/outlinks/ to crawl/webgraphdb/outlinks/current/.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1330) OutlinkDB to preserve back up
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253405#comment-13253405 ]
Lewis John McGibbney commented on NUTCH-1330:
---------------------------------------------
Hi Markus. Based upon the assumption that you add the new 'db.preserve.backup' property to nutch.default and set to true by default then yeah I'm a +1. This is a nice patch and saves some annoying manual work.
> OutlinkDB to preserve back up
> -----------------------------
>
> Key: NUTCH-1330
> URL: https://issues.apache.org/jira/browse/NUTCH-1330
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 1.4
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.6
>
> Attachments: NUTCH-1330-1.6-1.patch, NUTCH-1330-1.6-2.patch
>
>
> The webgraph's outlinkDB is the single source for all scoring jobs and GB's that eventually come out. In case of disaster, that didn't happen yet, it should be able to preserve back up just like other DB's. This means users with an existing outlinkdb must move it from a crawl/webgraphdb/outlinks/ to crawl/webgraphdb/outlinks/current/.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1330) OutlinkDB to preserve back up
Posted by "Markus Jelsma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Markus Jelsma updated NUTCH-1330:
---------------------------------
Attachment: NUTCH-1330-1.6-2.patch
Previous patch is bad and came from an old checkout. This is the proper patch.
> OutlinkDB to preserve back up
> -----------------------------
>
> Key: NUTCH-1330
> URL: https://issues.apache.org/jira/browse/NUTCH-1330
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 1.4
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.6
>
> Attachments: NUTCH-1330-1.6-1.patch, NUTCH-1330-1.6-2.patch
>
>
> The webgraph's outlinkDB is the single source for all scoring jobs and GB's that eventually come out. In case of disaster, that didn't happen yet, it should be able to preserve back up just like other DB's. This means users with an existing outlinkdb must move it from a crawl/webgraphdb/outlinks/ to crawl/webgraphdb/outlinks/current/.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1330) OutlinkDB to preserve back up
Posted by "Markus Jelsma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Markus Jelsma updated NUTCH-1330:
---------------------------------
Attachment: NUTCH-1330-1.6-1.patch
Patch for 1.6!
> OutlinkDB to preserve back up
> -----------------------------
>
> Key: NUTCH-1330
> URL: https://issues.apache.org/jira/browse/NUTCH-1330
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 1.4
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.6
>
> Attachments: NUTCH-1330-1.6-1.patch
>
>
> The webgraph's outlinkDB is the single source for all scoring jobs and GB's that eventually come out. In case of disaster, that didn't happen yet, it should be able to preserve back up just like other DB's. This means users with an existing outlinkdb must move it from a crawl/webgraphdb/outlinks/ to crawl/webgraphdb/outlinks/current/.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira