You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2019/10/24 14:11:00 UTC
[jira] [Commented] (NUTCH-2750) improve CrawlDbReader &
LinkDbReader reader handling
[ https://issues.apache.org/jira/browse/NUTCH-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958914#comment-16958914 ]
ASF GitHub Bot commented on NUTCH-2750:
---------------------------------------
lewismc commented on issue #483: Fix for NUTCH-2750
URL: https://github.com/apache/nutch/pull/483#issuecomment-545936257
@sju how would one go about testing this patch? Thank you
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> improve CrawlDbReader & LinkDbReader reader handling
> ----------------------------------------------------
>
> Key: NUTCH-2750
> URL: https://issues.apache.org/jira/browse/NUTCH-2750
> Project: Nutch
> Issue Type: Improvement
> Components: crawldb, linkdb
> Affects Versions: 1.16
> Reporter: Jurian Broertjes
> Priority: Minor
>
> The current implementation in the CrawlDbReader re-opens readers for every URL. This is not very efficient. I've implemented a modification time check that only re-opens readers on updated crawlDB.
> PR: https://github.com/apache/nutch/pull/483
--
This message was sent by Atlassian Jira
(v8.3.4#803005)