You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nutch.apache.org by ma...@apache.org on 2011/11/10 16:24:31 UTC
svn commit: r1200377 - in /nutch/trunk: CHANGES.txt
src/java/org/apache/nutch/util/domain/DomainStatistics.java
Author: markus
Date: Thu Nov 10 15:24:30 2011
New Revision: 1200377
URL: http://svn.apache.org/viewvc?rev=1200377&view=rev
Log:
NUTCH-1173 DomainStats doesn't count db_not_modified
Modified:
nutch/trunk/CHANGES.txt
nutch/trunk/src/java/org/apache/nutch/util/domain/DomainStatistics.java
Modified: nutch/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/nutch/trunk/CHANGES.txt?rev=1200377&r1=1200376&r2=1200377&view=diff
==============================================================================
--- nutch/trunk/CHANGES.txt (original)
+++ nutch/trunk/CHANGES.txt Thu Nov 10 15:24:30 2011
@@ -1,5 +1,7 @@
Nutch Change Log
+* NUTCH-1173 DomainStats doesn't count db_not_modified (markus)
+
* NUTCH-1155 Host/domain limit in generator is generate.max.count+1 (markus)
* NUTCH-1061 Migrate MoreIndexingFilter from Apache ORO to java.util.regex (markus)
Modified: nutch/trunk/src/java/org/apache/nutch/util/domain/DomainStatistics.java
URL: http://svn.apache.org/viewvc/nutch/trunk/src/java/org/apache/nutch/util/domain/DomainStatistics.java?rev=1200377&r1=1200376&r2=1200377&view=diff
==============================================================================
--- nutch/trunk/src/java/org/apache/nutch/util/domain/DomainStatistics.java (original)
+++ nutch/trunk/src/java/org/apache/nutch/util/domain/DomainStatistics.java Thu Nov 10 15:24:30 2011
@@ -141,9 +141,10 @@ implements Tool, Mapper<Text, CrawlDatum
public void map(Text urlText, CrawlDatum datum,
OutputCollector<Text, LongWritable> output, Reporter reporter)
throws IOException {
-
- if(datum.getStatus() == CrawlDatum.STATUS_DB_FETCHED
- || datum.getStatus() == CrawlDatum.STATUS_FETCH_SUCCESS) {
+
+ if(datum.getStatus() == CrawlDatum.STATUS_DB_FETCHED
+ || datum.getStatus() == CrawlDatum.STATUS_DB_NOTMODIFIED) {
+
try {
URL url = new URL(urlText.toString());
String out = null;