You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nutch.apache.org by te...@apache.org on 2014/01/24 18:45:51 UTC

svn commit: r1561087 - in /nutch/trunk: CHANGES.txt src/java/org/apache/nutch/protocol/RobotRulesParser.java

Author: tejasp
Date: Fri Jan 24 17:45:51 2014
New Revision: 1561087

URL: http://svn.apache.org/r1561087
Log:
NUTCH-1715 RobotRulesParser adds additional * to the robots name

Modified:
    nutch/trunk/CHANGES.txt
    nutch/trunk/src/java/org/apache/nutch/protocol/RobotRulesParser.java

Modified: nutch/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/nutch/trunk/CHANGES.txt?rev=1561087&r1=1561086&r2=1561087&view=diff
==============================================================================
--- nutch/trunk/CHANGES.txt (original)
+++ nutch/trunk/CHANGES.txt Fri Jan 24 17:45:51 2014
@@ -2,6 +2,8 @@ Nutch Change Log
 
 Nutch Development Trunk
 
+* NUTCH-1715 RobotRulesParser adds additional '*' to the robots name (tejasp)
+
 * NUTCH-356 Plugin repository cache can lead to memory leak (Enrico Triolo, Doğacan Güney via markus)
 
 * NUTCH-1413 Record response time (Yasin Kılınç, Talat Uyarer, snagel)

Modified: nutch/trunk/src/java/org/apache/nutch/protocol/RobotRulesParser.java
URL: http://svn.apache.org/viewvc/nutch/trunk/src/java/org/apache/nutch/protocol/RobotRulesParser.java?rev=1561087&r1=1561086&r2=1561087&view=diff
==============================================================================
--- nutch/trunk/src/java/org/apache/nutch/protocol/RobotRulesParser.java (original)
+++ nutch/trunk/src/java/org/apache/nutch/protocol/RobotRulesParser.java Fri Jan 24 17:45:51 2014
@@ -121,8 +121,6 @@ public abstract class RobotRulesParser i
         combinedAgentsString.append(", " + agents.get(index));
       }
 
-      // always make sure "*" is included in the end
-      combinedAgentsString.append(", *");
       this.agentNames = combinedAgentsString.toString();
     }
   }