You are viewing a plain text version of this content. The canonical link for it is here.
Posted to droids-commits@incubator.apache.org by ol...@apache.org on 2009/03/21 22:48:14 UTC

svn commit: r757049 - /incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java

Author: olegk
Date: Sat Mar 21 22:48:14 2009
New Revision: 757049

URL: http://svn.apache.org/viewvc?rev=757049&view=rev
Log:
Fixed bug causing IllegalStateException if robots.txt is not found

Modified:
    incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java

Modified: incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java
URL: http://svn.apache.org/viewvc/incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java?rev=757049&r1=757048&r2=757049&view=diff
==============================================================================
--- incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java (original)
+++ incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java Sat Mar 21 22:48:14 2009
@@ -94,14 +94,16 @@
    */
   public void parse(URI baseUri) throws IOException, NoRobotException {
     URI uri = resolveURI(baseUri, "robots.txt");
+    baseURI = baseUri;
+    robotsURI = uri;
+    rules = null;
+    wildcardRules = null;
     // fetch baseUrl+"robots.txt"
     if (!contentLoader.exists(uri)) {
       return;
     }
     InputStream instream = contentLoader.load(uri);
     doParseText(instream);
-    baseURI = baseUri;
-    robotsURI = uri;
   }
 
   public void parseText(InputStream instream) throws IOException, NoRobotException {
@@ -213,7 +215,7 @@
    * @throws IllegalStateException when parse has not been called
    */
   public boolean isUrlAllowed(URI uri) throws IllegalStateException, IllegalArgumentException {
-    if(rules == null || baseURI == null || robotsURI == null) {
+    if (baseURI == null || robotsURI == null) {
       throw new IllegalStateException("You must call parse before you call this method.  ");
     }
 
@@ -244,9 +246,9 @@
       // ASCII always supported
       return false;
     }
-    Boolean allowed = this.rules.isAllowed( path );
+    Boolean allowed = this.rules != null ? this.rules.isAllowed( path ) : null;
     if(allowed == null) {
-      allowed = this.wildcardRules.isAllowed( path );
+      allowed = this.wildcardRules != null ? this.wildcardRules.isAllowed( path ) : null;
     }
     if(allowed == null) {
       allowed = Boolean.TRUE;



Re: svn commit: r757049 - /incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java

Posted by Thorsten Scherler <th...@apache.org>.
On Sat, 2009-03-21 at 22:48 +0000, olegk@apache.org wrote:
> Author: olegk
> Date: Sat Mar 21 22:48:14 2009
> New Revision: 757049
> 
> URL: http://svn.apache.org/viewvc?rev=757049&view=rev
> Log:
> Fixed bug causing IllegalStateException if robots.txt is not found

Thanks very much oleg. :)

salu2
-- 
Thorsten Scherler <thorsten.at.apache.org>
Open Source <consulting, training and solutions>