You are viewing a plain text version of this content. The canonical link for it is here.
Posted to droids-commits@incubator.apache.org by ol...@apache.org on 2009/03/21 22:48:14 UTC
svn commit: r757049 -
/incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java
Author: olegk
Date: Sat Mar 21 22:48:14 2009
New Revision: 757049
URL: http://svn.apache.org/viewvc?rev=757049&view=rev
Log:
Fixed bug causing IllegalStateException if robots.txt is not found
Modified:
incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java
Modified: incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java
URL: http://svn.apache.org/viewvc/incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java?rev=757049&r1=757048&r2=757049&view=diff
==============================================================================
--- incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java (original)
+++ incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java Sat Mar 21 22:48:14 2009
@@ -94,14 +94,16 @@
*/
public void parse(URI baseUri) throws IOException, NoRobotException {
URI uri = resolveURI(baseUri, "robots.txt");
+ baseURI = baseUri;
+ robotsURI = uri;
+ rules = null;
+ wildcardRules = null;
// fetch baseUrl+"robots.txt"
if (!contentLoader.exists(uri)) {
return;
}
InputStream instream = contentLoader.load(uri);
doParseText(instream);
- baseURI = baseUri;
- robotsURI = uri;
}
public void parseText(InputStream instream) throws IOException, NoRobotException {
@@ -213,7 +215,7 @@
* @throws IllegalStateException when parse has not been called
*/
public boolean isUrlAllowed(URI uri) throws IllegalStateException, IllegalArgumentException {
- if(rules == null || baseURI == null || robotsURI == null) {
+ if (baseURI == null || robotsURI == null) {
throw new IllegalStateException("You must call parse before you call this method. ");
}
@@ -244,9 +246,9 @@
// ASCII always supported
return false;
}
- Boolean allowed = this.rules.isAllowed( path );
+ Boolean allowed = this.rules != null ? this.rules.isAllowed( path ) : null;
if(allowed == null) {
- allowed = this.wildcardRules.isAllowed( path );
+ allowed = this.wildcardRules != null ? this.wildcardRules.isAllowed( path ) : null;
}
if(allowed == null) {
allowed = Boolean.TRUE;
Re: svn commit: r757049 -
/incubator/droids/trunk/droids-norobots/src/main/java/org/apache/droids/norobots/NoRobotClient.java
Posted by Thorsten Scherler <th...@apache.org>.
On Sat, 2009-03-21 at 22:48 +0000, olegk@apache.org wrote:
> Author: olegk
> Date: Sat Mar 21 22:48:14 2009
> New Revision: 757049
>
> URL: http://svn.apache.org/viewvc?rev=757049&view=rev
> Log:
> Fixed bug causing IllegalStateException if robots.txt is not found
Thanks very much oleg. :)
salu2
--
Thorsten Scherler <thorsten.at.apache.org>
Open Source <consulting, training and solutions>