You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Eugene Dzhurinsky <bo...@redwerk.com> on 2006/07/04 17:46:09 UTC
invalid redirect
Hello, I facing strange issue with HttpClient
For this URL: http://www.emigrant.ie/cgi-bin/ads.pl?member=atwork_side3;page=781;setdest=atwork_side3
it reports 302 instead of 200, and
WARNING: Redirected location
'http://www.arguscarhire.com/Newimages/BANNERS/argus_car_hire_130x60_from?9.99.gif'
is malformed
You may find unit test below
import junit.framework.Assert;
import junit.framework.TestCase;
import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.cookie.CookiePolicy;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.params.HttpClientParams;
import org.apache.commons.httpclient.params.HttpMethodParams;
import java.io.IOException;
public class HTTPClientTest extends TestCase {
HttpClient client;
HttpMethod method;
public static void main(String[] args) {
TestCase test = new HTTPClientTest() {
public void runTest() {
testResultcode();
};
};
}
protected void setUp() throws Exception {
client = new HttpClient();
String url = "http://www.emigrant.ie/cgi-bin/ads.pl?member=atwork_side3;page=781;setdest=atwork_side3";
URI uri = new URI(url, url.indexOf('%') != -1);
method = new GetMethod();
method.setURI(uri);
method.setRequestHeader("HTTP_REFERER",
"http://www.emigrant.ie/category.asp?iCategoryID=510");
HttpMethodParams params = method.getParams();
params.setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY);
params.setParameter(HttpClientParams.USER_AGENT, "Mozilla/4.0"
+ "(compatible; MSIE 6.0; Windows NT 5.1;)");
method.setFollowRedirects(true);
}
public void testResultcode() {
try {
client.executeMethod(method);
Assert.assertEquals(HttpStatus.SC_OK, method.getStatusCode());
} catch (HttpException e) {
e.printStackTrace(System.out);
} catch (IOException e) {
e.printStackTrace(System.out);
}
}
}
--
Eugene N Dzhurinsky
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
Re: invalid redirect
Posted by Roland Weber <ht...@dubioso.net>.
>>But browsers like Firefox doesn't face such problems.
>
>
> HttpClient is not a browser
See also the last paragraphs in sections 3.1 and 3.2:
http://wiki.apache.org/jakarta-httpclient/ForAbsoluteBeginners#head-a110969063be34fcd964aeba55ae23bea12ac232
>>Should I take care of redirects somehow to normalize URLs if they are not
>>normalized yet?
>
> Yes, you should
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
Re: invalid redirect
Posted by Roland Weber <ht...@dubioso.net>.
Hello,
> Could you please explain, how would I deal with relative URIs, may be there is
> simple way to validate if the URI is relative and how to handle it?
The HttpClient URI class as well as the JDK URI and URL classes have
constructors that deal with relative URLs. For example this one:
http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/URI.html#URI(org.apache.commons.httpclient.URI,%20java.lang.String,%20boolean)
Pass in the target URL of the request as the base URL.
hope that helps,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
Re: invalid redirect
Posted by bo...@redwerk.com.
On Wed, Jul 05, 2006 at 07:42:53PM +0200, Roland Weber wrote:
> Hello Eugeny,
> > Could you please explain how can I handle redirects? Is there any way for
> > placing hook or register some callback?
> Disable automatic redirects so your application will get the 30x response.
> The redirect target is given in the Location: response header. Convert it
> to a valid, absolute URL and have your application send another request
> for the new URL.
Okay, so in general after executeMethod I need to test it for header
"location", if this header exists - I need to create another URI, create new
Get method and execute it unless redirect limit exceeds or location header is
missing.
Could you please explain, how would I deal with relative URIs, may be there is
simple way to validate if the URI is relative and how to handle it?
for instance, if i query page
http://somesite.com/somedir/someanotherdir/page.html and it returns
/index.html, the resulting URL must be http://somesite.com/index.html - not
http://somesite.com/somedir/someanotherdir/index.html
I know it is done internally in HttpClient somehow (when automatic redirects
are enabled), but I looked at sources and got confused.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
Re: invalid redirect
Posted by Roland Weber <ht...@dubioso.net>.
Hello Eugeny,
>>HttpClient is not a browser
>
>
> But as I can see in
> http://wiki.apache.org/jakarta-httpclient/ForAbsoluteBeginners#head-a110969063be34fcd964aeba55ae23bea12ac232
>
> it should provide handling of HTTP protocol, including some redirect support
> and cookie management?
Yes, and so it does. It provides support for valid redirect URLs, and some
support for not quite valid redirect URLs. Your problem is a URL that is
so invalid that it exceeds the fault tolerance of HttpClient.
> Could you please explain how can I handle redirects? Is there any way for
> placing hook or register some callback?
Disable automatic redirects so your application will get the 30x response.
The redirect target is given in the Location: response header. Convert it
to a valid, absolute URL and have your application send another request
for the new URL.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
Re: invalid redirect
Posted by Eugeny N Dzhurinsky <bo...@redwerk.com>.
On Tue, Jul 04, 2006 at 07:35:56PM +0200, Oleg Kalnichevski wrote:
> On Tue, 2006-07-04 at 19:32 +0200, bofh@redwerk.com wrote:
> > On Tue, Jul 04, 2006 at 06:49:28PM +0200, Oleg Kalnichevski wrote:
> > > On Tue, 2006-07-04 at 18:46 +0300, Eugene Dzhurinsky wrote:
> > > > Hello, I facing strange issue with HttpClient
> > > >
> > > > For this URL: http://www.emigrant.ie/cgi-bin/ads.pl?member=atwork_side3;page=781;setdest=atwork_side3
> > > >
> > > > it reports 302 instead of 200, and
> > > >
> > > > WARNING: Redirected location
> > > > 'http://www.arguscarhire.com/Newimages/BANNERS/argus_car_hire_130x60_from?9.99.gif'
> > > > is malformed
> > > >
> > >
> > > What is so strange about it? The second redirect location contains
> > > garbage.
> > But browsers like Firefox doesn't face such problems.
> HttpClient is not a browser
But as I can see in
http://wiki.apache.org/jakarta-httpclient/ForAbsoluteBeginners#head-a110969063be34fcd964aeba55ae23bea12ac232
it should provide handling of HTTP protocol, including some redirect support
and cookie management?
> > Should I take care of redirects somehow to normalize URLs if they are not
> > normalized yet?
> Yes, you should
Could you please explain how can I handle redirects? Is there any way for
placing hook or register some callback?
--
Eugene N Dzhurinsky
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
Re: invalid redirect
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2006-07-04 at 19:32 +0200, bofh@redwerk.com wrote:
> On Tue, Jul 04, 2006 at 06:49:28PM +0200, Oleg Kalnichevski wrote:
> > On Tue, 2006-07-04 at 18:46 +0300, Eugene Dzhurinsky wrote:
> > > Hello, I facing strange issue with HttpClient
> > >
> > > For this URL: http://www.emigrant.ie/cgi-bin/ads.pl?member=atwork_side3;page=781;setdest=atwork_side3
> > >
> > > it reports 302 instead of 200, and
> > >
> > > WARNING: Redirected location
> > > 'http://www.arguscarhire.com/Newimages/BANNERS/argus_car_hire_130x60_from?9.99.gif'
> > > is malformed
> > >
> >
> > What is so strange about it? The second redirect location contains
> > garbage.
>
> But browsers like Firefox doesn't face such problems.
HttpClient is not a browser
> Should I take care of redirects somehow to normalize URLs if they are not
> normalized yet?
>
Yes, you should
Oleg
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
Re: invalid redirect
Posted by bo...@redwerk.com.
On Tue, Jul 04, 2006 at 06:49:28PM +0200, Oleg Kalnichevski wrote:
> On Tue, 2006-07-04 at 18:46 +0300, Eugene Dzhurinsky wrote:
> > Hello, I facing strange issue with HttpClient
> >
> > For this URL: http://www.emigrant.ie/cgi-bin/ads.pl?member=atwork_side3;page=781;setdest=atwork_side3
> >
> > it reports 302 instead of 200, and
> >
> > WARNING: Redirected location
> > 'http://www.arguscarhire.com/Newimages/BANNERS/argus_car_hire_130x60_from?9.99.gif'
> > is malformed
> >
>
> What is so strange about it? The second redirect location contains
> garbage.
But browsers like Firefox doesn't face such problems.
Should I take care of redirects somehow to normalize URLs if they are not
normalized yet?
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
Re: invalid redirect
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2006-07-04 at 18:46 +0300, Eugene Dzhurinsky wrote:
> Hello, I facing strange issue with HttpClient
>
> For this URL: http://www.emigrant.ie/cgi-bin/ads.pl?member=atwork_side3;page=781;setdest=atwork_side3
>
> it reports 302 instead of 200, and
>
> WARNING: Redirected location
> 'http://www.arguscarhire.com/Newimages/BANNERS/argus_car_hire_130x60_from?9.99.gif'
> is malformed
>
What is so strange about it? The second redirect location contains
garbage.
Oleg
[DEBUG] header - >>
"GET /cgi-bin/ads.pl?member=atwork_side3;page=781;setdest=atwork_side3
HTTP/1.1[\r][\n]"
[DEBUG] header - >> "User-Agent: Mozilla/4.0 (compatible; MSIE 6.0;
Windows NT 5.0)[\r][\n]"
[DEBUG] header - >> "Host: www.emigrant.ie[\r][\n]"
[DEBUG] header - >> "[\r][\n]"
[DEBUG] header - << "HTTP/1.1 302 Object Moved[\r][\n]"
[DEBUG] header - << "Location:
http://www.emigrant.ie/cgi-bin/ads.pl?member=atwork_side3;page=781;setdest=atwork_side3;checkforcookie[\r][\n]"
[DEBUG] header - << "Server: Microsoft-IIS/5.0[\r][\n]"
[DEBUG] header - << "Content-Type: text/html[\r][\n]"
[DEBUG] header - << "Connection: close[\r][\n]"
[DEBUG] header - << "Content-Length: 225[\r][\n]"
[DEBUG] header - >>
"GET /cgi-bin/ads.pl?member=atwork_side3;page=781;setdest=atwork_side3;checkforcookie HTTP/1.1[\r][\n]"
[DEBUG] header - >> "User-Agent: Mozilla/4.0 (compatible; MSIE 6.0;
Windows NT 5.0)[\r][\n]"
[DEBUG] header - >> "Host: www.emigrant.ie[\r][\n]"
[DEBUG] header - >> "[\r][\n]"
[DEBUG] header - << "HTTP/1.1 302 Object Moved[\r][\n]"
[DEBUG] header - << "Location:
http://www.arguscarhire.com/Newimages/BANNERS/argus_car_hire_130x60_from[0xef][0xbf][0xbd]9.99.gif[\r][\n]"
[DEBUG] header - << "Server: Microsoft-IIS/5.0[\r][\n]"
[DEBUG] header - << "Content-Type: text/html[\r][\n]"
[DEBUG] header - << "Connection: close[\r][\n]"
[DEBUG] header - << "Content-Length: 204[\r][\n]"
> You may find unit test below
>
> import junit.framework.Assert;
> import junit.framework.TestCase;
> import org.apache.commons.httpclient.*;
> import org.apache.commons.httpclient.cookie.CookiePolicy;
> import org.apache.commons.httpclient.methods.GetMethod;
> import org.apache.commons.httpclient.params.HttpClientParams;
> import org.apache.commons.httpclient.params.HttpMethodParams;
>
> import java.io.IOException;
>
> public class HTTPClientTest extends TestCase {
>
> HttpClient client;
>
> HttpMethod method;
>
> public static void main(String[] args) {
> TestCase test = new HTTPClientTest() {
> public void runTest() {
> testResultcode();
> };
> };
> }
>
> protected void setUp() throws Exception {
> client = new HttpClient();
> String url = "http://www.emigrant.ie/cgi-bin/ads.pl?member=atwork_side3;page=781;setdest=atwork_side3";
> URI uri = new URI(url, url.indexOf('%') != -1);
> method = new GetMethod();
> method.setURI(uri);
> method.setRequestHeader("HTTP_REFERER",
> "http://www.emigrant.ie/category.asp?iCategoryID=510");
> HttpMethodParams params = method.getParams();
> params.setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY);
> params.setParameter(HttpClientParams.USER_AGENT, "Mozilla/4.0"
> + "(compatible; MSIE 6.0; Windows NT 5.1;)");
> method.setFollowRedirects(true);
> }
>
> public void testResultcode() {
> try {
> client.executeMethod(method);
> Assert.assertEquals(HttpStatus.SC_OK, method.getStatusCode());
> } catch (HttpException e) {
> e.printStackTrace(System.out);
> } catch (IOException e) {
> e.printStackTrace(System.out);
> }
> }
>
> }
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org