You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Quintin Beukes <qu...@last.za.net> on 2008/07/02 11:59:00 UTC
The Cookie jar's lid seems stuck
Hey,
Due to our search engine for one of our sites being hosted on a
separate domain, we can't do AJAX requests to the original domain.
This isn't much of a problem, as I figured I'd just use HttpClient to
make a servlet which is a proxy to the Apache server.
So... to ensure session state is still maintained, which is a
requirement, I will just proxy the cookies the client sends me as
well.
I made the following method (the connection manager and parameters are
final static members of the class):
private HttpEntity proxyRequest(String baseUrl, String
getParameters, Cookie[] cookies) throws Exception
{
DefaultHttpClient httpClient = new DefaultHttpClient(connManager,
httpParams);
HttpContext context = new BasicHttpContext(httpClient.getDefaultContext());
HttpGet request = new HttpGet(baseUrl + basketUri);
CookieStore cookieStore = httpClient.getCookieStore();
for (Cookie c : cookies)
{
BasicClientCookie newCookie = new BasicClientCookie(c.getName(),
c.getValue());
newCookie.setDomain(c.getDomain());
newCookie.setPath(c.getPath());
newCookie.setComment(c.getComment());
if (c.getMaxAge() > 0)
{
long expiryTime = new Date().getTime() + c.getMaxAge();
newCookie.setExpiryDate(new Date(expiryTime));
}
cookieStore.addCookie(newCookie);
}
HttpResponse response = httpClient.execute(request, context);
HttpEntity entity = response.getEntity();
if (entity == null)
{
throw new NoHttpResponseException("Basket request failed to
return a response.");
}
return entity;
}
The Cookie[] I pass in, is an array of cookies are received by
HttpServletRequest.getCookies().
I stepped the code, and the addCookie() method is definitely being
called. When recieving the response on the other side, I make a dump
of all received cookies to the output. This output is received by
HttpClient and returned to the servlet's output stream.
Then what I'm getting is a dump of "0" cookies. So the request is
successful, but the cookies added to the store is not being sent to
the client.
Can anyone see what I'm doing wrong?
--
Quintin Beukes
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: The Cookie jar's lid seems stuck
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2008-07-02 at 11:59 +0200, Quintin Beukes wrote:
> Hey,
>
> Due to our search engine for one of our sites being hosted on a
> separate domain, we can't do AJAX requests to the original domain.
> This isn't much of a problem, as I figured I'd just use HttpClient to
> make a servlet which is a proxy to the Apache server.
>
> So... to ensure session state is still maintained, which is a
> requirement, I will just proxy the cookies the client sends me as
> well.
>
> I made the following method (the connection manager and parameters are
> final static members of the class):
>
> private HttpEntity proxyRequest(String baseUrl, String
> getParameters, Cookie[] cookies) throws Exception
> {
> DefaultHttpClient httpClient = new DefaultHttpClient(connManager,
> httpParams);
> HttpContext context = new BasicHttpContext(httpClient.getDefaultContext());
> HttpGet request = new HttpGet(baseUrl + basketUri);
> CookieStore cookieStore = httpClient.getCookieStore();
>
> for (Cookie c : cookies)
> {
> BasicClientCookie newCookie = new BasicClientCookie(c.getName(),
> c.getValue());
> newCookie.setDomain(c.getDomain());
> newCookie.setPath(c.getPath());
> newCookie.setComment(c.getComment());
> if (c.getMaxAge() > 0)
> {
> long expiryTime = new Date().getTime() + c.getMaxAge();
> newCookie.setExpiryDate(new Date(expiryTime));
> }
> cookieStore.addCookie(newCookie);
> }
>
> HttpResponse response = httpClient.execute(request, context);
> HttpEntity entity = response.getEntity();
>
> if (entity == null)
> {
> throw new NoHttpResponseException("Basket request failed to
> return a response.");
> }
>
> return entity;
> }
>
> The Cookie[] I pass in, is an array of cookies are received by
> HttpServletRequest.getCookies().
>
> I stepped the code, and the addCookie() method is definitely being
> called. When recieving the response on the other side, I make a dump
> of all received cookies to the output. This output is received by
> HttpClient and returned to the servlet's output stream.
>
> Then what I'm getting is a dump of "0" cookies. So the request is
> successful, but the cookies added to the store is not being sent to
> the client.
>
> Can anyone see what I'm doing wrong?
>
Quintin,
Adding cookies to the cookie store is just a part of the story. A cookie
need to match the origin server in order to be included in subsequent
HTTP requests. Apparently there is something wrong with the domain or
the expiry date.
Take a look at the RequestAddCookies protocol interceptor
Hope this helps
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: The Cookie jar's lid seems stuck
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2008-07-07 at 14:24 +0200, Quintin Beukes wrote:
> Some more strange behaviour.
>
> Sometimes when I step this code, it sends the second header even when
> pausing for only 1 second (literally only a breakpoint, refresh,
> alt+tab to eclipse, press continue button).
>
> I can't really place why it's running through a new request. It's
> definitely not my code, as placing a breakpoint at:
> HttpResponse response = httpClient.execute(request, context);
>
> Only pauses it once, even when pausing twice in the cookie method. My
> only idea is that it has something to do with the retries of
> connections. Though I discovered this problem with intermittent
> failures of my requests. Seems like it sometimes happens even with
> millisecond gaps in between.
>
> Any ideas?
>
Quintin,
What version of HttpClient are you using? Is it 4.0-alpha4 or a
snapshot. In any case make sure you have the latest SVN code
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: The Cookie jar's lid seems stuck
Posted by Quintin Beukes <qu...@last.za.net>.
Some more strange behaviour.
Sometimes when I step this code, it sends the second header even when
pausing for only 1 second (literally only a breakpoint, refresh,
alt+tab to eclipse, press continue button).
I can't really place why it's running through a new request. It's
definitely not my code, as placing a breakpoint at:
HttpResponse response = httpClient.execute(request, context);
Only pauses it once, even when pausing twice in the cookie method. My
only idea is that it has something to do with the retries of
connections. Though I discovered this problem with intermittent
failures of my requests. Seems like it sometimes happens even with
millisecond gaps in between.
Any ideas?
Quintin
On 7/7/08, Quintin Beukes <qu...@last.za.net> wrote:
> Hey,
>
> I found the problem I had, cookies received through the Servlet
> doesn't have a domain set (because the cookie request header doesn't
> send it iirc). Either way, I am sending an appropriate domain for each
> request now.
>
> But this caused me to run into a possible bug in both HttpClient and
> PHP (oops). Well, the HttpClient one can possible be justified, though
> I'm not really sure, so I'll check with you. The request/response I'm
> getting is this:
>
> GET /basket.php HTTP/1.1
> Host: junkmail.dev.junkmail.co.za
> Connection: Keep-Alive
> Cookie: JSESSIONID=391E339150BC34CE0BC73696914EE561;
> JMT_SID=1215430789544.7596784026398062592.391E339150BC34CE0BC73696914EE561;
> __utma=50980651.4049797362423985700.1215431510.1215431510.1215431510.1;
> __utmb=50980651.2.10.1215431510; __utmc=50980651;
> __utmz=50980651.1215431510.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);
> JMPHPSESSID=10580e39d1dd79e077340e93169330b6
> Cookie2: $Version=1
> Cookie: JSESSIONID=391E339150BC34CE0BC73696914EE561;
> JMT_SID=1215430789544.7596784026398062592.391E339150BC34CE0BC73696914EE561;
> __utma=50980651.4049797362423985700.1215431510.1215431510.1215431510.1;
> __utmb=50980651.2.10.1215431510; __utmc=50980651;
> __utmz=50980651.1215431510.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);
> JMPHPSESSID=10580e39d1dd79e077340e93169330b6
> Cookie2: $Version=1
>
> HTTP/1.1 200 OK
> Date: Mon, 07 Jul 2008 11:54:04 GMT
> Server: Apache
> X-Powered-By: PHP/5.2.4_p20070914-pl2-gentoo
> Set-Cookie: JMPHPSESSID=a1ae0f6136f6f99e7fcc7542dc517cbd; path=/;
> domain=junkmail.dev.junkmail.co.za
> Expires: Thu, 19 Nov 1981 08:52:00 GMT
> Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
> Pragma: no-cache
> Content-Length: 610
> Keep-Alive: timeout=15, max=100
> Connection: Keep-Alive
> Content-Type: text/html
>
> Cookies: Array
> (
> [JSESSIONID] => 391E339150BC34CE0BC73696914EE561
> [JMT_SID] =>
> 1215430789544.7596784026398062592.391E339150BC34CE0BC73696914EE561
> [__utma] => 50980651.4049797362423985700.1215431510.1215431510.1215431510.1
> [__utmb] => 50980651.2.10.1215431510
> [__utmc] => 50980651
> [__utmz] =>
> 50980651.1215431510.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
> [JMPHPSESSID] => 10580e39d1dd79e077340e93169330b6,
> JSESSIONID=391E339150BC34CE0BC73696914EE561
> )
>
> As you can see, it's sending the cookie header TWICE. And this is
> causing PHP to "misparse" the last cookie. I know the latter isn't
> your problem, but this can cause applications to misbehave when
> reached through HttpClient.
>
> When I was stepping the HttpClient code to find my bug, I noticed that
> HC runs into RequestAddCookies.process() twice. This is definitely the
> reason.
>
> On further investigation I also noticed that this only happens when I
> am stepping the code for more than 10 seconds. So is it perhaps a
> connection manager that gives it another try? The fact that it adds
> ONTO the first request's headers doesn't sound good though.
>
> Also, one more question. Does the RFC allow one to split 2 cookie
> headers with a COMMA+SPACE? Because it seems PHP concatenates both
> with a ", ", and then parses it from here.
>
> Quintin
>
>
> On 7/2/08, Quintin Beukes <qu...@last.za.net> wrote:
> > Hey,
> >
> > Due to our search engine for one of our sites being hosted on a
> > separate domain, we can't do AJAX requests to the original domain.
> > This isn't much of a problem, as I figured I'd just use HttpClient to
> > make a servlet which is a proxy to the Apache server.
> >
> > So... to ensure session state is still maintained, which is a
> > requirement, I will just proxy the cookies the client sends me as
> > well.
> >
> > I made the following method (the connection manager and parameters are
> > final static members of the class):
> >
> > private HttpEntity proxyRequest(String baseUrl, String
> > getParameters, Cookie[] cookies) throws Exception
> > {
> > DefaultHttpClient httpClient = new DefaultHttpClient(connManager,
> > httpParams);
> > HttpContext context = new BasicHttpContext(httpClient.getDefaultContext());
> > HttpGet request = new HttpGet(baseUrl + basketUri);
> > CookieStore cookieStore = httpClient.getCookieStore();
> >
> > for (Cookie c : cookies)
> > {
> > BasicClientCookie newCookie = new BasicClientCookie(c.getName(),
> > c.getValue());
> > newCookie.setDomain(c.getDomain());
> > newCookie.setPath(c.getPath());
> > newCookie.setComment(c.getComment());
> > if (c.getMaxAge() > 0)
> > {
> > long expiryTime = new Date().getTime() + c.getMaxAge();
> > newCookie.setExpiryDate(new Date(expiryTime));
> > }
> > cookieStore.addCookie(newCookie);
> > }
> >
> > HttpResponse response = httpClient.execute(request, context);
> > HttpEntity entity = response.getEntity();
> >
> > if (entity == null)
> > {
> > throw new NoHttpResponseException("Basket request failed to
> > return a response.");
> > }
> >
> > return entity;
> > }
> >
> > The Cookie[] I pass in, is an array of cookies are received by
> > HttpServletRequest.getCookies().
> >
> > I stepped the code, and the addCookie() method is definitely being
> > called. When recieving the response on the other side, I make a dump
> > of all received cookies to the output. This output is received by
> > HttpClient and returned to the servlet's output stream.
> >
> > Then what I'm getting is a dump of "0" cookies. So the request is
> > successful, but the cookies added to the store is not being sent to
> > the client.
> >
> > Can anyone see what I'm doing wrong?
> >
> > --
> >
> > Quintin Beukes
> >
>
>
>
> --
>
> Quintin Beukes
>
--
Quintin Beukes
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: The Cookie jar's lid seems stuck
Posted by Quintin Beukes <qu...@last.za.net>.
Hey,
I found the problem I had, cookies received through the Servlet
doesn't have a domain set (because the cookie request header doesn't
send it iirc). Either way, I am sending an appropriate domain for each
request now.
But this caused me to run into a possible bug in both HttpClient and
PHP (oops). Well, the HttpClient one can possible be justified, though
I'm not really sure, so I'll check with you. The request/response I'm
getting is this:
GET /basket.php HTTP/1.1
Host: junkmail.dev.junkmail.co.za
Connection: Keep-Alive
Cookie: JSESSIONID=391E339150BC34CE0BC73696914EE561;
JMT_SID=1215430789544.7596784026398062592.391E339150BC34CE0BC73696914EE561;
__utma=50980651.4049797362423985700.1215431510.1215431510.1215431510.1;
__utmb=50980651.2.10.1215431510; __utmc=50980651;
__utmz=50980651.1215431510.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);
JMPHPSESSID=10580e39d1dd79e077340e93169330b6
Cookie2: $Version=1
Cookie: JSESSIONID=391E339150BC34CE0BC73696914EE561;
JMT_SID=1215430789544.7596784026398062592.391E339150BC34CE0BC73696914EE561;
__utma=50980651.4049797362423985700.1215431510.1215431510.1215431510.1;
__utmb=50980651.2.10.1215431510; __utmc=50980651;
__utmz=50980651.1215431510.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);
JMPHPSESSID=10580e39d1dd79e077340e93169330b6
Cookie2: $Version=1
HTTP/1.1 200 OK
Date: Mon, 07 Jul 2008 11:54:04 GMT
Server: Apache
X-Powered-By: PHP/5.2.4_p20070914-pl2-gentoo
Set-Cookie: JMPHPSESSID=a1ae0f6136f6f99e7fcc7542dc517cbd; path=/;
domain=junkmail.dev.junkmail.co.za
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 610
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html
Cookies: Array
(
[JSESSIONID] => 391E339150BC34CE0BC73696914EE561
[JMT_SID] =>
1215430789544.7596784026398062592.391E339150BC34CE0BC73696914EE561
[__utma] => 50980651.4049797362423985700.1215431510.1215431510.1215431510.1
[__utmb] => 50980651.2.10.1215431510
[__utmc] => 50980651
[__utmz] =>
50980651.1215431510.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
[JMPHPSESSID] => 10580e39d1dd79e077340e93169330b6,
JSESSIONID=391E339150BC34CE0BC73696914EE561
)
As you can see, it's sending the cookie header TWICE. And this is
causing PHP to "misparse" the last cookie. I know the latter isn't
your problem, but this can cause applications to misbehave when
reached through HttpClient.
When I was stepping the HttpClient code to find my bug, I noticed that
HC runs into RequestAddCookies.process() twice. This is definitely the
reason.
On further investigation I also noticed that this only happens when I
am stepping the code for more than 10 seconds. So is it perhaps a
connection manager that gives it another try? The fact that it adds
ONTO the first request's headers doesn't sound good though.
Also, one more question. Does the RFC allow one to split 2 cookie
headers with a COMMA+SPACE? Because it seems PHP concatenates both
with a ", ", and then parses it from here.
Quintin
On 7/2/08, Quintin Beukes <qu...@last.za.net> wrote:
> Hey,
>
> Due to our search engine for one of our sites being hosted on a
> separate domain, we can't do AJAX requests to the original domain.
> This isn't much of a problem, as I figured I'd just use HttpClient to
> make a servlet which is a proxy to the Apache server.
>
> So... to ensure session state is still maintained, which is a
> requirement, I will just proxy the cookies the client sends me as
> well.
>
> I made the following method (the connection manager and parameters are
> final static members of the class):
>
> private HttpEntity proxyRequest(String baseUrl, String
> getParameters, Cookie[] cookies) throws Exception
> {
> DefaultHttpClient httpClient = new DefaultHttpClient(connManager,
> httpParams);
> HttpContext context = new BasicHttpContext(httpClient.getDefaultContext());
> HttpGet request = new HttpGet(baseUrl + basketUri);
> CookieStore cookieStore = httpClient.getCookieStore();
>
> for (Cookie c : cookies)
> {
> BasicClientCookie newCookie = new BasicClientCookie(c.getName(),
> c.getValue());
> newCookie.setDomain(c.getDomain());
> newCookie.setPath(c.getPath());
> newCookie.setComment(c.getComment());
> if (c.getMaxAge() > 0)
> {
> long expiryTime = new Date().getTime() + c.getMaxAge();
> newCookie.setExpiryDate(new Date(expiryTime));
> }
> cookieStore.addCookie(newCookie);
> }
>
> HttpResponse response = httpClient.execute(request, context);
> HttpEntity entity = response.getEntity();
>
> if (entity == null)
> {
> throw new NoHttpResponseException("Basket request failed to
> return a response.");
> }
>
> return entity;
> }
>
> The Cookie[] I pass in, is an array of cookies are received by
> HttpServletRequest.getCookies().
>
> I stepped the code, and the addCookie() method is definitely being
> called. When recieving the response on the other side, I make a dump
> of all received cookies to the output. This output is received by
> HttpClient and returned to the servlet's output stream.
>
> Then what I'm getting is a dump of "0" cookies. So the request is
> successful, but the cookies added to the store is not being sent to
> the client.
>
> Can anyone see what I'm doing wrong?
>
> --
>
> Quintin Beukes
>
--
Quintin Beukes
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org