You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "Karl Wright (JIRA)" <ji...@apache.org> on 2012/12/03 01:17:58 UTC
[jira] [Created] (HTTPCLIENT-1267) There seems to be no way to get
HttpClient to include cookies from a custom store
Karl Wright created HTTPCLIENT-1267:
---------------------------------------
Summary: There seems to be no way to get HttpClient to include cookies from a custom store
Key: HTTPCLIENT-1267
URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
Project: HttpComponents HttpClient
Issue Type: Bug
Components: HttpCookie
Affects Versions: 4.2.2
Reporter: Karl Wright
In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
{code}
DefaultHttpClient httpClient = new DefaultHttpClient();
...
CookieStore cs = new BasicCookieStore();
cs.addCookie(...); // for each saved cookie
httpClient.setCookieStore(cs);
HttpGet httpget = new HttpGet(...);
HttpResponse response = httpClient.execute(httpget);
Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
[save the cookies aside]
{code}
What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508760#comment-13508760 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Hi Oleg,
Thanks for the pointer. I'll have a crack at debugging when I'm home from work this evening. But there's only one cookie attribute: path, and offhand that looked like it should match...
Karl
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509712#comment-13509712 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Oleg,
At the moment we receive a Cookie instance from the CookieStore. We do not care at all what kind of cookie it is; all that we want to do is store it and later retrieve it to pass back to the server. All the rest of what happens is simply to try to accomodate httpclient.
If you have a way of saving the cookie in its original form and reconstituting a Cookie object from it that can be retransmitted to the server, that would be infinitely preferable to needing to figure out what kind of cookie it is etc. That was never the goal here.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509715#comment-13509715 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Hi Francois,
Yes, thanks, I'd already made a similar change about an hour ago, but that did not fix the problem by itself for me. I wound up creating a modified BasicCookieStore in the ThrottledFetcher class. It's checked into branches/CONNECTORS-120-1 now, so it is possible your change and that one are both in place.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509755#comment-13509755 ]
Oleg Kalnichevski commented on HTTPCLIENT-1267:
-----------------------------------------------
@Karl
HTTP cookie management is such a bloody mess because too many people do not care or did not care in the past. An HTTP cookie is logical concept that can have different representations depending on a particular context. Various cookie specs treat cookie differently depending on their version and properties. Standard java object serialization based on ObjectInputStream and ObjectOutputStream should work out of the box. Since your project implements a custom cookie manager and a custom cookie store which decomposes cookies down to individual bits and pieces and stores their state in a custom database, I do not understand why it is my personal fault or that of HttpClient if does not do it quite right. In order fix the problem the fetcher should either preserve the original type of the cookie or use cookie representation compatible with the cookie policy used by the fetcher.
Hope this helps
Oleg
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509295#comment-13509295 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Changing the code back to calling httpClient.setCookieStore() now allows fetches to proceed, with the original symptom. I have debugging code in the place that Oleg recommended, as follows:
>>>>>>
public boolean match(final Cookie cookie, final CookieOrigin origin) {
System.out.println("Matching cookie "+cookie);
if (cookie == null) {
throw new IllegalArgumentException("Cookie may not be null");
}
if (origin == null) {
throw new IllegalArgumentException("Cookie origin may not be null");
}
for (CookieAttributeHandler handler: getAttribHandlers()) {
if (!handler.match(cookie, origin)) {
System.out.println(" Cookie failed match! handle class: "+handler.getClass().getName());
return false;
}
}
return true;
}
<<<<<<
This debugging code is never called. I therefore have to conclude that the cookiestore bound to httpclient is somehow getting cleared at the beginning of the execute() method.
I'll pick this up later and see if I can find out where the clear is taking place. Since it is a BasicCookieStore instance, I should be able to get a stack trace whenever clear() is called.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509871#comment-13509871 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Given that we're working with 4.2.2 for the time being, it sounds like option 1 is the right thing to do at the moment. I'm buried right now but once I get some stuff off my plate I may be able to assist on HTTPCLIENT-1264, for the future.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509696#comment-13509696 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Also, it *does* appear that the date of 0 is not treated specially; even when I disable the discard (so that I do cookie.setDiscard(false)), it does not still add the cookie to the cookiestore. That is certainly not correct.
Can you let me know what you intend to do about this issue? It's perfectly possible for me to override BasicCookieStore if necessary, but this functionality is clearly problematic, seems to me.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508750#comment-13508750 ]
Oleg Kalnichevski commented on HTTPCLIENT-1267:
-----------------------------------------------
If can run your code under debugger if should be fairly easy to find which cookie attribute results in mismatch by looking at this method:
http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/impl/cookie/CookieSpecBase.html#122
Oleg
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Francois-Xavier Bonnet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509639#comment-13509639 ]
Francois-Xavier Bonnet commented on HTTPCLIENT-1267:
----------------------------------------------------
Oleg,
I think Expires is the header for the response, not the expiry of the cookie. Unless I miss something, Expires header on the response should have no impact on how cookies are handled.
I am using httpclient for a long time with applications that use transient cookies (typically java applications with JSESSIONID cookie) and it works fine. The problem may be somewhere else.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509625#comment-13509625 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Hi Francois,
I never had a copy of HttpClient 4.0-alpha1 here.
I built all of this using Maven as per the instructions, from a fresh checkout of (in this case) httpcore. Here is the class you say is not supposed to be present:
>>>>>>
C:\wip\httpcore\trunk\httpcore\src\main\java\org\apache\http\protocol>dir
Volume in drive C is OS
Volume Serial Number is 22F7-8854
Directory of C:\wip\httpcore\trunk\httpcore\src\main\java\org\apache\http\proto
col
12/03/2012 05:24 PM <DIR> .
12/03/2012 05:24 PM <DIR> ..
12/03/2012 05:24 PM 3,091 BasicHttpContext.java
12/03/2012 05:24 PM 8,723 BasicHttpProcessor.java
12/03/2012 05:24 PM 3,533 ChainBuilder.java
12/03/2012 05:24 PM 2,852 DefaultedHttpContext.java
12/03/2012 05:24 PM 2,763 ExecutionContext.java
12/03/2012 05:24 PM 4,469 HTTP.java
12/03/2012 05:24 PM 2,842 HttpContext.java
12/03/2012 05:24 PM 3,933 HttpCoreContext.java
12/03/2012 05:24 PM 2,650 HttpDateGenerator.java
12/03/2012 05:24 PM 3,344 HttpExpectationVerifier.java
12/03/2012 05:24 PM 2,375 HttpProcessor.java
12/03/2012 05:24 PM 4,777 HttpProcessorBuilder.java
12/03/2012 05:24 PM 12,453 HttpRequestExecutor.java
12/03/2012 05:24 PM 2,456 HttpRequestHandler.java
12/03/2012 05:24 PM 1,863 HttpRequestHandlerMapper.java
12/03/2012 05:24 PM 3,657 HttpRequestHandlerRegistry.java
12/03/2012 05:24 PM 1,910 HttpRequestHandlerResolver.java
12/03/2012 05:24 PM 3,465 HttpRequestInterceptorList.java
12/03/2012 05:24 PM 3,490 HttpResponseInterceptorList.java
12/03/2012 05:24 PM 17,893 HttpService.java
12/03/2012 05:24 PM 5,726 ImmutableHttpProcessor.java
12/03/2012 05:24 PM 1,381 package.html
12/03/2012 05:24 PM 2,508 RequestConnControl.java
12/03/2012 05:24 PM 5,376 RequestContent.java
12/03/2012 05:24 PM 2,412 RequestDate.java
12/03/2012 05:24 PM 3,208 RequestExpectContinue.java
12/03/2012 05:24 PM 3,756 RequestTargetHost.java
12/03/2012 05:24 PM 2,729 RequestUserAgent.java
12/03/2012 05:24 PM 4,431 ResponseConnControl.java
12/03/2012 05:24 PM 5,647 ResponseContent.java
12/03/2012 05:24 PM 2,446 ResponseDate.java
12/03/2012 05:24 PM 2,411 ResponseServer.java
12/03/2012 05:24 PM 2,242 SyncBasicHttpContext.java
12/03/2012 05:24 PM 4,046 UriHttpRequestHandlerMapper.java
12/03/2012 05:24 PM 5,205 UriPatternMatcher.java
35 File(s) 146,063 bytes
2 Dir(s) 57,939,709,952 bytes free
C:\wip\httpcore\trunk\httpcore\src\main\java\org\apache\http\protocol>
<<<<<<
Karl
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508667#comment-13508667 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
The easiest way to reproduce this is the following:
(0) Make sure you have ant >=1.7 installed, and have your JAVA_HOME set to point at a jdk 1.6 or above.
(1) Check out https://svn.apache.org/repos/asf/manifoldcf/branches/CONNECTORS-120-1.
(2) Run the following:
ant make-core-deps
This will download all necessary dependencies for the test, including httpcomponents core and httpclient.
(3) To exercise the test itself, which fires up a mock session-protected web site:
ant run-webcrawler-tests-derby
In its broken state it will run until the test times out, generating reams of logging output into the file:
tests/webcrawler/test-output-derby/manifoldcf.log
^C out of the test to stop it before it fills your disk. If you want to inspect the pertinent code itself, you can find it all localized in:
connectors/webcrawler/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/webcrawler/ThrottledFetcher.java
All functional access to httpcomponents is localized there, so luckily there is no other place to look.
The httpcomponents libraries, if you should want to replace them, are in the "lib" directory, as "httpcore.jar" and "httpclient.jar". Rerun after a failed test by:
ant clean run-webcrawler-tests-derby
Thanks,
Karl
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509704#comment-13509704 ]
Oleg Kalnichevski commented on HTTPCLIENT-1267:
-----------------------------------------------
Karl
Why are using BasicClientCookie2 class (a RFC 2965 compliant Set-Cookie2) to represent something which is clearly a non-standard Netscape style cookie? The discard attribute is applicable for RFC 2965 compliant cookies only.
Oleg
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509796#comment-13509796 ]
Oleg Kalnichevski commented on HTTPCLIENT-1267:
-----------------------------------------------
Karl
There are several options varying in terms of complexity, standard compliance and development efforts:
(1) You can change your code to use BasicClientCookie instead of BasicClientCookie2 and be done with it. The fetcher cant make proper use of RFC 2965 style cookies anyway given it is using a customized BROWSER_COMPATIBILITY spec.
(2) Further customize the cookie spec implementation used by the fetcher to produce Cookie instances backed JAXB or JSON objects (or some other technology). This should relieve you from having to rely upon binary object serialization. You would need to override #parse(HeaderElement[], CookieOrigin) protected method.
(3) Most difficult but 'cleanest' approach would be to resolve HTTPCLIENT-1264, replace the custom cookie spec with the BEST_MATCH from HttpClient and change HttpClient in the 4.3 branch to use an abstract factory to instantiate Cookie objects.
Hope this helps
Oleg
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Francois-Xavier Bonnet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509713#comment-13509713 ]
Francois-Xavier Bonnet commented on HTTPCLIENT-1267:
----------------------------------------------------
I just made a test where I replaced in method org.apache.manifoldcf.crawler.connectors.webcrawler.CookieManager.updateCookies(String, LoginCookies) the following line:
map.put(discardField,booleanToString(!c.isPersistent()));
by
map.put(discardField,booleanToString(false));
And now the cookie is sent correctly:
DEBUG 2012-12-04 13:59:38,689 (Worker thread '0') - WEB: Adding 1 cookies for '/web/loginpage.html'
DEBUG 2012-12-04 13:59:38,689 (Worker thread '0') - WEB: Cookie '[version: 0][name: JSESSIONID][value: 1fn14sizhjvg47bp9359v4ky2][domain: localhost][path: /web][expiry: null]' added
DEBUG 2012-12-04 13:59:38,690 (Thread-375) - >> GET /web/loginpage.html?user=foo&password=bar HTTP/1.1
DEBUG 2012-12-04 13:59:38,690 (Thread-375) - >> User-Agent: Mozilla/5.0 (ApacheManifoldCFWebCrawler; someone@somewhere.com)
DEBUG 2012-12-04 13:59:38,690 (Thread-375) - >> From: someone@somewhere.com
DEBUG 2012-12-04 13:59:38,690 (Thread-375) - >> Host: localhost:8191
DEBUG 2012-12-04 13:59:38,690 (Thread-375) - >> Connection: Keep-Alive
DEBUG 2012-12-04 13:59:38,690 (Thread-375) - >> Cookie: JSESSIONID=1fn14sizhjvg47bp9359v4ky2
DEBUG 2012-12-04 13:59:38,695 (Thread-375) - << HTTP/1.1 302 Found
DEBUG 2012-12-04 13:59:38,695 (Thread-375) - << Location: http://localhost:8191/web/index.html
DEBUG 2012-12-04 13:59:38,695 (Thread-375) - << Content-Length: 0
DEBUG 2012-12-04 13:59:38,695 (Thread-375) - << Server: Jetty(7.5.4.v20111024)
DEBUG 2012-12-04 13:59:38,695 (Worker thread '0') - WEB: Retrieving cookies...
DEBUG 2012-12-04 13:59:38,698 (Worker thread '0') - WEB: Cookie '[version: 0][name: JSESSIONID][value: 1fn14sizhjvg47bp9359v4ky2][domain: localhost][path: /web][expiry: null]'
INFO 2012-12-04 13:59:38,701 (Worker thread '0') - WEB: FETCH LOGIN|http://localhost:8191/web/loginpage.html|1354625978688+12|302|0|
DEBUG 2012-12-04 13:59:38,702 (Worker thread '0') - WEB: Tried to match raw url 'http://localhost:8191/web/index.html'
DEBUG 2012-12-04 13:59:38,702 (Worker thread '0') - WEB: Tried to match cooked url 'http://localhost:8191/web/index.html'
DEBUG 2012-12-04 13:59:38,702 (Worker thread '0') - WEB: Redirection link lookup matched 'http://localhost:8191/web/index.html'
DEBUG 2012-12-04 13:59:38,702 (Worker thread '0') - WEB: Document 'http://localhost:8191/web/loginpage.html' matches preferred redirection, so determined to be login page for sequence '/web/'
DEBUG 2012-12-04 13:59:38,704 (Worker thread '0') - WEB: Waiting to start getting a connection to http://127.0.0.1:8191
DEBUG 2012-12-04 13:59:38,704 (Worker thread '0') - WEB: Attempting to get connection to http://127.0.0.1:8191 (0 ms)
DEBUG 2012-12-04 13:59:38,704 (Worker thread '0') - WEB: Successfully got connection to http://127.0.0.1:8191 (0 ms)
DEBUG 2012-12-04 13:59:38,704 (Worker thread '0') - WEB: Waiting for an HttpClient object
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509634#comment-13509634 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
A zero value means a transient cookie, no? Does httpcomponents not support transient cookies? If so we cannot use this library for session login, since most session-protected sites use this. Perhaps the problem is that httpcomponents does not understand the special semantics of transient cookies.
FWIW, the cookie is being generated by Jetty, automatically, for tracking session variables. Nothing special was done at all. It operates fine within a browser context.
May I ask how expired cookies are supposedly cleared? If you are calling the cookie store's expire method, then that is not what is happening; I instrumented that and it never gets called during this test.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509630#comment-13509630 ]
Oleg Kalnichevski commented on HTTPCLIENT-1267:
-----------------------------------------------
Karl
I fixed the problem in SVN trunk and was causing the class cast exception. Both core and client trunks are in the early ALPHA development stage and are unstable. Please refrain from using them unless you are willing to port your code to the new APIs. Besides, there have been no changes to the cookie processing logic in the 4.3 code (and no plans to make any changes there).
Please also note that cookies can get cleaned up automatically when expired. Given the stuff the server is sending back it is quite likely to be the case
DEBUG 2012-12-02 19:28:05,499 (Thread-434) - << Expires: Thu, 01 Jan 1970 00:00:00 GMT
Oleg
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Comment Edited] (HTTPCLIENT-1267) There seems to be no way
to get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508667#comment-13508667 ]
Karl Wright edited comment on HTTPCLIENT-1267 at 12/3/12 11:55 AM:
-------------------------------------------------------------------
The easiest way to reproduce this is the following:
(0) Make sure you have ant >=1.7 installed, svn 1.7, and have your JAVA_HOME set to point at a jdk 1.6 or above.
(1) Check out https://svn.apache.org/repos/asf/manifoldcf/branches/CONNECTORS-120-1.
(2) Run the following:
ant make-core-deps
This will download all necessary dependencies for the test, including httpcomponents core and httpclient.
(3) To exercise the test itself, which fires up a mock session-protected web site:
ant run-webcrawler-tests-derby
In its broken state it will run until the test times out, generating reams of logging output into the file:
tests/webcrawler/test-output-derby/manifoldcf.log
^C out of the test to stop it before it fills your disk. If you want to inspect the pertinent code itself, you can find it all localized in:
connectors/webcrawler/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/webcrawler/ThrottledFetcher.java
All functional access to httpcomponents is localized there, so luckily there is no other place to look.
The httpcomponents libraries, if you should want to replace them, are in the "lib" directory, as "httpcore.jar" and "httpclient.jar". Rerun after a failed test by:
ant clean run-webcrawler-tests-derby
Thanks,
Karl
was (Author: kwright@metacarta.com):
The easiest way to reproduce this is the following:
(0) Make sure you have ant >=1.7 installed, and have your JAVA_HOME set to point at a jdk 1.6 or above.
(1) Check out https://svn.apache.org/repos/asf/manifoldcf/branches/CONNECTORS-120-1.
(2) Run the following:
ant make-core-deps
This will download all necessary dependencies for the test, including httpcomponents core and httpclient.
(3) To exercise the test itself, which fires up a mock session-protected web site:
ant run-webcrawler-tests-derby
In its broken state it will run until the test times out, generating reams of logging output into the file:
tests/webcrawler/test-output-derby/manifoldcf.log
^C out of the test to stop it before it fills your disk. If you want to inspect the pertinent code itself, you can find it all localized in:
connectors/webcrawler/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/webcrawler/ThrottledFetcher.java
All functional access to httpcomponents is localized there, so luckily there is no other place to look.
The httpcomponents libraries, if you should want to replace them, are in the "lib" directory, as "httpcore.jar" and "httpclient.jar". Rerun after a failed test by:
ant clean run-webcrawler-tests-derby
Thanks,
Karl
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509274#comment-13509274 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
I built both httpcore.jar and httpclient.jar from their respective trunk areas, and put them into place in ManifoldCF. Now I'm getting the following exception, with the same code that runs without outputting any cookies:
>>>>>>
java.lang.ClassCastException
at java.lang.Class.cast(Class.java:2990)
at org.apache.http.protocol.HttpCoreContext.getAttribute(HttpCoreContext.java:89)
at org.apache.http.client.protocol.HttpClientContext.getLookup(HttpClientContext.java:99)
at org.apache.http.client.protocol.HttpClientContext.getCookieSpecRegistry(HttpClientContext.java:103)
at org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:98)
at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:131)
at org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:166)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:515)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:857)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:70)
at org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ExecuteMethodThread.run(ThrottledFetcher.java:2436)
<<<<<<
It seems like it is clearly looking for the cookie policy and finding something it doesn't like. Hard to see how I could have caused this either, since my code only sets the cookie store in the context:
>>>>>>
HttpContext context = new BasicHttpContext();
context.setAttribute(ClientContext.COOKIE_STORE,cookieStore);
response = httpClient.execute(executeMethod, context);
<<<<<<
There are too many changes between 4.2.2 and trunk for me to get sensible results; I think I'm going to work off the 4.2.x branch in hopes of getting something more stable.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509289#comment-13509289 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Ok, 4.2.x fails in a manner similar to trunk:
>>>>>>
java.lang.ClassCastException
at java.lang.Class.cast(Class.java:2990)
at org.apache.http.protocol.HttpCoreContext.getAttribute(HttpCoreContext.java:89)
at org.apache.http.client.protocol.HttpClientContext.getLookup(HttpClientContext.java:99)
at org.apache.http.client.protocol.HttpClientContext.getCookieSpecRegistry(HttpClientContext.java:103)
at org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:98)
at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:131)
at org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:166)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:515)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:857)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:70)
at org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottledFetcher$ExecuteMethodThread.run(ThrottledFetcher.java:2436)
<<<<<<
Given that, I'm going to change ThrottledFetcher back to not use the HttpContext implementation, and instead use HttpClient.setCookieStore().
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509724#comment-13509724 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
> But you should, because cookie representations differ.
I don't understand why this should matter. All we want to do is return the cookie to the server.
If there is no choice but to pick apart the cookie and reassemble it, I would like to know of a reference for how to properly do that under httpcomponents. In other words:
(a) Given an existing Cookie object, what Cookie class should I construct to represent the cookie?
OR
(b) How do you already serialize and deserialize cookie objects when you read them out of headers or put them back into headers? There would seem to be no need to invent other serialization mechanisms than that for a cookie.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509689#comment-13509689 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Hmm, so there is still a problem, I think. If a cookie has the "discard" attribute set, it cannot be added to a cookie store, correct? Effectively that means I cannot retransmit cookies of this kind to the server, unless I change them to not have the "discard" attribute. Is this consistent with the meaning of the "discard" attribute?
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509683#comment-13509683 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Hi Francois,
Thank you for letting me know where the cookie loss is taking place.
The next question is: how is this supposed to be done? The cookie being restored from the database is identical (according to Cookie.toString()) to the cookie I saved in the database, so there seems to be no problem there. Should the cookie.isExpired() method treat transient cookies (those with date zero) specially? I think it should.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509319#comment-13509319 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
The only BasicCookieStore creation is being done by the ManifoldCF code. The store is never cleared. When httpClient.getCookieStore() is called, it returns a class of type BasicCookieStore also.
So there must be a logic flaw that prevents the cookie store bound to the httpclient instance from being used to generate request cookies. Any pointers as to where that logic resides?
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509284#comment-13509284 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
The httpclient/branches/4.2.x branch fails tests:
Tests in error:
testStatefulConnections(org.apache.http.impl.client.TestStatefulConnManagement): Connect to localhost:56070 timed out
I'll skip tests in hopes that this is not an important problem...
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Francois-Xavier Bonnet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509592#comment-13509592 ]
Francois-Xavier Bonnet commented on HTTPCLIENT-1267:
----------------------------------------------------
Karl, in your stack trace you can see a class named org.apache.http.protocol.HttpCoreContext
This class exists only in HttpClient 4.0-alpha1
Maybe the cause of your problem is that you have an old version of HttpClient somewhere in your classpath.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Francois-Xavier Bonnet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509722#comment-13509722 ]
Francois-Xavier Bonnet commented on HTTPCLIENT-1267:
----------------------------------------------------
Oleg,
Don't you think it would be a good idea to improve the toString() method of classes BasicClientCookie and BasicClientCookie2 to display all the information they contain ? This could make debugging easier.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Francois-Xavier Bonnet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509708#comment-13509708 ]
Francois-Xavier Bonnet commented on HTTPCLIENT-1267:
----------------------------------------------------
The restored cookie is not identical. It looks like it is just because the toString() method returns the same result but it is not. If you look at the toString() methode in BasicClientCookie it does not show all the attributes (comment, secure...) and BasicClientCookie2 just inherits toString() method from BasicClientCookie so you don't see its specific attributes.
Discard attribute comes from Set-Cookie2 headers where the rfc2965 says the server can use this attribute to destroy a cookie on the client so you should not set it to true.
Discard attribute has nothing to do with session cookies (persistent=false)
In my opinion you don't need to override BasicCookieStore nor BrowserCompatSpec.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509329#comment-13509329 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
FWIW, I was able to get the 4.2.x HttpContext approach to not exception, anyhow - looks like there was a build cleanliness issue. But it still didn't behave any differently from the HttpClient-bound cookiestore.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509703#comment-13509703 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
I created my own cookie store, without the restriction on adding "expired" cookies, and now everything works fine here.
I still think perhaps BasicCookieStore's logic for rejecting cookies is incorrect, since the notion of what a "session" is really depends not on httpclient library but rather on the user of httpclient library.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508417#comment-13508417 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Tried the HttpContext approach; no difference. Here's a chunk of ManifoldCF log output showing header logging for HttpClient:
{code}
DEBUG 2012-12-02 19:28:05,499 (Worker thread '1') - WEB: Adding 1 cookies for '/web/loginpage.html'
DEBUG 2012-12-02 19:28:05,499 (Worker thread '1') - WEB: Cookie '[version: 0][name: JSESSIONID][value: zlammg91w3dnsuc1j9ga3lln][domain: localhost][path: /web][expiry: null]' added
DEBUG 2012-12-02 19:28:05,499 (Thread-434) - >> GET /web/loginpage.html?user=foo&password=bar HTTP/1.1
DEBUG 2012-12-02 19:28:05,499 (Thread-434) - >> User-Agent: Mozilla/5.0 (ApacheManifoldCFWebCrawler; someone@somewhere.com)
DEBUG 2012-12-02 19:28:05,499 (Thread-434) - >> From: someone@somewhere.com
DEBUG 2012-12-02 19:28:05,499 (Thread-434) - >> Host: localhost:8191
DEBUG 2012-12-02 19:28:05,499 (Thread-434) - >> Connection: Keep-Alive
DEBUG 2012-12-02 19:28:05,499 (Thread-434) - << HTTP/1.1 302 Found
DEBUG 2012-12-02 19:28:05,499 (Thread-434) - << Set-Cookie: JSESSIONID=9qxi0v4tn2fi1lcaw5304be7b;Path=/web
DEBUG 2012-12-02 19:28:05,499 (Thread-434) - << Expires: Thu, 01 Jan 1970 00:00:00 GMT
DEBUG 2012-12-02 19:28:05,499 (Thread-434) - << Location: http://localhost:8191/web/index.html
DEBUG 2012-12-02 19:28:05,499 (Thread-434) - << Content-Length: 0
DEBUG 2012-12-02 19:28:05,499 (Thread-434) - << Server: Jetty(7.5.4.v20111024)
DEBUG 2012-12-02 19:28:05,499 (Worker thread '1') - WEB: Retrieving cookies...
DEBUG 2012-12-02 19:28:05,499 (Worker thread '1') - WEB: Cookie '[version: 0][name: JSESSIONID][value: 9qxi0v4tn2fi1lcaw5304be7b][domain: localhost][path: /web][expiry: null]'
{code}
As you can see, the cookie gets added to the cookie store, but is then ignored during the Http GET request that follows.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509766#comment-13509766 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Hi Oleg,
I am sorry if you got the impression that I was blaming you in any way for the way cookies work in general. All I'm trying to do is explain what we'd hoped to do in our project, and how we have done it in the past. How we do it in the future is open to negotiation, although it would be good I think to make sure we understand one another first. So let's first go into where I think things stand.
I understand that there are lots of different forms of cookies. I understand also that in order to pull information out of the cookie one needs to know what form it is. I also know that the allowable forms are determined by the kind of header used to set the cookie, e.g. Set-Cookie or Set-Cookie2. However, as far as ManifoldCF is concerned, none of that is actually important. All that ManifoldCF wants to do is record the cookie (in its original header form), and retransmit it later (using the right header and in its original form).
Unfortunately, commons-httpclient did not permit this, because there was no way to get the header form back out of a cookie. So we created a database schema that included all the fields that commons-httpclient seemed to support, and picked the cookie apart and put it back together for transmission to the server. However, I stress again that all of that is unnecessary from ManifoldCF's perspective; all we ever wanted to do was save the cookie in its unaltered header form in the database.
For httpcomponents httpclient, there are two options for ManifoldCF. One option is to keep trying to do what we have done in the past, and break the cookie up. If that approach is to work, we need to understand how to properly disassemble the cookie using httpcomponents httpclient constructs, and reassemble it. Currently I clearly don't understand how to do that reliably. The second option is to figure out how to get the header form of the cookie, which is what we wanted to do in the first place, and also how to restore a Cookie object from its header form. It sounds, however, like that may still not be an option in httpcomponents at this time. Another option, which you've suggested, is to use Java serialization - however, in my experience, serialized objects are not a good thing to be putting into a persistent database, because new versions of software will come along and make those objects not deserialize in a reasonable manner. So I don't think we can use that approach.
I am open to your suggestions as to how best to proceed.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509738#comment-13509738 ]
Oleg Kalnichevski commented on HTTPCLIENT-1267:
-----------------------------------------------
@Francois-Xavier
Absolutely. Any reasonable contribution to to the project will be incorporated one way or another. Feel free to raise a new JIRA and attach a patch to it.
Oleg
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-1267) There seems to be no way to get
HttpClient to include cookies from a custom store
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oleg Kalnichevski resolved HTTPCLIENT-1267.
-------------------------------------------
Resolution: Invalid
> We do not care at all what kind of cookie it is;
But you should, because cookie representations differ.
> If you have a way of saving the cookie in its original form and reconstituting a Cookie object from it that can be retransmitted to the server, that would be infinitely preferable to needing to figure out what kind of cookie it is etc. That was never the goal here.
Java serialization is supported out of the box. We could support another serialization mechanisms as long as they do not require additional dependencies.
Oleg
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Francois-Xavier Bonnet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508584#comment-13508584 ]
Francois-Xavier Bonnet commented on HTTPCLIENT-1267:
----------------------------------------------------
Hi Karl,
Could you attach the complete code to reproduce the problem ?
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Francois-Xavier Bonnet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509675#comment-13509675 ]
Francois-Xavier Bonnet commented on HTTPCLIENT-1267:
----------------------------------------------------
Karl,
I debugged your code and I think I found the problem. When you retrieve the cookies from the first request, HttpClient gives you a BasicClientCookie, then you store it in the database but for the field "discard" you do this:
map.put(discardField,booleanToString(!c.isPersistent()));
As your cookie is not persistent, you get discard=true
Then when you restore your cookie from database, you create a BasicClientCookie2 and if you look at the code of BasicCookieStore:
public synchronized void addCookie(Cookie cookie) {
if (cookie != null) {
// first remove any old cookie that is equivalent
cookies.remove(cookie);
if (!cookie.isExpired(new Date())) {
cookies.add(cookie);
}
}
}
And then the code of BasicClientCookie2:
public boolean isExpired(final Date date) {
return this.discard || super.isExpired(date);
}
So you cookie is not even added to the store as it is considered as expired.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509646#comment-13509646 ]
Oleg Kalnichevski commented on HTTPCLIENT-1267:
-----------------------------------------------
It does. In fact, per default HttpClient retains all cookies for the duration of the HTTP session only unless cookies are carried over to another session programmatically.
See RequestAddCookies protocol interceptor for details as to how expired are handled. Expired cookies do not physically removed from the store. That is the responsibility of the CookieStore implementation. They just never get passed onto the CookieSpec.
http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/client/protocol/RequestAddCookies.html#181
Oleg
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Comment Edited] (HTTPCLIENT-1267) There seems to be no way
to get HttpClient to include cookies from a custom store
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509646#comment-13509646 ]
Oleg Kalnichevski edited comment on HTTPCLIENT-1267 at 12/4/12 10:29 AM:
-------------------------------------------------------------------------
It does. In fact, per default HttpClient retains all cookies for the duration of the HTTP session only unless cookies are carried over to another session programmatically.
See RequestAddCookies protocol interceptor for details as to how expired are handled. Expired cookies do not get physically removed from the store. That is the responsibility of the CookieStore implementation. They just never get passed onto the CookieSpec.
http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/client/protocol/RequestAddCookies.html#181
Oleg
was (Author: olegk):
It does. In fact, per default HttpClient retains all cookies for the duration of the HTTP session only unless cookies are carried over to another session programmatically.
See RequestAddCookies protocol interceptor for details as to how expired are handled. Expired cookies do not physically removed from the store. That is the responsibility of the CookieStore implementation. They just never get passed onto the CookieSpec.
http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/client/protocol/RequestAddCookies.html#181
Oleg
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1267) There seems to be no way to
get HttpClient to include cookies from a custom store
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509685#comment-13509685 ]
Karl Wright commented on HTTPCLIENT-1267:
-----------------------------------------
Hi Francois,
Never mind my previous comment; the date is not the issue, as you pointed out - it's the discard field. I'll modify code accordingly and try it.
> There seems to be no way to get HttpClient to include cookies from a custom store
> ---------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1267
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1267
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCookie
> Affects Versions: 4.2.2
> Reporter: Karl Wright
>
> In the port of ManifoldCF from commons-httpclient to httpcomponents, we discovered a problem or maybe a design flaw with how cookies are handled.
> What we're trying to do is to follow a certain series of html pages, and record the cookies after that is done, and use exactly those cookies for a subsequent series of pages. So I have code that looks something like this:
> {code}
> DefaultHttpClient httpClient = new DefaultHttpClient();
> ...
> CookieStore cs = new BasicCookieStore();
> cs.addCookie(...); // for each saved cookie
> httpClient.setCookieStore(cs);
> HttpGet httpget = new HttpGet(...);
> HttpResponse response = httpClient.execute(httpget);
> Cookie[] cookiesToSave = httpClient.getCookieStore().getCookies();
> [save the cookies aside]
> {code}
> What happens when I do this against a page that sets a cookie, is I get a perfectly reasonable cookie, and save it. For the next page, the code to add the cookie to the custom store then seems to work fine as well. But when I watch what is transmitted for the request (via header logging) I see no Cookie header being sent, and indeed the cookie seems to be lost.
> Similar logic was done with HttpState in commons-httpclient, and worked perfectly there against the same website.
> I have not yet tried the HttpContext approach for implementing this - I may try that while awaiting an answer to this ticket, but the above seems like it ought to work, according to the documentation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org