You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Adam Patacchiola <ad...@2fours.com> on 2013/04/11 18:13:28 UTC
4.2.3 gzip caching still broken?
I'm using 4.2.3 with gzip compression and CachingHttpClient. Initially I
implemented the custom request/response interceptors as described here:
https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientGZipContentCompression.java
which
did not work, resulting in the issue described here:
https://issues.apache.org/jira/browse/HTTPCLIENT-1163.
It appeared to me from reading this issue that using the
"CompressionDecorator" would resolve the issue so I modified my code to use
DecompressingHttpClient (
https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/DecompressingHttpClient.html)
but the issue still persists as we can see from the below log output. It is
caching using one (broken?) key but then looking it up using a different
(correct?) key which is consistent with the bug above:
04-11 09:32:54.760: ... putting cache entry, url: {Accept-Encoding=}
https://www.surespot.me:8080/images/b:f1/165
04-11 09:32:55.965: ... Cache miss [host: https://www.surespot.me:8080;
uri: https://www.surespot.me:8080/images/b:f1/165]
Am I missing something or is this still broken?
Thanks,
Adam
Re: 4.2.3 gzip caching still broken?
Posted by Adam Patacchiola <ad...@2fours.com>.
This is still broken in 4.2.5. Created an issue with attached test:
https://issues.apache.org/jira/browse/HTTPCLIENT-1347
Thanks,
Adam
On Mon, Apr 15, 2013 at 4:36 AM, Oleg Kalnichevski <ol...@apache.org> wrote:
> On Sat, 2013-04-13 at 08:31 -0600, Adam Patacchiola wrote:
> > Jon and Oleg,
> >
> > Thanks for your responses. Caching both the variant and non variant URLs
> > makes sense, thanks for the explanation, but I'm still not sure it's
> > functioning correctly. In BasicHttpCache I don't see how it would cache
> > both URLs without my faulty updateEntry behavior. If I were checking the
> > key set by the parentUri in updateEntry, this would return null because
> > putEntry is never called with the parentUri explicitly.
> >
> > Unless you are saying updateEntry is supposed to update non-existent
> URLs?
> >
> > Oleg, the storeVariantEntry code in BasicHttpCache looks the same as
> 4.2.3
> > here:
> >
> https://github.com/apache/httpclient/blob/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/BasicHttpCache.java
> > so
> > I'm not sure how it could be fixed?
> >
> > Thanks,
> >
> > Adam
> >
>
> Adam,
>
> I was more referring to refactoring of the way the caching aspect gets
> wired into the HTTP request processing pipeline. If you think there is
> something wrong with the BasicHttpCache#storeVariantEntry Jonathan is
> the expert.
>
> Anyway, feel free to raise a JIRA for this issue to make sure it does
> not fall between the cracks. A test case demonstrating (reproducing) the
> issue would help a great deal to ensure it gets resolved sooner.
>
> Oleg
>
> >
> > On Thu, Apr 11, 2013 at 12:59 PM, Moore, Jonathan (CIM) <
> > Jonathan_Moore@comcast.com> wrote:
> >
> > > Hi Adam,
> > >
> > > This is related to the way the caching module handles cache variants.
> In
> > > this case I suspect the origin is (correctly) setting Vary:
> Accept-Encoding.
> > >
> > > There are two cache entries here; however only one of them should have
> the
> > > response body present, IIRC. The version without the prepended request
> > > headers is treated as the "parent" entry and the one with the headers
> is
> > > the actual cached variant.
> > >
> > > This structure is in place because certain requests that pass through
> the
> > > cache are required to invalidate all the variants for that URL, so we
> need
> > > someplace to tie those together. It doubles the header space used but
> not
> > > double the response body space. As there are more variants the
> overhead for
> > > the duplicated headers drops further.
> > >
> > > Now, because you are seeing {Accept-Encoding=} this indicates that when
> > > the cache saw the request come through it did not have the
> Accept-Encoding:
> > > gzip on it at that point. I think this means you have the
> > > DecompressingHttpClient and CachingHttpClient wired up backwards.
> > >
> > > You want the CachingHttpClient as close to the final DefaultHttpClient
> as
> > > possible. So these should be layered as:
> > >
> > > DecompressingHttpClient -> CachingHttpClient -> DefaultHttpClient
> > >
> > > One of the updates in the 4.3 release will take more care of this
> wiring
> > > for you out of the box.
> > >
> > > Jon
> > > ........
> > > Jon Moore
> > >
> > >
> > > On Apr 11, 2013, at 2:20 PM, "Adam Patacchiola" <ad...@2fours.com>
> wrote:
> > >
> > > > I have done some more research on this and it appears that the
> caching is
> > > > working, however it is adding 2 entries to the backing cache: one
> each
> > > with
> > > > and without the url pre-pended by {Accept-Encoding=}. This results
> in a
> > > > cache miss for the get with the pre-pended url, and uses double the
> > > storage
> > > > space in whatever mechanism you are backing the client with. There
> was a
> > > > bug in my backing store which led to me initially believing it was
> not
> > > > caching the (correct) url.
> > > >
> > > > tl;dr it is caching but adding a duplicate invalid entry that never
> gets
> > > > hit.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Apr 11, 2013 at 10:13 AM, Adam Patacchiola <ad...@2fours.com>
> > > wrote:
> > > >
> > > >> I'm using 4.2.3 with gzip compression and CachingHttpClient.
> Initially I
> > > >> implemented the custom request/response interceptors as described
> here:
> > > >>
> > >
> https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientGZipContentCompression.javawhich
> > > >> did not work, resulting in the issue described here:
> > > >> https://issues.apache.org/jira/browse/HTTPCLIENT-1163.
> > > >>
> > > >>
> > > >>
> > > >> It appeared to me from reading this issue that using the
> > > >> "CompressionDecorator" would resolve the issue so I modified my
> code to
> > > use
> > > >> DecompressingHttpClient (
> > > >>
> > >
> https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/DecompressingHttpClient.html
> > > )
> > > >> but the issue still persists as we can see from the below log
> output.
> > > It is
> > > >> caching using one (broken?) key but then looking it up using a
> different
> > > >> (correct?) key which is consistent with the bug above:
> > > >>
> > > >> 04-11 09:32:54.760: ... putting cache entry, url: {Accept-Encoding=}
> > > >> https://www.surespot.me:8080/images/b:f1/165
> > > >> 04-11 09:32:55.965: ... Cache miss [host:
> https://www.surespot.me:8080;
> > > >> uri: https://www.surespot.me:8080/images/b:f1/165]
> > > >>
> > > >> Am I missing something or is this still broken?
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Adam
> > > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> > >
> > >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
Re: 4.2.3 gzip caching still broken?
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sat, 2013-04-13 at 08:31 -0600, Adam Patacchiola wrote:
> Jon and Oleg,
>
> Thanks for your responses. Caching both the variant and non variant URLs
> makes sense, thanks for the explanation, but I'm still not sure it's
> functioning correctly. In BasicHttpCache I don't see how it would cache
> both URLs without my faulty updateEntry behavior. If I were checking the
> key set by the parentUri in updateEntry, this would return null because
> putEntry is never called with the parentUri explicitly.
>
> Unless you are saying updateEntry is supposed to update non-existent URLs?
>
> Oleg, the storeVariantEntry code in BasicHttpCache looks the same as 4.2.3
> here:
> https://github.com/apache/httpclient/blob/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/BasicHttpCache.java
> so
> I'm not sure how it could be fixed?
>
> Thanks,
>
> Adam
>
Adam,
I was more referring to refactoring of the way the caching aspect gets
wired into the HTTP request processing pipeline. If you think there is
something wrong with the BasicHttpCache#storeVariantEntry Jonathan is
the expert.
Anyway, feel free to raise a JIRA for this issue to make sure it does
not fall between the cracks. A test case demonstrating (reproducing) the
issue would help a great deal to ensure it gets resolved sooner.
Oleg
>
> On Thu, Apr 11, 2013 at 12:59 PM, Moore, Jonathan (CIM) <
> Jonathan_Moore@comcast.com> wrote:
>
> > Hi Adam,
> >
> > This is related to the way the caching module handles cache variants. In
> > this case I suspect the origin is (correctly) setting Vary: Accept-Encoding.
> >
> > There are two cache entries here; however only one of them should have the
> > response body present, IIRC. The version without the prepended request
> > headers is treated as the "parent" entry and the one with the headers is
> > the actual cached variant.
> >
> > This structure is in place because certain requests that pass through the
> > cache are required to invalidate all the variants for that URL, so we need
> > someplace to tie those together. It doubles the header space used but not
> > double the response body space. As there are more variants the overhead for
> > the duplicated headers drops further.
> >
> > Now, because you are seeing {Accept-Encoding=} this indicates that when
> > the cache saw the request come through it did not have the Accept-Encoding:
> > gzip on it at that point. I think this means you have the
> > DecompressingHttpClient and CachingHttpClient wired up backwards.
> >
> > You want the CachingHttpClient as close to the final DefaultHttpClient as
> > possible. So these should be layered as:
> >
> > DecompressingHttpClient -> CachingHttpClient -> DefaultHttpClient
> >
> > One of the updates in the 4.3 release will take more care of this wiring
> > for you out of the box.
> >
> > Jon
> > ........
> > Jon Moore
> >
> >
> > On Apr 11, 2013, at 2:20 PM, "Adam Patacchiola" <ad...@2fours.com> wrote:
> >
> > > I have done some more research on this and it appears that the caching is
> > > working, however it is adding 2 entries to the backing cache: one each
> > with
> > > and without the url pre-pended by {Accept-Encoding=}. This results in a
> > > cache miss for the get with the pre-pended url, and uses double the
> > storage
> > > space in whatever mechanism you are backing the client with. There was a
> > > bug in my backing store which led to me initially believing it was not
> > > caching the (correct) url.
> > >
> > > tl;dr it is caching but adding a duplicate invalid entry that never gets
> > > hit.
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Thu, Apr 11, 2013 at 10:13 AM, Adam Patacchiola <ad...@2fours.com>
> > wrote:
> > >
> > >> I'm using 4.2.3 with gzip compression and CachingHttpClient. Initially I
> > >> implemented the custom request/response interceptors as described here:
> > >>
> > https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientGZipContentCompression.javawhich
> > >> did not work, resulting in the issue described here:
> > >> https://issues.apache.org/jira/browse/HTTPCLIENT-1163.
> > >>
> > >>
> > >>
> > >> It appeared to me from reading this issue that using the
> > >> "CompressionDecorator" would resolve the issue so I modified my code to
> > use
> > >> DecompressingHttpClient (
> > >>
> > https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/DecompressingHttpClient.html
> > )
> > >> but the issue still persists as we can see from the below log output.
> > It is
> > >> caching using one (broken?) key but then looking it up using a different
> > >> (correct?) key which is consistent with the bug above:
> > >>
> > >> 04-11 09:32:54.760: ... putting cache entry, url: {Accept-Encoding=}
> > >> https://www.surespot.me:8080/images/b:f1/165
> > >> 04-11 09:32:55.965: ... Cache miss [host: https://www.surespot.me:8080;
> > >> uri: https://www.surespot.me:8080/images/b:f1/165]
> > >>
> > >> Am I missing something or is this still broken?
> > >>
> > >> Thanks,
> > >>
> > >> Adam
> > >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: 4.2.3 gzip caching still broken?
Posted by Adam Patacchiola <ad...@2fours.com>.
Jon and Oleg,
Thanks for your responses. Caching both the variant and non variant URLs
makes sense, thanks for the explanation, but I'm still not sure it's
functioning correctly. In BasicHttpCache I don't see how it would cache
both URLs without my faulty updateEntry behavior. If I were checking the
key set by the parentUri in updateEntry, this would return null because
putEntry is never called with the parentUri explicitly.
Unless you are saying updateEntry is supposed to update non-existent URLs?
Oleg, the storeVariantEntry code in BasicHttpCache looks the same as 4.2.3
here:
https://github.com/apache/httpclient/blob/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/BasicHttpCache.java
so
I'm not sure how it could be fixed?
Thanks,
Adam
On Thu, Apr 11, 2013 at 12:59 PM, Moore, Jonathan (CIM) <
Jonathan_Moore@comcast.com> wrote:
> Hi Adam,
>
> This is related to the way the caching module handles cache variants. In
> this case I suspect the origin is (correctly) setting Vary: Accept-Encoding.
>
> There are two cache entries here; however only one of them should have the
> response body present, IIRC. The version without the prepended request
> headers is treated as the "parent" entry and the one with the headers is
> the actual cached variant.
>
> This structure is in place because certain requests that pass through the
> cache are required to invalidate all the variants for that URL, so we need
> someplace to tie those together. It doubles the header space used but not
> double the response body space. As there are more variants the overhead for
> the duplicated headers drops further.
>
> Now, because you are seeing {Accept-Encoding=} this indicates that when
> the cache saw the request come through it did not have the Accept-Encoding:
> gzip on it at that point. I think this means you have the
> DecompressingHttpClient and CachingHttpClient wired up backwards.
>
> You want the CachingHttpClient as close to the final DefaultHttpClient as
> possible. So these should be layered as:
>
> DecompressingHttpClient -> CachingHttpClient -> DefaultHttpClient
>
> One of the updates in the 4.3 release will take more care of this wiring
> for you out of the box.
>
> Jon
> ........
> Jon Moore
>
>
> On Apr 11, 2013, at 2:20 PM, "Adam Patacchiola" <ad...@2fours.com> wrote:
>
> > I have done some more research on this and it appears that the caching is
> > working, however it is adding 2 entries to the backing cache: one each
> with
> > and without the url pre-pended by {Accept-Encoding=}. This results in a
> > cache miss for the get with the pre-pended url, and uses double the
> storage
> > space in whatever mechanism you are backing the client with. There was a
> > bug in my backing store which led to me initially believing it was not
> > caching the (correct) url.
> >
> > tl;dr it is caching but adding a duplicate invalid entry that never gets
> > hit.
> >
> >
> >
> >
> >
> >
> > On Thu, Apr 11, 2013 at 10:13 AM, Adam Patacchiola <ad...@2fours.com>
> wrote:
> >
> >> I'm using 4.2.3 with gzip compression and CachingHttpClient. Initially I
> >> implemented the custom request/response interceptors as described here:
> >>
> https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientGZipContentCompression.javawhich
> >> did not work, resulting in the issue described here:
> >> https://issues.apache.org/jira/browse/HTTPCLIENT-1163.
> >>
> >>
> >>
> >> It appeared to me from reading this issue that using the
> >> "CompressionDecorator" would resolve the issue so I modified my code to
> use
> >> DecompressingHttpClient (
> >>
> https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/DecompressingHttpClient.html
> )
> >> but the issue still persists as we can see from the below log output.
> It is
> >> caching using one (broken?) key but then looking it up using a different
> >> (correct?) key which is consistent with the bug above:
> >>
> >> 04-11 09:32:54.760: ... putting cache entry, url: {Accept-Encoding=}
> >> https://www.surespot.me:8080/images/b:f1/165
> >> 04-11 09:32:55.965: ... Cache miss [host: https://www.surespot.me:8080;
> >> uri: https://www.surespot.me:8080/images/b:f1/165]
> >>
> >> Am I missing something or is this still broken?
> >>
> >> Thanks,
> >>
> >> Adam
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
Re: 4.2.3 gzip caching still broken?
Posted by "Moore, Jonathan (CIM)" <Jo...@Comcast.com>.
Hi Adam,
This is related to the way the caching module handles cache variants. In this case I suspect the origin is (correctly) setting Vary: Accept-Encoding.
There are two cache entries here; however only one of them should have the response body present, IIRC. The version without the prepended request headers is treated as the "parent" entry and the one with the headers is the actual cached variant.
This structure is in place because certain requests that pass through the cache are required to invalidate all the variants for that URL, so we need someplace to tie those together. It doubles the header space used but not double the response body space. As there are more variants the overhead for the duplicated headers drops further.
Now, because you are seeing {Accept-Encoding=} this indicates that when the cache saw the request come through it did not have the Accept-Encoding: gzip on it at that point. I think this means you have the DecompressingHttpClient and CachingHttpClient wired up backwards.
You want the CachingHttpClient as close to the final DefaultHttpClient as possible. So these should be layered as:
DecompressingHttpClient -> CachingHttpClient -> DefaultHttpClient
One of the updates in the 4.3 release will take more care of this wiring for you out of the box.
Jon
........
Jon Moore
On Apr 11, 2013, at 2:20 PM, "Adam Patacchiola" <ad...@2fours.com> wrote:
> I have done some more research on this and it appears that the caching is
> working, however it is adding 2 entries to the backing cache: one each with
> and without the url pre-pended by {Accept-Encoding=}. This results in a
> cache miss for the get with the pre-pended url, and uses double the storage
> space in whatever mechanism you are backing the client with. There was a
> bug in my backing store which led to me initially believing it was not
> caching the (correct) url.
>
> tl;dr it is caching but adding a duplicate invalid entry that never gets
> hit.
>
>
>
>
>
>
> On Thu, Apr 11, 2013 at 10:13 AM, Adam Patacchiola <ad...@2fours.com> wrote:
>
>> I'm using 4.2.3 with gzip compression and CachingHttpClient. Initially I
>> implemented the custom request/response interceptors as described here:
>> https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientGZipContentCompression.java which
>> did not work, resulting in the issue described here:
>> https://issues.apache.org/jira/browse/HTTPCLIENT-1163.
>>
>>
>>
>> It appeared to me from reading this issue that using the
>> "CompressionDecorator" would resolve the issue so I modified my code to use
>> DecompressingHttpClient (
>> https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/DecompressingHttpClient.html)
>> but the issue still persists as we can see from the below log output. It is
>> caching using one (broken?) key but then looking it up using a different
>> (correct?) key which is consistent with the bug above:
>>
>> 04-11 09:32:54.760: ... putting cache entry, url: {Accept-Encoding=}
>> https://www.surespot.me:8080/images/b:f1/165
>> 04-11 09:32:55.965: ... Cache miss [host: https://www.surespot.me:8080;
>> uri: https://www.surespot.me:8080/images/b:f1/165]
>>
>> Am I missing something or is this still broken?
>>
>> Thanks,
>>
>> Adam
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: 4.2.3 gzip caching still broken?
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2013-04-11 at 12:12 -0600, Adam Patacchiola wrote:
> I have done some more research on this and it appears that the caching is
> working, however it is adding 2 entries to the backing cache: one each with
> and without the url pre-pended by {Accept-Encoding=}. This results in a
> cache miss for the get with the pre-pended url, and uses double the storage
> space in whatever mechanism you are backing the client with. There was a
> bug in my backing store which led to me initially believing it was not
> caching the (correct) url.
>
> tl;dr it is caching but adding a duplicate invalid entry that never gets
> hit.
>
Adam
This problem has been fixed properly in the 4.3 branch. Give 4.3-beta1 a
try.
Oleg
>
>
>
>
>
> On Thu, Apr 11, 2013 at 10:13 AM, Adam Patacchiola <ad...@2fours.com> wrote:
>
> > I'm using 4.2.3 with gzip compression and CachingHttpClient. Initially I
> > implemented the custom request/response interceptors as described here:
> > https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientGZipContentCompression.java which
> > did not work, resulting in the issue described here:
> > https://issues.apache.org/jira/browse/HTTPCLIENT-1163.
> >
> >
> >
> > It appeared to me from reading this issue that using the
> > "CompressionDecorator" would resolve the issue so I modified my code to use
> > DecompressingHttpClient (
> > https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/DecompressingHttpClient.html)
> > but the issue still persists as we can see from the below log output. It is
> > caching using one (broken?) key but then looking it up using a different
> > (correct?) key which is consistent with the bug above:
> >
> > 04-11 09:32:54.760: ... putting cache entry, url: {Accept-Encoding=}
> > https://www.surespot.me:8080/images/b:f1/165
> > 04-11 09:32:55.965: ... Cache miss [host: https://www.surespot.me:8080;
> > uri: https://www.surespot.me:8080/images/b:f1/165]
> >
> > Am I missing something or is this still broken?
> >
> > Thanks,
> >
> > Adam
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: 4.2.3 gzip caching still broken?
Posted by Adam Patacchiola <ad...@2fours.com>.
More investigation. The problem seems to arise in BasicHttpCache:
void storeVariantEntry(
final HttpHost target,
final HttpRequest req,
final HttpCacheEntry entry) throws IOException {
final String parentURI = uriExtractor.getURI(target, req);
final String variantURI = uriExtractor.getVariantURI(target, req,
entry);
storage.putEntry(variantURI, entry);
HttpCacheUpdateCallback callback = new HttpCacheUpdateCallback() {
public HttpCacheEntry update(HttpCacheEntry existing) throws
IOException {
return doGetUpdatedParentEntry(
req.getRequestLine().getUri(), existing, entry,
uriExtractor.getVariantKey(req, entry),
variantURI);
}
};
try {
storage.updateEntry(parentURI, callback);
} catch (HttpCacheUpdateException e) {
log.warn("Could not update key [" + parentURI + "]", e);
}
}
the variantURI and parentURI are different. In my storage implementation my
updateEntry looks like:
@Override
public void updateEntry(String arg0, HttpCacheUpdateCallback arg1) throws
IOException, HttpCacheUpdateException {
try {
SurespotLog.v(TAG, "updating entry, url: " + arg0);
putEntry(arg0, arg1.update(getEntry(arg0)));
}
catch (Exception e) {
SurespotLog.w(TAG, "updateEntry", e);
}
}
so it ended up adding 2 entries in my cache, one for the parentURI and one
for the variantURI which had the prepended {Accept-Encoding=}.
My updateEntry implementation is probably flawed in that I should get the
entry and check it exists before blindly updating it, but I am curious as
to why BasicHttpCache would put the entry with one key and update the entry
with a different key?
Thanks,
Adam
On Thu, Apr 11, 2013 at 12:12 PM, Adam Patacchiola <ad...@2fours.com> wrote:
> I have done some more research on this and it appears that the caching is
> working, however it is adding 2 entries to the backing cache: one each with
> and without the url pre-pended by {Accept-Encoding=}. This results in a
> cache miss for the get with the pre-pended url, and uses double the storage
> space in whatever mechanism you are backing the client with. There was a
> bug in my backing store which led to me initially believing it was not
> caching the (correct) url.
>
> tl;dr it is caching but adding a duplicate invalid entry that never gets
> hit.
>
>
>
>
>
>
> On Thu, Apr 11, 2013 at 10:13 AM, Adam Patacchiola <ad...@2fours.com>wrote:
>
>> I'm using 4.2.3 with gzip compression and CachingHttpClient. Initially I
>> implemented the custom request/response interceptors as described here:
>> https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientGZipContentCompression.java which
>> did not work, resulting in the issue described here:
>> https://issues.apache.org/jira/browse/HTTPCLIENT-1163.
>>
>>
>>
>> It appeared to me from reading this issue that using the
>> "CompressionDecorator" would resolve the issue so I modified my code to use
>> DecompressingHttpClient (
>> https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/DecompressingHttpClient.html)
>> but the issue still persists as we can see from the below log output. It is
>> caching using one (broken?) key but then looking it up using a different
>> (correct?) key which is consistent with the bug above:
>>
>> 04-11 09:32:54.760: ... putting cache entry, url: {Accept-Encoding=}
>> https://www.surespot.me:8080/images/b:f1/165
>> 04-11 09:32:55.965: ... Cache miss [host: https://www.surespot.me:8080;
>> uri: https://www.surespot.me:8080/images/b:f1/165]
>>
>> Am I missing something or is this still broken?
>>
>> Thanks,
>>
>> Adam
>>
>
>
Re: 4.2.3 gzip caching still broken?
Posted by Adam Patacchiola <ad...@2fours.com>.
I have done some more research on this and it appears that the caching is
working, however it is adding 2 entries to the backing cache: one each with
and without the url pre-pended by {Accept-Encoding=}. This results in a
cache miss for the get with the pre-pended url, and uses double the storage
space in whatever mechanism you are backing the client with. There was a
bug in my backing store which led to me initially believing it was not
caching the (correct) url.
tl;dr it is caching but adding a duplicate invalid entry that never gets
hit.
On Thu, Apr 11, 2013 at 10:13 AM, Adam Patacchiola <ad...@2fours.com> wrote:
> I'm using 4.2.3 with gzip compression and CachingHttpClient. Initially I
> implemented the custom request/response interceptors as described here:
> https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientGZipContentCompression.java which
> did not work, resulting in the issue described here:
> https://issues.apache.org/jira/browse/HTTPCLIENT-1163.
>
>
>
> It appeared to me from reading this issue that using the
> "CompressionDecorator" would resolve the issue so I modified my code to use
> DecompressingHttpClient (
> https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/DecompressingHttpClient.html)
> but the issue still persists as we can see from the below log output. It is
> caching using one (broken?) key but then looking it up using a different
> (correct?) key which is consistent with the bug above:
>
> 04-11 09:32:54.760: ... putting cache entry, url: {Accept-Encoding=}
> https://www.surespot.me:8080/images/b:f1/165
> 04-11 09:32:55.965: ... Cache miss [host: https://www.surespot.me:8080;
> uri: https://www.surespot.me:8080/images/b:f1/165]
>
> Am I missing something or is this still broken?
>
> Thanks,
>
> Adam
>