You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "Sam Perman (JIRA)" <ji...@apache.org> on 2012/07/04 17:10:34 UTC

[jira] [Created] (HTTPCLIENT-1213) performance issue with CachingHttpClient

Sam Perman created HTTPCLIENT-1213:
--------------------------------------

             Summary: performance issue with CachingHttpClient
                 Key: HTTPCLIENT-1213
                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1213
             Project: HttpComponents HttpClient
          Issue Type: Bug
          Components: Cache
    Affects Versions: 4.2 Final
            Reporter: Sam Perman


We're using the CachingHttpClient and are seeing a spike in CPU usage when it is enabled. We've profiled our application and see that most of the time is being spent parsing dates. Specifically, it is trying to get the age of a cache entry on a cache hit by parsing the "Date" header on the HttpCacheEntry.  I had a couple questions:

1) Why can't this use the responseDate value that lives on HttpCacheEntry? (This would avoid the overhead of parsing)
2) If it needs to parse, is it possible to remember the result on the HttpCacheEntry so it doesn't need to be parsed every time?

We are using version 4.2

Here is the full backtrace we are seeing:

org.apache.http.impl.cookie.DateUtils.parseDate(String)
  org.apache.http.impl.client.cache.CacheValidityPolicy.getDateValue(HttpCacheEntry)
    org.apache.http.impl.client.cache.CacheValidityPolicy.getApparentAgeSecs(HttpCacheEntry)
      org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedReceivedAgeSecs(HttpCacheEntry)
        org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedInitialAgeSecs(HttpCacheEntry)
          org.apache.http.impl.client.cache.CacheValidityPolicy.getCurrentAgeSecs(HttpCacheEntry, Date)

There are a couple callers to "getCorrectedAgeSecs":

CacheValidityPolicy.isResponseFresh(HttpCacheEntry, Date)
  CachedResponseSuitabilityChecker.isFreshEnough(HttpCacheEntry, HttpRequest, Date)
    CachedResponseSuitabilityChecker.canCachedResponseBeUsed(HttpHost, HttpRequest, HttpCacheEntry, Date)
      CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)

CachedHttpResponseGenerator.generateResponse(HttpCacheEntry)
  CachingHttpClient.generateCachedResponse(HttpRequest, HttpContext, HttpCacheEntry, Date)
    CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)

Looking at the code, it looks like this section from CachingHttpClient.handleCacheHit will result in parsing the date twice (apologies if I'm misreading this)

if (suitabilityChecker.canCachedResponseBeUsed(target, request, entry, now)) {
    return generateCachedResponse(request, context, entry, now);
}

Both the call to "canCachedResponseBeUsed" and the call to "generatedCachedResponse" will ultimately call "getCurrentAgeSecs" and parse the Date header.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


[jira] [Updated] (HTTPCLIENT-1213) performance issue with CachingHttpClient

Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HTTPCLIENT-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Oleg Kalnichevski updated HTTPCLIENT-1213:
------------------------------------------

    Fix Version/s: 4.3 Final
       Issue Type: Improvement  (was: Bug)
    
> performance issue with CachingHttpClient
> ----------------------------------------
>
>                 Key: HTTPCLIENT-1213
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1213
>             Project: HttpComponents HttpClient
>          Issue Type: Improvement
>          Components: Cache
>    Affects Versions: 4.2 Final
>            Reporter: Sam Perman
>             Fix For: 4.3 Final
>
>
> We're using the CachingHttpClient and are seeing a spike in CPU usage when it is enabled. We've profiled our application and see that most of the time is being spent parsing dates. Specifically, it is trying to get the age of a cache entry on a cache hit by parsing the "Date" header on the HttpCacheEntry.  I had a couple questions:
> 1) Why can't this use the responseDate value that lives on HttpCacheEntry? (This would avoid the overhead of parsing)
> 2) If it needs to parse, is it possible to remember the result on the HttpCacheEntry so it doesn't need to be parsed every time?
> We are using version 4.2
> Here is the full backtrace we are seeing:
> org.apache.http.impl.cookie.DateUtils.parseDate(String)
>   org.apache.http.impl.client.cache.CacheValidityPolicy.getDateValue(HttpCacheEntry)
>     org.apache.http.impl.client.cache.CacheValidityPolicy.getApparentAgeSecs(HttpCacheEntry)
>       org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedReceivedAgeSecs(HttpCacheEntry)
>         org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedInitialAgeSecs(HttpCacheEntry)
>           org.apache.http.impl.client.cache.CacheValidityPolicy.getCurrentAgeSecs(HttpCacheEntry, Date)
> There are a couple callers to "getCorrectedAgeSecs":
> CacheValidityPolicy.isResponseFresh(HttpCacheEntry, Date)
>   CachedResponseSuitabilityChecker.isFreshEnough(HttpCacheEntry, HttpRequest, Date)
>     CachedResponseSuitabilityChecker.canCachedResponseBeUsed(HttpHost, HttpRequest, HttpCacheEntry, Date)
>       CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> CachedHttpResponseGenerator.generateResponse(HttpCacheEntry)
>   CachingHttpClient.generateCachedResponse(HttpRequest, HttpContext, HttpCacheEntry, Date)
>     CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> Looking at the code, it looks like this section from CachingHttpClient.handleCacheHit will result in parsing the date twice (apologies if I'm misreading this)
> if (suitabilityChecker.canCachedResponseBeUsed(target, request, entry, now)) {
>     return generateCachedResponse(request, context, entry, now);
> }
> Both the call to "canCachedResponseBeUsed" and the call to "generatedCachedResponse" will ultimately call "getCurrentAgeSecs" and parse the Date header.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


[jira] [Updated] (HTTPCLIENT-1213) performance issue with CachingHttpClient

Posted by "Sam Perman (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HTTPCLIENT-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sam Perman updated HTTPCLIENT-1213:
-----------------------------------

    Attachment: httpclient-1213.patch

Here is a proposed fix for this issue. It will parse the Date header with the HttpCacheEntry is created rather than each time it is accessed.
                
> performance issue with CachingHttpClient
> ----------------------------------------
>
>                 Key: HTTPCLIENT-1213
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1213
>             Project: HttpComponents HttpClient
>          Issue Type: Improvement
>          Components: Cache
>    Affects Versions: 4.2 Final
>            Reporter: Sam Perman
>             Fix For: 4.3 Final
>
>         Attachments: httpclient-1213.patch
>
>
> We're using the CachingHttpClient and are seeing a spike in CPU usage when it is enabled. We've profiled our application and see that most of the time is being spent parsing dates. Specifically, it is trying to get the age of a cache entry on a cache hit by parsing the "Date" header on the HttpCacheEntry.  I had a couple questions:
> 1) Why can't this use the responseDate value that lives on HttpCacheEntry? (This would avoid the overhead of parsing)
> 2) If it needs to parse, is it possible to remember the result on the HttpCacheEntry so it doesn't need to be parsed every time?
> We are using version 4.2
> Here is the full backtrace we are seeing:
> org.apache.http.impl.cookie.DateUtils.parseDate(String)
>   org.apache.http.impl.client.cache.CacheValidityPolicy.getDateValue(HttpCacheEntry)
>     org.apache.http.impl.client.cache.CacheValidityPolicy.getApparentAgeSecs(HttpCacheEntry)
>       org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedReceivedAgeSecs(HttpCacheEntry)
>         org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedInitialAgeSecs(HttpCacheEntry)
>           org.apache.http.impl.client.cache.CacheValidityPolicy.getCurrentAgeSecs(HttpCacheEntry, Date)
> There are a couple callers to "getCorrectedAgeSecs":
> CacheValidityPolicy.isResponseFresh(HttpCacheEntry, Date)
>   CachedResponseSuitabilityChecker.isFreshEnough(HttpCacheEntry, HttpRequest, Date)
>     CachedResponseSuitabilityChecker.canCachedResponseBeUsed(HttpHost, HttpRequest, HttpCacheEntry, Date)
>       CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> CachedHttpResponseGenerator.generateResponse(HttpCacheEntry)
>   CachingHttpClient.generateCachedResponse(HttpRequest, HttpContext, HttpCacheEntry, Date)
>     CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> Looking at the code, it looks like this section from CachingHttpClient.handleCacheHit will result in parsing the date twice (apologies if I'm misreading this)
> if (suitabilityChecker.canCachedResponseBeUsed(target, request, entry, now)) {
>     return generateCachedResponse(request, context, entry, now);
> }
> Both the call to "canCachedResponseBeUsed" and the call to "generatedCachedResponse" will ultimately call "getCurrentAgeSecs" and parse the Date header.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


[jira] [Updated] (HTTPCLIENT-1213) performance issue with CachingHttpClient

Posted by "Sam Perman (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HTTPCLIENT-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sam Perman updated HTTPCLIENT-1213:
-----------------------------------

    Attachment: httpclient-1213.patch

Here is a version of the patch that retains that protected method (and deprecates it.)
                
> performance issue with CachingHttpClient
> ----------------------------------------
>
>                 Key: HTTPCLIENT-1213
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1213
>             Project: HttpComponents HttpClient
>          Issue Type: Improvement
>          Components: Cache
>    Affects Versions: 4.2 Final
>            Reporter: Sam Perman
>             Fix For: 4.3 Final
>
>         Attachments: httpclient-1213.patch
>
>
> We're using the CachingHttpClient and are seeing a spike in CPU usage when it is enabled. We've profiled our application and see that most of the time is being spent parsing dates. Specifically, it is trying to get the age of a cache entry on a cache hit by parsing the "Date" header on the HttpCacheEntry.  I had a couple questions:
> 1) Why can't this use the responseDate value that lives on HttpCacheEntry? (This would avoid the overhead of parsing)
> 2) If it needs to parse, is it possible to remember the result on the HttpCacheEntry so it doesn't need to be parsed every time?
> We are using version 4.2
> Here is the full backtrace we are seeing:
> org.apache.http.impl.cookie.DateUtils.parseDate(String)
>   org.apache.http.impl.client.cache.CacheValidityPolicy.getDateValue(HttpCacheEntry)
>     org.apache.http.impl.client.cache.CacheValidityPolicy.getApparentAgeSecs(HttpCacheEntry)
>       org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedReceivedAgeSecs(HttpCacheEntry)
>         org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedInitialAgeSecs(HttpCacheEntry)
>           org.apache.http.impl.client.cache.CacheValidityPolicy.getCurrentAgeSecs(HttpCacheEntry, Date)
> There are a couple callers to "getCorrectedAgeSecs":
> CacheValidityPolicy.isResponseFresh(HttpCacheEntry, Date)
>   CachedResponseSuitabilityChecker.isFreshEnough(HttpCacheEntry, HttpRequest, Date)
>     CachedResponseSuitabilityChecker.canCachedResponseBeUsed(HttpHost, HttpRequest, HttpCacheEntry, Date)
>       CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> CachedHttpResponseGenerator.generateResponse(HttpCacheEntry)
>   CachingHttpClient.generateCachedResponse(HttpRequest, HttpContext, HttpCacheEntry, Date)
>     CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> Looking at the code, it looks like this section from CachingHttpClient.handleCacheHit will result in parsing the date twice (apologies if I'm misreading this)
> if (suitabilityChecker.canCachedResponseBeUsed(target, request, entry, now)) {
>     return generateCachedResponse(request, context, entry, now);
> }
> Both the call to "canCachedResponseBeUsed" and the call to "generatedCachedResponse" will ultimately call "getCurrentAgeSecs" and parse the Date header.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


[jira] [Commented] (HTTPCLIENT-1213) performance issue with CachingHttpClient

Posted by "Sam Perman (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HTTPCLIENT-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407096#comment-13407096 ] 

Sam Perman commented on HTTPCLIENT-1213:
----------------------------------------

Just a note on that patch: I deleted the protected method "CacheValidityPolicy.getDateValue". I'm not sure what your policy is on doing things like that, so if it needs to be left in, it would look like this:


protected Date getDateValue(final HttpCacheEntry entry) {
    return entry.getDateHeaderValue();
}

                
> performance issue with CachingHttpClient
> ----------------------------------------
>
>                 Key: HTTPCLIENT-1213
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1213
>             Project: HttpComponents HttpClient
>          Issue Type: Improvement
>          Components: Cache
>    Affects Versions: 4.2 Final
>            Reporter: Sam Perman
>             Fix For: 4.3 Final
>
>         Attachments: httpclient-1213.patch
>
>
> We're using the CachingHttpClient and are seeing a spike in CPU usage when it is enabled. We've profiled our application and see that most of the time is being spent parsing dates. Specifically, it is trying to get the age of a cache entry on a cache hit by parsing the "Date" header on the HttpCacheEntry.  I had a couple questions:
> 1) Why can't this use the responseDate value that lives on HttpCacheEntry? (This would avoid the overhead of parsing)
> 2) If it needs to parse, is it possible to remember the result on the HttpCacheEntry so it doesn't need to be parsed every time?
> We are using version 4.2
> Here is the full backtrace we are seeing:
> org.apache.http.impl.cookie.DateUtils.parseDate(String)
>   org.apache.http.impl.client.cache.CacheValidityPolicy.getDateValue(HttpCacheEntry)
>     org.apache.http.impl.client.cache.CacheValidityPolicy.getApparentAgeSecs(HttpCacheEntry)
>       org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedReceivedAgeSecs(HttpCacheEntry)
>         org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedInitialAgeSecs(HttpCacheEntry)
>           org.apache.http.impl.client.cache.CacheValidityPolicy.getCurrentAgeSecs(HttpCacheEntry, Date)
> There are a couple callers to "getCorrectedAgeSecs":
> CacheValidityPolicy.isResponseFresh(HttpCacheEntry, Date)
>   CachedResponseSuitabilityChecker.isFreshEnough(HttpCacheEntry, HttpRequest, Date)
>     CachedResponseSuitabilityChecker.canCachedResponseBeUsed(HttpHost, HttpRequest, HttpCacheEntry, Date)
>       CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> CachedHttpResponseGenerator.generateResponse(HttpCacheEntry)
>   CachingHttpClient.generateCachedResponse(HttpRequest, HttpContext, HttpCacheEntry, Date)
>     CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> Looking at the code, it looks like this section from CachingHttpClient.handleCacheHit will result in parsing the date twice (apologies if I'm misreading this)
> if (suitabilityChecker.canCachedResponseBeUsed(target, request, entry, now)) {
>     return generateCachedResponse(request, context, entry, now);
> }
> Both the call to "canCachedResponseBeUsed" and the call to "generatedCachedResponse" will ultimately call "getCurrentAgeSecs" and parse the Date header.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


[jira] [Commented] (HTTPCLIENT-1213) performance issue with CachingHttpClient

Posted by "Sam Perman (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HTTPCLIENT-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406574#comment-13406574 ] 

Sam Perman commented on HTTPCLIENT-1213:
----------------------------------------

As mentioned on the mailing list, using the responseDate on the HttpCacheEntry per RFC2616 but maybe the result of the parsing can be remembered.
                
> performance issue with CachingHttpClient
> ----------------------------------------
>
>                 Key: HTTPCLIENT-1213
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1213
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: Cache
>    Affects Versions: 4.2 Final
>            Reporter: Sam Perman
>
> We're using the CachingHttpClient and are seeing a spike in CPU usage when it is enabled. We've profiled our application and see that most of the time is being spent parsing dates. Specifically, it is trying to get the age of a cache entry on a cache hit by parsing the "Date" header on the HttpCacheEntry.  I had a couple questions:
> 1) Why can't this use the responseDate value that lives on HttpCacheEntry? (This would avoid the overhead of parsing)
> 2) If it needs to parse, is it possible to remember the result on the HttpCacheEntry so it doesn't need to be parsed every time?
> We are using version 4.2
> Here is the full backtrace we are seeing:
> org.apache.http.impl.cookie.DateUtils.parseDate(String)
>   org.apache.http.impl.client.cache.CacheValidityPolicy.getDateValue(HttpCacheEntry)
>     org.apache.http.impl.client.cache.CacheValidityPolicy.getApparentAgeSecs(HttpCacheEntry)
>       org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedReceivedAgeSecs(HttpCacheEntry)
>         org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedInitialAgeSecs(HttpCacheEntry)
>           org.apache.http.impl.client.cache.CacheValidityPolicy.getCurrentAgeSecs(HttpCacheEntry, Date)
> There are a couple callers to "getCorrectedAgeSecs":
> CacheValidityPolicy.isResponseFresh(HttpCacheEntry, Date)
>   CachedResponseSuitabilityChecker.isFreshEnough(HttpCacheEntry, HttpRequest, Date)
>     CachedResponseSuitabilityChecker.canCachedResponseBeUsed(HttpHost, HttpRequest, HttpCacheEntry, Date)
>       CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> CachedHttpResponseGenerator.generateResponse(HttpCacheEntry)
>   CachingHttpClient.generateCachedResponse(HttpRequest, HttpContext, HttpCacheEntry, Date)
>     CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> Looking at the code, it looks like this section from CachingHttpClient.handleCacheHit will result in parsing the date twice (apologies if I'm misreading this)
> if (suitabilityChecker.canCachedResponseBeUsed(target, request, entry, now)) {
>     return generateCachedResponse(request, context, entry, now);
> }
> Both the call to "canCachedResponseBeUsed" and the call to "generatedCachedResponse" will ultimately call "getCurrentAgeSecs" and parse the Date header.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


[jira] [Commented] (HTTPCLIENT-1213) performance issue with CachingHttpClient

Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HTTPCLIENT-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407150#comment-13407150 ] 

Oleg Kalnichevski commented on HTTPCLIENT-1213:
-----------------------------------------------

Ideally we should preserve full binary compatibility with the 4.2.x line, so the protected method in question should be deprecated, not removed. 

You can use 'mvn:clirr check' command to make sure your private copy is still fully compatible with the 4.2 GA release.

Oleg
                
> performance issue with CachingHttpClient
> ----------------------------------------
>
>                 Key: HTTPCLIENT-1213
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1213
>             Project: HttpComponents HttpClient
>          Issue Type: Improvement
>          Components: Cache
>    Affects Versions: 4.2 Final
>            Reporter: Sam Perman
>             Fix For: 4.3 Final
>
>         Attachments: httpclient-1213.patch
>
>
> We're using the CachingHttpClient and are seeing a spike in CPU usage when it is enabled. We've profiled our application and see that most of the time is being spent parsing dates. Specifically, it is trying to get the age of a cache entry on a cache hit by parsing the "Date" header on the HttpCacheEntry.  I had a couple questions:
> 1) Why can't this use the responseDate value that lives on HttpCacheEntry? (This would avoid the overhead of parsing)
> 2) If it needs to parse, is it possible to remember the result on the HttpCacheEntry so it doesn't need to be parsed every time?
> We are using version 4.2
> Here is the full backtrace we are seeing:
> org.apache.http.impl.cookie.DateUtils.parseDate(String)
>   org.apache.http.impl.client.cache.CacheValidityPolicy.getDateValue(HttpCacheEntry)
>     org.apache.http.impl.client.cache.CacheValidityPolicy.getApparentAgeSecs(HttpCacheEntry)
>       org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedReceivedAgeSecs(HttpCacheEntry)
>         org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedInitialAgeSecs(HttpCacheEntry)
>           org.apache.http.impl.client.cache.CacheValidityPolicy.getCurrentAgeSecs(HttpCacheEntry, Date)
> There are a couple callers to "getCorrectedAgeSecs":
> CacheValidityPolicy.isResponseFresh(HttpCacheEntry, Date)
>   CachedResponseSuitabilityChecker.isFreshEnough(HttpCacheEntry, HttpRequest, Date)
>     CachedResponseSuitabilityChecker.canCachedResponseBeUsed(HttpHost, HttpRequest, HttpCacheEntry, Date)
>       CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> CachedHttpResponseGenerator.generateResponse(HttpCacheEntry)
>   CachingHttpClient.generateCachedResponse(HttpRequest, HttpContext, HttpCacheEntry, Date)
>     CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> Looking at the code, it looks like this section from CachingHttpClient.handleCacheHit will result in parsing the date twice (apologies if I'm misreading this)
> if (suitabilityChecker.canCachedResponseBeUsed(target, request, entry, now)) {
>     return generateCachedResponse(request, context, entry, now);
> }
> Both the call to "canCachedResponseBeUsed" and the call to "generatedCachedResponse" will ultimately call "getCurrentAgeSecs" and parse the Date header.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


[jira] [Updated] (HTTPCLIENT-1213) performance issue with CachingHttpClient

Posted by "Sam Perman (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HTTPCLIENT-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sam Perman updated HTTPCLIENT-1213:
-----------------------------------

    Attachment:     (was: httpclient-1213.patch)
    
> performance issue with CachingHttpClient
> ----------------------------------------
>
>                 Key: HTTPCLIENT-1213
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1213
>             Project: HttpComponents HttpClient
>          Issue Type: Improvement
>          Components: Cache
>    Affects Versions: 4.2 Final
>            Reporter: Sam Perman
>             Fix For: 4.3 Final
>
>         Attachments: httpclient-1213.patch
>
>
> We're using the CachingHttpClient and are seeing a spike in CPU usage when it is enabled. We've profiled our application and see that most of the time is being spent parsing dates. Specifically, it is trying to get the age of a cache entry on a cache hit by parsing the "Date" header on the HttpCacheEntry.  I had a couple questions:
> 1) Why can't this use the responseDate value that lives on HttpCacheEntry? (This would avoid the overhead of parsing)
> 2) If it needs to parse, is it possible to remember the result on the HttpCacheEntry so it doesn't need to be parsed every time?
> We are using version 4.2
> Here is the full backtrace we are seeing:
> org.apache.http.impl.cookie.DateUtils.parseDate(String)
>   org.apache.http.impl.client.cache.CacheValidityPolicy.getDateValue(HttpCacheEntry)
>     org.apache.http.impl.client.cache.CacheValidityPolicy.getApparentAgeSecs(HttpCacheEntry)
>       org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedReceivedAgeSecs(HttpCacheEntry)
>         org.apache.http.impl.client.cache.CacheValidityPolicy.getCorrectedInitialAgeSecs(HttpCacheEntry)
>           org.apache.http.impl.client.cache.CacheValidityPolicy.getCurrentAgeSecs(HttpCacheEntry, Date)
> There are a couple callers to "getCorrectedAgeSecs":
> CacheValidityPolicy.isResponseFresh(HttpCacheEntry, Date)
>   CachedResponseSuitabilityChecker.isFreshEnough(HttpCacheEntry, HttpRequest, Date)
>     CachedResponseSuitabilityChecker.canCachedResponseBeUsed(HttpHost, HttpRequest, HttpCacheEntry, Date)
>       CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> CachedHttpResponseGenerator.generateResponse(HttpCacheEntry)
>   CachingHttpClient.generateCachedResponse(HttpRequest, HttpContext, HttpCacheEntry, Date)
>     CachingHttpClient.handleCacheHit(HttpHost, HttpRequest, HttpContext, HttpCacheEntry)
> Looking at the code, it looks like this section from CachingHttpClient.handleCacheHit will result in parsing the date twice (apologies if I'm misreading this)
> if (suitabilityChecker.canCachedResponseBeUsed(target, request, entry, now)) {
>     return generateCachedResponse(request, context, entry, now);
> }
> Both the call to "canCachedResponseBeUsed" and the call to "generatedCachedResponse" will ultimately call "getCurrentAgeSecs" and parse the Date header.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org