You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "ok2c (via GitHub)" <gi...@apache.org> on 2026/07/15 16:00:52 UTC

[PR] Added support for request body to the cache key generator [httpcomponents-client]

ok2c opened a new pull request, #850:
URL: https://github.com/apache/httpcomponents-client/pull/850

   This change-set extends the cache key generator with support for request with an enclosed content body.
   
   @desiderantes Please review and let me know if this approach works for you.
   
   @arturobernalg Please double-check.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Added support for request body to the cache key generator [httpcomponents-client]

Posted by "ok2c (via GitHub)" <gi...@apache.org>.
ok2c commented on code in PR #850:
URL: https://github.com/apache/httpcomponents-client/pull/850#discussion_r3589483913


##########
httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/CacheKeyGenerator.java:
##########
@@ -135,9 +140,33 @@ public String generateKey(final URI requestUri) {
      * @param host The host for this request
      * @param request the {@link HttpRequest}
      * @return cache key
+     *
+     * @deprecated Use {@link #generateKey(HttpHost, HttpRequest, Function)}
      */
+    @Deprecated
     public String generateKey(final HttpHost host, final HttpRequest request) {
+        return generateKey(host, request, r -> null);
+    }
+
+    /**
+     * Computes a root key for the given {@link HttpHost}, {@link HttpRequest} and
+     * request body that can be used as a unique identifier for cached resources.
+     *
+     * @param host The host for this request
+     * @param request the {@link HttpRequest}
+     * @param bodyExtractor function to extract request body. May be {@code null}
+     * @return cache key
+     *
+     * @since 5.7
+     */
+    public <T extends HttpRequest> String generateKey(final HttpHost host,
+                                                      final T request,
+                                                      final Function<T, String> bodyExtractor) {
         final String s = CacheSupport.requestUriRaw(host, request);
+        final String body = bodyExtractor != null ? bodyExtractor.apply(request) : null;
+        if (body != null) {

Review Comment:
   @desiderantes This is a temporary sanity check as the caching layer simply does not work correctly with entity enclosing requests. This is something I presume you are going to fix in your pull request.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Added support for request body to the cache key generator [httpcomponents-client]

Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on code in PR #850:
URL: https://github.com/apache/httpcomponents-client/pull/850#discussion_r3589779012


##########
httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/AsyncCachingExec.java:
##########
@@ -302,7 +302,7 @@ public void completed(final CacheMatch result) {
                 final CacheHit root = result != null ? result.root : null;
                 if (hit == null) {
                     if (requestCollapsingEnabled && root == null && !requestCacheControl.isOnlyIfCached()) {
-                        final String cacheKey = CacheKeyGenerator.INSTANCE.generateKey(target, cacheRequest);
+                        final String cacheKey = CacheKeyGenerator.INSTANCE.generateKey(target, cacheRequest, SimpleHttpRequest::getBodyText);

Review Comment:
   Would it make more sense to use byte[] here and SimpleHttpRequest::getBodyBytes at the call sites? getBodyText() can collapse different binary or invalid byte sequences into the same String, which would make the eventual cache key ambiguous.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Added support for request body to the cache key generator [httpcomponents-client]

Posted by "desiderantes (via GitHub)" <gi...@apache.org>.
desiderantes commented on code in PR #850:
URL: https://github.com/apache/httpcomponents-client/pull/850#discussion_r3589389448


##########
httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/CacheKeyGenerator.java:
##########
@@ -135,9 +140,33 @@ public String generateKey(final URI requestUri) {
      * @param host The host for this request
      * @param request the {@link HttpRequest}
      * @return cache key
+     *
+     * @deprecated Use {@link #generateKey(HttpHost, HttpRequest, Function)}
      */
+    @Deprecated
     public String generateKey(final HttpHost host, final HttpRequest request) {
+        return generateKey(host, request, r -> null);
+    }
+
+    /**
+     * Computes a root key for the given {@link HttpHost}, {@link HttpRequest} and
+     * request body that can be used as a unique identifier for cached resources.
+     *
+     * @param host The host for this request
+     * @param request the {@link HttpRequest}
+     * @param bodyExtractor function to extract request body. May be {@code null}
+     * @return cache key
+     *
+     * @since 5.7
+     */
+    public <T extends HttpRequest> String generateKey(final HttpHost host,
+                                                      final T request,
+                                                      final Function<T, String> bodyExtractor) {
         final String s = CacheSupport.requestUriRaw(host, request);
+        final String body = bodyExtractor != null ? bodyExtractor.apply(request) : null;
+        if (body != null) {

Review Comment:
   Is it supposed to fail now if the body extractor succeeds?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Added support for request body to the cache key generator [httpcomponents-client]

Posted by "desiderantes (via GitHub)" <gi...@apache.org>.
desiderantes commented on code in PR #850:
URL: https://github.com/apache/httpcomponents-client/pull/850#discussion_r3590967982


##########
httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/AsyncCachingExec.java:
##########
@@ -302,7 +302,7 @@ public void completed(final CacheMatch result) {
                 final CacheHit root = result != null ? result.root : null;
                 if (hit == null) {
                     if (requestCollapsingEnabled && root == null && !requestCacheControl.isOnlyIfCached()) {
-                        final String cacheKey = CacheKeyGenerator.INSTANCE.generateKey(target, cacheRequest);
+                        final String cacheKey = CacheKeyGenerator.INSTANCE.generateKey(target, cacheRequest, SimpleHttpRequest::getBodyText);

Review Comment:
   true, not all content is text-like, sorry, this is a good suggestion



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Added support for request body to the cache key generator [httpcomponents-client]

Posted by "ok2c (via GitHub)" <gi...@apache.org>.
ok2c merged PR #850:
URL: https://github.com/apache/httpcomponents-client/pull/850


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Added support for request body to the cache key generator [httpcomponents-client]

Posted by "desiderantes (via GitHub)" <gi...@apache.org>.
desiderantes commented on code in PR #850:
URL: https://github.com/apache/httpcomponents-client/pull/850#discussion_r3590080660


##########
httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/AsyncCachingExec.java:
##########
@@ -302,7 +302,7 @@ public void completed(final CacheMatch result) {
                 final CacheHit root = result != null ? result.root : null;
                 if (hit == null) {
                     if (requestCollapsingEnabled && root == null && !requestCacheControl.isOnlyIfCached()) {
-                        final String cacheKey = CacheKeyGenerator.INSTANCE.generateKey(target, cacheRequest);
+                        final String cacheKey = CacheKeyGenerator.INSTANCE.generateKey(target, cacheRequest, SimpleHttpRequest::getBodyText);

Review Comment:
   hmm... it is also possible that subclasses of Request might have a certain representation for certain content types (like a sorted object/array for json, or ordered form fields, etc), which might affect the caching as different bodies should be cached because they converge to the same canonical representation



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Added support for request body to the cache key generator [httpcomponents-client]

Posted by "ok2c (via GitHub)" <gi...@apache.org>.
ok2c commented on code in PR #850:
URL: https://github.com/apache/httpcomponents-client/pull/850#discussion_r3593324045


##########
httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/AsyncCachingExec.java:
##########
@@ -302,7 +302,7 @@ public void completed(final CacheMatch result) {
                 final CacheHit root = result != null ? result.root : null;
                 if (hit == null) {
                     if (requestCollapsingEnabled && root == null && !requestCacheControl.isOnlyIfCached()) {
-                        final String cacheKey = CacheKeyGenerator.INSTANCE.generateKey(target, cacheRequest);
+                        final String cacheKey = CacheKeyGenerator.INSTANCE.generateKey(target, cacheRequest, SimpleHttpRequest::getBodyText);

Review Comment:
   @desiderantes @arturobernalg I can hardly imagine a QUERY request with binary query content but so be it. Please do another pass.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Added support for request body to the cache key generator [httpcomponents-client]

Posted by "desiderantes (via GitHub)" <gi...@apache.org>.
desiderantes commented on code in PR #850:
URL: https://github.com/apache/httpcomponents-client/pull/850#discussion_r3589520900


##########
httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/CacheKeyGenerator.java:
##########
@@ -135,9 +140,33 @@ public String generateKey(final URI requestUri) {
      * @param host The host for this request
      * @param request the {@link HttpRequest}
      * @return cache key
+     *
+     * @deprecated Use {@link #generateKey(HttpHost, HttpRequest, Function)}
      */
+    @Deprecated
     public String generateKey(final HttpHost host, final HttpRequest request) {
+        return generateKey(host, request, r -> null);
+    }
+
+    /**
+     * Computes a root key for the given {@link HttpHost}, {@link HttpRequest} and
+     * request body that can be used as a unique identifier for cached resources.
+     *
+     * @param host The host for this request
+     * @param request the {@link HttpRequest}
+     * @param bodyExtractor function to extract request body. May be {@code null}
+     * @return cache key
+     *
+     * @since 5.7
+     */
+    public <T extends HttpRequest> String generateKey(final HttpHost host,
+                                                      final T request,
+                                                      final Function<T, String> bodyExtractor) {
         final String s = CacheSupport.requestUriRaw(host, request);
+        final String body = bodyExtractor != null ? bodyExtractor.apply(request) : null;
+        if (body != null) {

Review Comment:
   good, just wanted to confirm



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Added support for request body to the cache key generator [httpcomponents-client]

Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on code in PR #850:
URL: https://github.com/apache/httpcomponents-client/pull/850#discussion_r3590136221


##########
httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/AsyncCachingExec.java:
##########
@@ -302,7 +302,7 @@ public void completed(final CacheMatch result) {
                 final CacheHit root = result != null ? result.root : null;
                 if (hit == null) {
                     if (requestCollapsingEnabled && root == null && !requestCacheControl.isOnlyIfCached()) {
-                        final String cacheKey = CacheKeyGenerator.INSTANCE.generateKey(target, cacheRequest);
+                        final String cacheKey = CacheKeyGenerator.INSTANCE.generateKey(target, cacheRequest, SimpleHttpRequest::getBodyText);

Review Comment:
    That still does not require String. A request implementation could canonicalize the content and return the resulting bytes. That preserves the option for content-type-specific normalization without forcing arbitrary content through text decoding.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org