[GitHub] [pulsar] poorbarcode opened a new issue, #19826: PIP-259: Make the config httpMaxRequestHeaderSize to configurable

["poorbarcode (via GitHub)" <gi...@apache.org>]

poorbarcode opened a new issue, #19826:
URL: https://github.com/apache/pulsar/issues/19826

   ### Motivation
   
   We have two ways to manage pulsar's resources:
   - By client API (Can manage some resources, such as `create topic`, `create subscriber`, and so on)
   - By admin API (Can manage all the resources)
   
   The `client API` has no limit on the request length. And the `admin API` has a limit on the request length(such as HTTP request line and HTTP request headers), this restriction is done by the built-in web container Jetty.
   
   Almost resources can be created by two APIs, but can only be modified and deleted by `admin API`. This causes us to be unable to modify or delete resources created by `client API` with too long a name because it exceeds Jetty's default HTTP request URI length limit.
   
   ### Goal
   
   #### 1. For web servers
   Provide a way to modify Jetty's `httpMaxRequestHeaderSize` configuration (involves two servers: the web server in pulsar and the web server in pulsar-proxy)
   
   #### 2.For the internal client in pulsar-proxy
   Provide a way to modify Jetty-client's `httpClientRequestBufferSize` configuration.
   
   Since the pulsar-proxy handles HTTP requests like this： `pulsar-admin.sh` -> `proxy web server` -> `(highlight) internal client in proxy` -> `pulsar web server`. 
   
   When the internal client forwards a request, it forwards the request header and the request body, and all the data passes through a buffer( we call it Buf ), like this:
   - Receive a request
   - Put the request line and request headers input to the Buf.
   - <strong>(highlight)</strong>Flush the Buf ( If the data in the request line and request header exceeds the length of the buf, an error is reported )
   - Put the request body input to the Buf.
   - Flush the Buf if it is full.
   
   So we need a config to set the `buff size` of the Buf: `pulsar-proxy.conf.httpClientRequestBufferSize` -> `buf size of the internal client`.
   
   ### API Changes
   
   #### ServiceConfiguration.java
   ```java
      @FieldContext(
               category = CATEGORY_HTTP,
               doc = """
                   The maximum size in bytes of the request header.
                   Larger headers will allow for more and/or larger cookies plus larger form content encoded in a URL.
                   However, larger headers consume more memory and can make a server more vulnerable to denial of service
                   attacks.
                 """
       )
      private int httpMaxRequestHeaderSize = 8 * 1024;
   ```
   
   #### ProxyConfiguration.java
   
   ```java
       @FieldContext(
           minValue = 1,
           category = CATEGORY_HTTP,
           doc = """
                   The maximum size in bytes of the request header.
                   Larger headers will allow for more and/or larger cookies plus larger form content encoded in a URL.
                   However, larger headers consume more memory and can make a server more vulnerable to denial of service
                   attacks.
                 """
       )
       private int httpMaxRequestHeaderSize = 8 * 1024;
   
       @FieldContext(
           minValue = 1,
           category = CATEGORY_HTTP,
           doc = """
                    the size of the buffer used to write requests to Broker.
                    if "httpMaxRequestHeaderSize" is large than "httpClientRequestBufferSize", will set
                    "httpClientRequestBufferSize" to the value of "httpMaxRequestHeaderSize"
                 """
       )
       private int httpClientRequestBufferSize = httpMaxRequestHeaderSize;
   ```
   
   ### Implementation
   
   -
   
   ### Security Considerations
   
   -
   
   ### Alternatives
   
   _No response_
   
   ### Anything else?
   
   This change should cherry-pick into the previous branches ( includes `2.9~2.11` )
   
   If the user uses the features `RETRY Topic` or `DLQ`, it is possible that pulsar will automatically create some topics with names that are too long and cannot be managed, the [scenario has been discussed in the email](https://lists.apache.org/thread/q1m23ckyy10wvtzy65v8bwqwnh7r0gc8) before


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] poorbarcode closed issue #19826: PIP-259: Make the config httpMaxRequestHeaderSize of the pulsar web server to configurable

["poorbarcode (via GitHub)" <gi...@apache.org>]

poorbarcode closed issue #19826: PIP-259: Make the config httpMaxRequestHeaderSize of the pulsar web server to configurable
URL: https://github.com/apache/pulsar/issues/19826


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org