[PROPOSAL] New API to look up strings against our set of WKS constants

[Leif Hedstrom <zw...@apache.org>]

Hi,

In our code, we have a concept of well-known-strings (WKSs). These are used to sort of normalize common header and URL strings, and can be used in some cases for optimizing lookups in request header heaps etc. When a plugin uses a WKS directly, such as TS_MIME_FIELD_CACHE_CONTROL to look for a header, the core will automatically use this optimally, because this constant points to the internal WKS (which is MIME_FIELD_CACHE_CONTROL internally).

Now, this is all fine, if the plugin knows what headers it’s looking for. But for plugins such as header_rewrite, or TxnBox, where an external configuration specifies which headers to work with, there’s currently no easy way (rather than iterations) to find the WKS corresponding to a user specified string. As such, I’m suggesting that we expose the internal WKS lookup functionality with a public API:

    tsapi const char *TSMimeHdrStringToWKS(const char *str, int length);


The length parameter is “optional” as per normal APIs here, i.e. if you specify a negative value for the length, the API will do a strlen() of the input str for you.

I’ve made a PR with these changes, which also includes the necessary support for the core header_rewrite plugin to use this new API. If there are no concerns with the API changes, after discussions here and on the PR, I will change the PR from a Draft to normal PR for proper approval.

Thanks,

— Leif

https://github.com/apache/trafficserver/pull/9643


The implementation as such is simple, and hdrtoken_string_to_wks() is what the core uses to populate the constants mentioned above:

const char *
TSMimeHdrStringToWKS(const char *str, int length)
{
  if (length < 0) {
    return hdrtoken_string_to_wks(str);
  } else {
    return hdrtoken_string_to_wks(str, length);
  }
}

Re: [E] [PROPOSAL] New API to look up strings against our set of WKS constants

[Walt Karas <wk...@yahooinc.com.INVALID>]

As a background question, is there a trie somewhere that the HTTP header
parser (in the core) uses to return WKS pointers for WKSs in the headers?
I'm having trouble seeing how the speedup of comparing to WKS pointers is
not negated by the time needed to detect WKSs in HTTP headers.

On Sun, May 14, 2023 at 3:32 PM Leif Hedstrom <zw...@apache.org> wrote:

> Hi,
>
> In our code, we have a concept of well-known-strings (WKSs). These are
> used to sort of normalize common header and URL strings, and can be used in
> some cases for optimizing lookups in request header heaps etc. When a
> plugin uses a WKS directly, such as TS_MIME_FIELD_CACHE_CONTROL to look for
> a header, the core will automatically use this optimally, because this
> constant points to the internal WKS (which is MIME_FIELD_CACHE_CONTROL
> internally).
>
> Now, this is all fine, if the plugin knows what headers it’s looking for.
> But for plugins such as header_rewrite, or TxnBox, where an external
> configuration specifies which headers to work with, there’s currently no
> easy way (rather than iterations) to find the WKS corresponding to a user
> specified string. As such, I’m suggesting that we expose the internal WKS
> lookup functionality with a public API:
>
>     tsapi const char *TSMimeHdrStringToWKS(const char *str, int length);
>
>
> The length parameter is “optional” as per normal APIs here, i.e. if you
> specify a negative value for the length, the API will do a strlen() of the
> input str for you.
>
> I’ve made a PR with these changes, which also includes the necessary
> support for the core header_rewrite plugin to use this new API. If there
> are no concerns with the API changes, after discussions here and on the PR,
> I will change the PR from a Draft to normal PR for proper approval.
>
> Thanks,
>
> — Leif
>
>
> https://urldefense.com/v3/__https://github.com/apache/trafficserver/pull/9643__;!!Op6eflyXZCqGR5I!GQiq8sxqnrTtiPDV0N2AFjlwplZCt038HQ3siD6M7grsYzYi3OZzkzoK2NpEXy-6j-8teTmlJ5UH7w$
>
>
> The implementation as such is simple, and hdrtoken_string_to_wks() is what
> the core uses to populate the constants mentioned above:
>
> const char *
> TSMimeHdrStringToWKS(const char *str, int length)
> {
>   if (length < 0) {
>     return hdrtoken_string_to_wks(str);
>   } else {
>     return hdrtoken_string_to_wks(str, length);
>   }
> }