You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@trafficserver.apache.org by "bneradt (via GitHub)" <gi...@apache.org> on 2026/03/23 21:56:09 UTC
[PR] Add log field header fallbacks [trafficserver]
bneradt opened a new pull request, #13018:
URL: https://github.com/apache/trafficserver/pull/13018
Allow custom log formats to fall back between header fields with
`?:` so configs can log a secondary header when the primary one is
absent. This keeps the feature in the existing `%<...>` syntax and
preserves the current distinction between missing and empty values.
Store fallback chains as a single log field so the selected header
still uses normal escaping and its own slice settings. Update the
logging admin guide and extend the logging AuTest coverage for
primary, secondary, and missing header cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add log field header fallbacks [trafficserver]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #13018:
URL: https://github.com/apache/trafficserver/pull/13018#discussion_r2977859758
##########
doc/admin-guide/logging/formatting.en.rst:
##########
@@ -314,8 +314,27 @@ prior to the log field's name, as so::
Format = '%<{User-agent}cqh>'
The above would insert the User Agent string from the client request headers
-into your log entry (or a blank string if no such header was present, or it did
-not contain a value).
+into your log entry (or ``-`` if no such header was present).
+
+Header fields can also be chained with a fallback operator, `?:`, when you want
Review Comment:
The inline markup for the fallback operator uses single backticks (`?:`), which in reStructuredText is interpreted text and can render unexpectedly / trigger Sphinx warnings. Use a literal (double-backtick) inline code block (``?:``) for consistency with the surrounding literals (e.g., ``-``).
```suggestion
Header fields can also be chained with a fallback operator, ``?:``, when you want
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add log field fallbacks [trafficserver]
Posted by "bneradt (via GitHub)" <gi...@apache.org>.
bneradt merged PR #13018:
URL: https://github.com/apache/trafficserver/pull/13018
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add log field header fallbacks [trafficserver]
Posted by "serrislew (via GitHub)" <gi...@apache.org>.
serrislew commented on code in PR #13018:
URL: https://github.com/apache/trafficserver/pull/13018#discussion_r3011284050
##########
doc/admin-guide/logging/formatting.en.rst:
##########
@@ -314,8 +314,27 @@ prior to the log field's name, as so::
Format = '%<{User-agent}cqh>'
The above would insert the User Agent string from the client request headers
-into your log entry (or a blank string if no such header was present, or it did
-not contain a value).
+into your log entry (or ``-`` if no such header was present).
+
+Header fields can also be chained with a fallback operator, `?:`, when you want
+the log to use the first header that exists. For example::
+
+ Format = '%<{x-primary-id}cqh?:{x-secondary-id}cqh>'
+
+|TS| evaluates the candidates from left to right and logs the first header that
+exists. If none of the headers exist, |TS| logs ``-``. A header that exists but
+has an empty value is considered present, so |TS| logs the empty value instead
+of falling back. So in the example above, the value of x-primary-id of the
+client request is logged if it exists, otherwise the value of x-secondary-id is
+logged if it exists, otherwise ``-`` is logged if neither of the headers is
+present.
+
+Note that this fallback chain feature is only supported for header field
Review Comment:
So `%<{x-primary-id}cqh?:Foo>` wouldn't be supported? I guess you can do a workaround to make an internal header with value "Foo" and then use that instead? `%<{x-primary-id}cqh?:{@Foo}>`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add log field header fallbacks [trafficserver]
Posted by "bneradt (via GitHub)" <gi...@apache.org>.
bneradt commented on code in PR #13018:
URL: https://github.com/apache/trafficserver/pull/13018#discussion_r3011443892
##########
doc/admin-guide/logging/formatting.en.rst:
##########
@@ -314,8 +314,27 @@ prior to the log field's name, as so::
Format = '%<{User-agent}cqh>'
The above would insert the User Agent string from the client request headers
-into your log entry (or a blank string if no such header was present, or it did
-not contain a value).
+into your log entry (or ``-`` if no such header was present).
+
+Header fields can also be chained with a fallback operator, `?:`, when you want
+the log to use the first header that exists. For example::
+
+ Format = '%<{x-primary-id}cqh?:{x-secondary-id}cqh>'
+
+|TS| evaluates the candidates from left to right and logs the first header that
+exists. If none of the headers exist, |TS| logs ``-``. A header that exists but
+has an empty value is considered present, so |TS| logs the empty value instead
+of falling back. So in the example above, the value of x-primary-id of the
+client request is logged if it exists, otherwise the value of x-secondary-id is
+logged if it exists, otherwise ``-`` is logged if neither of the headers is
+present.
+
+Note that this fallback chain feature is only supported for header field
Review Comment:
We can add a different hard-coded fallback. Would that be useful though? Currently the final fallback is the default `-`, when would some other hard-coded fallback have value over that? Maybe there is a use case for that which I'm not thinking of.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add log field header fallbacks [trafficserver]
Posted by "bneradt (via GitHub)" <gi...@apache.org>.
bneradt commented on code in PR #13018:
URL: https://github.com/apache/trafficserver/pull/13018#discussion_r3018903295
##########
doc/admin-guide/logging/formatting.en.rst:
##########
@@ -314,8 +314,27 @@ prior to the log field's name, as so::
Format = '%<{User-agent}cqh>'
The above would insert the User Agent string from the client request headers
-into your log entry (or a blank string if no such header was present, or it did
-not contain a value).
+into your log entry (or ``-`` if no such header was present).
+
+Header fields can also be chained with a fallback operator, `?:`, when you want
+the log to use the first header that exists. For example::
+
+ Format = '%<{x-primary-id}cqh?:{x-secondary-id}cqh>'
+
+|TS| evaluates the candidates from left to right and logs the first header that
+exists. If none of the headers exist, |TS| logs ``-``. A header that exists but
+has an empty value is considered present, so |TS| logs the empty value instead
+of falling back. So in the example above, the value of x-primary-id of the
+client request is logged if it exists, otherwise the value of x-secondary-id is
+logged if it exists, otherwise ``-`` is logged if neither of the headers is
+present.
+
+Note that this fallback chain feature is only supported for header field
Review Comment:
Sounds good. Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add log field fallbacks [trafficserver]
Posted by "zwoop (via GitHub)" <gi...@apache.org>.
zwoop commented on code in PR #13018:
URL: https://github.com/apache/trafficserver/pull/13018#discussion_r3030427518
##########
doc/admin-guide/logging/formatting.en.rst:
##########
@@ -314,8 +314,54 @@ prior to the log field's name, as so::
Format = '%<{User-agent}cqh>'
The above would insert the User Agent string from the client request headers
-into your log entry (or a blank string if no such header was present, or it did
-not contain a value).
+into your log entry (or ``-`` if no such header was present).
+
+Header fields can also be chained with a fallback operator, ``??``, when you want
+the log to use the first header that exists among n headers. For example::
+
+ Format = '%<{x-primary-id}cqh??{x-secondary-id}cqh??{x-tertiary-id}cqh>'
+
+|TS| evaluates the candidates from left to right and logs the first header that
+exists. If none of the headers exist, |TS| logs ``-`` by default. A header that
+exists but has an empty value is considered present, so |TS| logs the empty
+value instead of falling back. So in the example above, the value of
+x-primary-id of the client request is logged if it exists, otherwise the value
+of x-secondary-id is logged if it exists, otherwise ``-`` is logged if neither
+of the headers is present.
+
+The final log field in the chain can be a non-header log field whose value
+specifies the final fallback in the chian. This symbol is used only after all
Review Comment:
Typo, chian -> chain
##########
src/proxy/logging/LogFormat.cc:
##########
@@ -475,6 +501,41 @@ LogFormat::parse_symbol_string(const char *symbol_string, LogFieldList *field_li
"')' in %s",
symbol);
}
+ } else if (LogFieldFallback::has_fallback(symbol)) {
+ Dbg(dbg_ctl_log_format, "Field fallback symbol: %s", symbol);
+ std::string parse_error;
+ auto parsed = LogFieldFallback::parse(symbol, parse_error);
+ if (parsed.has_value()) {
+ std::unique_ptr<LogField> fallback_field;
+ bool valid_fallback = true;
+
+ if (parsed->fallback_symbol.has_value()) {
+ std::string fallback_symbol = *parsed->fallback_symbol;
+ std::string field_symbol = fallback_symbol;
Review Comment:
Do we really need two copies here for the string?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add log field header fallbacks [trafficserver]
Posted by "bneradt (via GitHub)" <gi...@apache.org>.
bneradt commented on code in PR #13018:
URL: https://github.com/apache/trafficserver/pull/13018#discussion_r3011432263
##########
src/proxy/logging/LogFormat.cc:
##########
@@ -55,6 +58,94 @@ DbgCtl dbg_ctl_log_format{"log-format"};
DbgCtl dbg_ctl_log_agg{"log-agg"};
DbgCtl dbg_ctl_log_slice{"log-slice"};
+char *
+trim_log_field_token(char *token)
+{
+ while (*token != '\0' && std::isspace(static_cast<unsigned char>(*token))) {
+ ++token;
+ }
+
+ size_t len = std::strlen(token);
+ while (len > 0 && std::isspace(static_cast<unsigned char>(token[len - 1]))) {
+ token[--len] = '\0';
+ }
+
+ return token;
+}
+
+bool
+parse_header_fallback_candidate(char *candidate, LogField::HeaderField &field, const std::string &original_symbol)
+{
+ candidate = trim_log_field_token(candidate);
+ if (*candidate == '\0') {
+ Note("Invalid header fallback field specification: empty candidate in %s", original_symbol.c_str());
+ return false;
+ }
+
+ if (*candidate != '{') {
+ Note("Invalid header fallback field specification: %s is not a header field candidate in %s", candidate,
+ original_symbol.c_str());
+ return false;
+ }
+
+ char *name_end = std::strchr(candidate, '}');
+ if (name_end == nullptr) {
+ Note("Invalid header fallback field specification: no trailing '}' in %s", original_symbol.c_str());
+ return false;
+ }
+
+ *name_end = '\0';
+ char *name = candidate + 1;
+ char *symbol = trim_log_field_token(name_end + 1);
+ LogSlice slice(symbol);
+
+ if (*name == '\0') {
+ Note("Invalid header fallback field specification: empty header name in %s", original_symbol.c_str());
+ return false;
+ }
+
+ auto container = LogField::valid_container_name(symbol);
+ if (!LogField::isHeaderContainer(container)) {
+ Note("Invalid header fallback field specification: %s is not a supported header container in %s", symbol,
+ original_symbol.c_str());
+ return false;
+ }
+
+ field.name = name;
+ field.container = container;
+ field.slice = slice;
+ return true;
+}
+
+LogField *
+parse_header_fallback_field(char *symbol)
+{
+ std::string original_symbol(symbol);
+ std::vector<LogField::HeaderField> header_fields;
+ char *candidate = symbol;
+
+ while (candidate != nullptr) {
Review Comment:
Yes! They can be chained n times:
```
%<{x-primary-id}cqh?:{x-secondary-id}cqh?:{x-tertiary-id}cqh>
```
There is, btw, a dev mailing list email about this if you want to chime in there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add log field header fallbacks [trafficserver]
Posted by "serrislew (via GitHub)" <gi...@apache.org>.
serrislew commented on code in PR #13018:
URL: https://github.com/apache/trafficserver/pull/13018#discussion_r3016769351
##########
doc/admin-guide/logging/formatting.en.rst:
##########
@@ -314,8 +314,27 @@ prior to the log field's name, as so::
Format = '%<{User-agent}cqh>'
The above would insert the User Agent string from the client request headers
-into your log entry (or a blank string if no such header was present, or it did
-not contain a value).
+into your log entry (or ``-`` if no such header was present).
+
+Header fields can also be chained with a fallback operator, `?:`, when you want
+the log to use the first header that exists. For example::
+
+ Format = '%<{x-primary-id}cqh?:{x-secondary-id}cqh>'
+
+|TS| evaluates the candidates from left to right and logs the first header that
+exists. If none of the headers exist, |TS| logs ``-``. A header that exists but
+has an empty value is considered present, so |TS| logs the empty value instead
+of falling back. So in the example above, the value of x-primary-id of the
+client request is logged if it exists, otherwise the value of x-secondary-id is
+logged if it exists, otherwise ``-`` is logged if neither of the headers is
+present.
+
+Note that this fallback chain feature is only supported for header field
Review Comment:
I guess it would just make the default customizable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add log field fallbacks [trafficserver]
Posted by "bneradt (via GitHub)" <gi...@apache.org>.
bneradt commented on PR #13018:
URL: https://github.com/apache/trafficserver/pull/13018#issuecomment-4283288795
10.2.x backport PR: https://github.com/apache/trafficserver/pull/13105
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add log field fallbacks [trafficserver]
Posted by "cmcfarlen (via GitHub)" <gi...@apache.org>.
cmcfarlen commented on PR #13018:
URL: https://github.com/apache/trafficserver/pull/13018#issuecomment-4289904563
Added to milestone 10.2.0 via #13109
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add log field header fallbacks [trafficserver]
Posted by "bneradt (via GitHub)" <gi...@apache.org>.
bneradt commented on PR #13018:
URL: https://github.com/apache/trafficserver/pull/13018#issuecomment-4119037385
[approve ci autest 1]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add log field header fallbacks [trafficserver]
Posted by "serrislew (via GitHub)" <gi...@apache.org>.
serrislew commented on code in PR #13018:
URL: https://github.com/apache/trafficserver/pull/13018#discussion_r3011326934
##########
src/proxy/logging/LogFormat.cc:
##########
@@ -55,6 +58,94 @@ DbgCtl dbg_ctl_log_format{"log-format"};
DbgCtl dbg_ctl_log_agg{"log-agg"};
DbgCtl dbg_ctl_log_slice{"log-slice"};
+char *
+trim_log_field_token(char *token)
+{
+ while (*token != '\0' && std::isspace(static_cast<unsigned char>(*token))) {
+ ++token;
+ }
+
+ size_t len = std::strlen(token);
+ while (len > 0 && std::isspace(static_cast<unsigned char>(token[len - 1]))) {
+ token[--len] = '\0';
+ }
+
+ return token;
+}
+
+bool
+parse_header_fallback_candidate(char *candidate, LogField::HeaderField &field, const std::string &original_symbol)
+{
+ candidate = trim_log_field_token(candidate);
+ if (*candidate == '\0') {
+ Note("Invalid header fallback field specification: empty candidate in %s", original_symbol.c_str());
+ return false;
+ }
+
+ if (*candidate != '{') {
+ Note("Invalid header fallback field specification: %s is not a header field candidate in %s", candidate,
+ original_symbol.c_str());
+ return false;
+ }
+
+ char *name_end = std::strchr(candidate, '}');
+ if (name_end == nullptr) {
+ Note("Invalid header fallback field specification: no trailing '}' in %s", original_symbol.c_str());
+ return false;
+ }
+
+ *name_end = '\0';
+ char *name = candidate + 1;
+ char *symbol = trim_log_field_token(name_end + 1);
+ LogSlice slice(symbol);
+
+ if (*name == '\0') {
+ Note("Invalid header fallback field specification: empty header name in %s", original_symbol.c_str());
+ return false;
+ }
+
+ auto container = LogField::valid_container_name(symbol);
+ if (!LogField::isHeaderContainer(container)) {
+ Note("Invalid header fallback field specification: %s is not a supported header container in %s", symbol,
+ original_symbol.c_str());
+ return false;
+ }
+
+ field.name = name;
+ field.container = container;
+ field.slice = slice;
+ return true;
+}
+
+LogField *
+parse_header_fallback_field(char *symbol)
+{
+ std::string original_symbol(symbol);
+ std::vector<LogField::HeaderField> header_fields;
+ char *candidate = symbol;
+
+ while (candidate != nullptr) {
Review Comment:
Does this mean there can be multiple fallback candidates?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org