You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tooling.apache.org by "sebbASF (via GitHub)" <gi...@apache.org> on 2025/11/21 19:24:40 UTC
[I] Document why f-strings are used for logging calls (tooling-trusted-releases)
sebbASF opened a new issue, #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339
https://github.com/apache/tooling-trusted-releases/blob/1406cebaaf3c075497441179884e40a3e4429949/atr/docs/code-conventions.md?plain=1#L250-L252
Using f-strings for logging calls means that the string will be evaluated even if it is not actually needed.
This is counter-intuitive, so the reason for it should be documented.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3567218695
> This is a moot optimization, at the cost of clarity. IMO, of course.
If you can fully automate it with ruff (that it will even correct it automatically) and have a common agreement on the convention, I think it's worth it IMHO.
It's of course pre-emptive measure and I am also a big fan of "optimise what mkes impact", but in this case It's a metter of automation of convention and sticking to it (especially if that convention is widely followed and teaches good practices.
But - to be honest - it's not something I am going to argue about - first and more important people to decide aboout it are those who actively work on it and who contribute most. I wish we had a rule that you can comment on the conventions after you made significant number of commits to the repo, otherwise this is just "idle talking".
First "walk the walk" in the repo before you "talk the talk" - and yes that also applies to me, that's why i try to make my comments mostly as helpful and reasoning rather than "demanding changes".
I hope it's seen like that, and I hope the issue itself is not "demanding changes" but it's more an invitation do discussion, despite the "should be documented" tone that indicates some universal rule that has no context to understand and "put your hands dirty" first before making demands.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "sebbASF (via GitHub)" <gi...@apache.org>.
sebbASF commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3573126328
Note that ```__repr__``` methods can also be invoked by f-strings; these are generally rather more expensive than ```__str__```.
For example the following useful debugging shortcut uses it:
```log.debug(f"{bigobject=}")```
Also the construct:
```
if logger.isEnabledFor(logging.INFO):
logger.info("...")
```
relies on the same level being used for the check and the log; they can get out of sync.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3569575141
> Note that the policy specifically refers to using f-strings for logging, rather than the 'usually expected' printf style. However it does not explain the rationale for this 'un-usual' decision.
Do you have some data to back up "un-usual" and why the rationale needs an explanation ?
I think we both shown with Dave that there is nothign `usual` there - there are multiple ways logging can be used. I am quite surprised by your insistence of having to explain this particular thing - sounds like focusing on completely meaningless thing that takes a lot of time people to discuss, just to satisfy an obsession on something to be honest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3571222964
Oh. Absolutely. Sorry !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "sebbASF (via GitHub)" <gi...@apache.org>.
sebbASF commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3571133908
> Seems that only you find it unusual, so maybe consider it's your problem not others?
The policy says that log.info usually uses printf, i.e. requiring f-strings is unusual.
Indeed you said:
> Quite agree here. This is a common pitfall to use f-strings for logging in Python.
My ask here is either to document why the policy goes against the usual logging style, or change the policy to allow what you suggested, i.e.
> the "usual" logging style (with args and kwargs)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "sbp (via GitHub)" <gi...@apache.org>.
sbp closed issue #339: Document why f-strings are used for logging calls
URL: https://github.com/apache/tooling-trusted-releases/issues/339
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3566878841
BTW. Very good idea to document code conventions in general, and your convention here is excellent in general - it contains mostly things that are difficult to automatically check and enforce, and is rather short thanks to that.
My rule of thumb when introducing conventions:
* if enforcable by linting/ruff automation - do it this way (and in prek hook with that) - if someone violates them, ruff error usually explains reasoning, links you to examples and often discusses pros/cons in a concise way. This is a fantastic learning for new contributors they can learn by doing (mistakes) and being corrected (automatically) - usually without too much of oversight
* document what can't be enforced only (so that in case of reviews you can point people to those)
Very good job on those @sbp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3569824619
> A policy without a rationale cannot be evaluated and is harder to maintain if circumstances change.
Sure. There are about 20 of those in the convention. Are you going to obsess on every single one of them and spend hours of people to satisfy your definition of "unusual"?
Seems that only you find it unusual, so maybe consider it's your problem not others?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3569597317
And one comment for the future - the more you focus on completely meaningless things and take time of others - the less likely they will be treating you seriously @sebb - I think by responding to such question at all we are showing that we care what you have to say - your comments are often valuable for the Foundation, but the more you obsess on things like that, the more you increase probablity that people will just do `meh ....`, shrug, and move on - even if you raise an important point. This creates a lot of noise and makes it difficult to separate signal from noice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "sebbASF (via GitHub)" <gi...@apache.org>.
sebbASF commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3569761747
As I wrote: the policy specifically says:
> printf style is usually expected, such as in `log.info` calls
yet it insists that f-strings be used instead. To my mind, this is 'un-usual', i.e. not 'usual'.
AIUI, the logging interface is specifically designed to avoid unnecessary construction of strings, so the fact that the policy goes against this is a decision that needs to be explained.
A policy without a rationale cannot be evaluated and is harder to maintain if circumstances change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "gstein (via GitHub)" <gi...@apache.org>.
gstein commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3567191191
f-string formatting is so little overhead that it just doesn't matter whether it will be used or not. They are clearer, and readability improves the maintainability of the code. Obfuscating code for 0.12ms gain is not worthwhile.
Please demonstrate that computation of an f-string has an impact on performance.
Also note that logging calls are relatively rare within the overall app performance. There are not 10,000 logs written per second. That would be unreasonable and useless. So you're getting what? Maybe 10 logs per second? Absolute maximum? And the f-string construction will take about 1 millisecond of that?
This is a moot optimization, at the cost of clarity. IMO, of course.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "sebbASF (via GitHub)" <gi...@apache.org>.
sebbASF commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3567608530
As I wrote: the rationale for not following the usual convention needs to be documented.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "dave2wave (via GitHub)" <gi...@apache.org>.
dave2wave commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3568457257
Also, nothing stops someone from mixing the two methods.
```python
logger.info(f"Processing item {item_id}: %s", extra_detail)
```
Now let's look at:
```python
logger.info(f"Processing item {item_id}: %s", expensive_detail())
```
In this case `expensive_detail()` will execute immediately, whether or not logging is enabled.
So then you ought to do:
```python
if logger.isEnabledFor(logging.INFO):
logger.info(f"Processing item {item_id}: %s", expensive_detail())
```
Or better:
```python
if logger.isEnabledFor(logging.INFO):
logger.info(f"Processing item {item_id}: {expensive_detail()}")
```
We can show some examples easily about why we are constraining logging to more elemental versions and recommend use of `logger.isEnabled` if performance is an issue.
A documentation PR would be welcome.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3571200411
I don't feel like it needs documenting. Not more than any other decision in the convention guide. Your obsession in that particular point is quite, well. obsessive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "sebb (via GitHub)" <gi...@apache.org>.
sebb commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3571218696
hi @potiuk, could you tag @sebbASF rather than me next time please ;)
thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3575452370
> This has been an extreme amount of discussion to save a single millisecond of processing time, during a single wall-clock second of time.
Indeed. Probably we lost more compute time typing it all rather than we could have saved in the entire universe of ATR running
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3569447761
> The more I think about this the more I think that Python's default behavior allows very subtle defects to propagate. Suppose `expensive_detail()` has a side effect that changes application behavior? If it does then your application is dependent on logging configuration and may behave differently in obfuscated ways as you change your environment.
Yep. It does happen. Generally Python is a lot more lax for those than other languages and side-effects are more often than you'd think - but that's the price to pay for more approachable language that is not constraining you a lot and leave a lot of freedom, but also allows you learning from mistakes rather than prevent them upfront. This is what makes it so successful as well :). More freedom -> more creativity -> wider reach -> success.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "dave2wave (via GitHub)" <gi...@apache.org>.
dave2wave commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3568418670
Throughout my career I have often miscounted the number of arguments needed when using an printf pattern (mostly in C and C++). Have never had that issue with f-strings in python. If you are concerned with those few microseconds then another pattern I've seen is to use something like an isDebug branch.
I would much rather create and work on code I can see in front of my face as I am coding than something that gets "prechecked" by a workflow.
I also really do not think it is a good practice to use f-string everywhere except for logging. We all know how quickly a normal string assignment adds a log statement during debugging. It is likely that you will log info your f-string while working out formatting details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3566874150
> Using f-strings for logging calls means that the string will be evaluated even if it is not actually needed.
This is counter-intuitive, so the reason for it should be documented.
Quite agree here. This is a common pitfall to use f-strings for logging in Python. Instead of documenting it here i would rather recommend to the "usual" logging style (with args and kwargs) and enforcing it not (at least not only) by a rule but by ruff rule https://docs.astral.sh/ruff/rules/logging-string-format/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "sbp (via GitHub)" <gi...@apache.org>.
sbp commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3571921474
[CPython issue #90358](https://github.com/python/cpython/issues/90358), which later became a [Python forum thread](https://discuss.python.org/t/safer-logging-methods-for-f-strings-and-new-style-formatting/13802), discusses a security issue with f-strings. The poster points out that `logger.info(f"{untrusted}")` does not incur a template injection attack, but `logger.info(f"{untrusted}", some_dict)` does because the untrusted input may contain `%(example)999999999s` which will add nearly a GB of padding if `example` is a key in `some_dict`.
One mitigation here is to disallow parameters at the interface layer. We can, and should, implement this in [`log.py`](https://github.com/apache/tooling-trusted-releases/blob/main/atr/log.py), which is a useful existing place to attach such constraints. This would enable us to keep using f-string style, but I will suggest a different approach in this comment.
Another problem raised in the linked discussion is that raw templates allow simple grouping of logging statements, as in Sentry, which is lost with f-strings. I don't think that this is a consideration for ATR, but it may be for other internal ASF projects to which our code conventions may eventually apply. I think, however, that if there is to be a grouping facility it should be explicit and it should accept a symbol, probably an enum member, to identify the group.
A consideration which has arisen multiple times in the present issue and in the linked discussion is performance. The focus seems to be mostly on values which are expensive to compute. Python will evaluate argument expressions before sending them to the logger, and in order to defer the computation we either need to add a conditional wrapper, as suggested by @dave2wave, or send a function into the logging mechanism itself, i.e. for continuation passing style. Both solutions have obvious disadvantages.
On the other hand there has been less focus on the performance of the logger string formatting itself. The linked discussion contends that f-strings are generally fastest. What has received less to no attention is that this would depend on the speed of the `__str__` methods of the parameters, but I think in general this should be negligible and we do not need to consider it for ATR or in the ASF broadly.
The original issue as opened by @sebbASF only asks that we document our decision, but our decision is contingent on our stance on all of the issues summarised above. The linked discussion mentions adding delayed f-strings to Python, or using [PEP 501](https://peps.python.org/pep-0501/), as a solution to some of these issues. PEP 501 eventually inspired t-strings, which were added to Python 3.14, released on 7 Oct 2025; we may already be able to migrate to Python 3.14. The PEP which introduced t-strings, [PEP 750](https://peps.python.org/pep-0750/), has an [entire example section about structured logging](https://peps.python.org/pep-0750/#example-structured-logging).
Although t-strings do not inherently support delayed evaluation, they do support a controlled form in that lambda expressions can be used as inline values. You can write, for example, `t"this is an {(lambda: expensive_value())}"`, and then ensure that your t-string evaluator calls any function with zero arguments in its parameter specification only when the string is actually being logged. We can define these evaluators as `logging.Formatter` subclasses to achieve that, as demonstrated in the example logging section in PEP 750.
Using t-strings would also enable a pattern that was suggested by @potiuk not here but [in the linked discussion](https://discuss.python.org/t/safer-logging-methods-for-f-strings-and-new-style-formatting/13802/20):
> Having f-strings, the only way we can actually do that is to get the “content” of secrets we want to mask and search the whole logging message to see if they are not in the logging message. This is SLOW and brittle. If we could use only debugf/infof style of messages, finding out if you are using a secret in one of your parameters, could be WAY faster
We could perform scanning of secrets for masking in the t-string evaluator. Again, this would be efficient because it would only be called when evaluation was strictly necessary for logging. It would not require any change of syntax in the calling code, unlike the `(lambda: ...)` syntax required for deferred evaluation of variables.
In short, t-strings carry the existing advantages of f-strings but with mitigations or solutions for various f-string design problems. The main drawback is that they require migration to Python 3.14. I have opened a separate issue to track the investigation into the feasibility of this migration, #346, but since we will migrate to Python 3.14 eventually whether or not it is immediately feasible, I suggest that we consider further evaluation of t-strings as a solution to the present issue. We can then link to this thread in the documentation to resolve the original issue as stated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "sbp (via GitHub)" <gi...@apache.org>.
sbp commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3576316874
We are not migrating to t-strings for performance reasons. We are migrating to t-strings to enable more accurate scanning of secret values, to allow us to more easily perform different styles of logging simultaneously as described in PEP 750, to give us better control over the interpolation of variables from potentially untrusted sources to improve upon our mitigation, and to regain the ability to perform Sentry style grouping based on the template (though I would prefer to use enums or potentially introspection for that).
Only documenting this was a priority. Implementing t-strings is of a low priority now that I have committed d444d20be776bc9ae5f99631630744f53a5b36e2 which mitigates template padding attacks at the interface level and makes all of our logging strings consistent. We don't even have an issue tracking this, only a TODO item in the source code. Implementing t-strings moreover depends on migrating to Python 3.14, which might not yet be possible. I am not presently interested in performance improvements for ATR except where they directly affect user experience. I expect not to have to use the `lambda:` mechanism in t-strings in any existing ATR code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "dave2wave (via GitHub)" <gi...@apache.org>.
dave2wave commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3568706554
The more I think about this the more I think that Python's default behavior allows very subtle defects to propagate. Suppose `expensive_detail()` has a side effect that changes application behavior? If it does then your application is dependent on logging configuration and may behave differently in obfuscated ways as you change your environment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3568110685
> As I wrote: the rationale for not following the usual convention needs to be documented.
I think it very much depends on "usual" and "needs". Yes It **might** be documented IMHO, but definition of "usual" when it comes to convention very much depends on context - the fact that one person thinks it's usual, does not make it "usual" if you ask me. And it does not **need** to be documented always, but of course it's good when it does so asking nicely "should this be documented maybe ?" is probably better way of raising concerns. Even in ruff that rule (G0001) is not enabled by default because it's not seen as "universally followed". Actually if you look further and want to optimize things as logging library authors (core devs) suggested, you should use `isEnabledFor` - https://docs.python.org/3/howto/logging.html#optimization . And there is the whole discussion in stdlib on adapting the style of your formatting to your application https://docs.python.org/3/howto/logging-cookbook.html#formatting-styles - so I am not sure if there is a "one universal recipe" for logging,
and apparently the tooling team made and documented their choice - but whether it "needs to be explained because in this case one person has a different preference that they think is standard" is likely not something I would "demand". I would never write "should" in the code base that I never touched. That seems a bit pushy and demanding, expressing patronising approach in general.
Maybe a better way (again) is to start with asking question and seeing what original author thinks, and raising your concerns rather than believe that "Everyone around me have the same judgment as I do". We all have biases, some of us have more need for having things "perfect" and even exhibit sometimes kind of obsessive desire to fix things, and it's fine as long as they are the ones who care about this, rather than trying to make others think like them. That's all I wanted to say. It's not too collaborative approach when first interaction you have with something is to tell them what THEY should do because YOU know it's the only right way. Or at least that's the trait I see in some of the comments you have - lack of inquisiiiveness and curiosity, but simplly "knowing" that your way is the only way.
I am not questioning your issue @sebbASF - in case you did not notice I quite support the merit, but I question the way it has been raised. Could be more ... I don't know ... human way? rather than robotic? Or more collaborative rather than demanding and patronising ?
I'd suggest you to work on this communication approach, I saw it works miracles in terms of achieving your goals. Pushy and demanding approach often raises unnecessary bad emotions and feelings. Treat is as food for thoughts, maybe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "sebbASF (via GitHub)" <gi...@apache.org>.
sebbASF commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3569553679
I think any language that allows methods to change shared state and has method calls that can be disabled - e.g. assertions, as well as logging - can have such behaviour. Thorough testing needs to exercise all pathways, including logging and assertions.
However, this is getting way off-topic.
Note that the policy specifically refers to using f-strings for logging, rather than the 'usually expected' printf style. However it does not explain the rationale for this 'un-usual' decision.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3569828332
I'd propose to just close this one. It's becoming a bit ridiculous to discuss it :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org
Re: [I] Document why f-strings are used for logging calls (tooling-trusted-releases)
Posted by "gstein (via GitHub)" <gi...@apache.org>.
gstein commented on issue #339:
URL: https://github.com/apache/tooling-trusted-releases/issues/339#issuecomment-3574888257
This has been an extreme amount of discussion to save a single millisecond of processing time, during a single wall-clock second of time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tooling.apache.org
For additional commands, e-mail: dev-help@tooling.apache.org