You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@logging.apache.org by "rgoers (via GitHub)" <gi...@apache.org> on 2024/03/27 21:50:02 UTC
[PR] Scoped context (logging-log4j2)
rgoers opened a new pull request, #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419
This adds support for a ScopedContext and a ResourceLogger as requested in https://github.com/apache/logging-log4j-kotlin/issues/71 and similar to https://github.com/apache/logging-log4j2/discussions/2214
Git has a funny way of mangling old PR branches so I deleted the old one and recreated it. Unfortunately, that causes comments on the previous PR to be lost here. This has several changes to ScopedContext from the previous PR.
1. ScopedContext.INITIAL_CONTEXT is no longer present.
2. ScopedContext.current() is no longer necessary.
3. Support for running child threads and propagating the ThreadContext and ScopedContext was added.
Example usage for wrapping a simple code block.
```
ScopedContext.where("id", UUID.randomUUID())
.where("ipAddress", request.getRemoteAddr())
.where("loginId", session.getAttribute("loginId"))
.where("hostName", request.getServerName())
.run(new Worker());
private class Worker implements Runnable {
private static final Logger LOGGER = LogManager.getLogger(Worker.class);
public void run() {
LOGGER.debug("Performing work");
String loginId = ScopedContext.get("loginId");
}
}
```
Creating Threads:
```
BlockingQueue<Runnable> workQueue = new ArrayBlockingQueue<>(5);
ExecutorService executorService = new ThreadPoolExecutor(1, 2, 30, TimeUnit.SECONDS, workQueue);
Future<?> future = ScopedContext.where("id", UUID.randomUUID())
.where("ipAddress", request.getRemoteAddr())
.where("loginId", session.getAttribute("loginId"))
.where("hostName", request.getServerName())
.run(executorService, new Worker());
try {
future.get();
} catch (ExecutionException ex) {
logger.warn("Exception in worker thread: {}", ex.getMessage());
}
```
Note that this syntax supports virtual threads.
ResourceLogger can be used to include resource data in all log events coming from a specific Logger.
```
private class User {
private final String loginId;
private final String role;
private int loginAttempts;
private final ResourceLogger logger;
public User(final String loginId, final String role) {
this.loginId = loginId;
this.role = role;
logger = ResourceLogger.newBuilder()
.withClass(this.getClass())
.withSupplier(new UserSupplier())
.build();
}
public void login() throws Exception {
++loginAttempts;
try {
authenticator.authenticate(loginId);
logger.info("Login succeeded");
} catch (Exception ex) {
logger.warn("Failed login");
throw ex;
}
}
private class UserSupplier implements Supplier<Map<String, String>> {
public Map<String, String> get() {
Map<String, String> map = new HashMap<>();
map.put("LoginId", loginId);
map.put("Role", role);
map.put("Count", Integer.toString(loginAttempts));
return map;
}
}
}
```
All events in a ResourceLogger use ParameterizedMapMessage. When logging it behaves like a ParameterizedMessage in that formatMessage only returns the message directly being logged. However, it is a MapMessage so all the attributes can be accessed via the normal Layout, Lookup, and Filter capabilities we have added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2024398297
@rocketraman I did look into that and what you are asking for just can't be implemented in a reasonable way without significant mods to the API and will likely impact performance. The problem is that Loggers are singletons and the code where the LogEvent is constructed and the ContextMap is populated is pretty far down the stack. We defer creating LogEvents to avoid the performance overhead when they aren't going to be used. Messages, on the other hand, are constructed very early as they are fairly light-weight. So doing that in a per-instance Logger that wraps the singleton was fairly easy to do.
You will need to point me to some docs on Google Cloud Logging, however at my employer we are running a cloud infrastructure and log to an ELK stack and making this work the way you would like would be fairly easy, although it might be nice to enhance JsonTemplateLayout if it makes sense. To give you an idea, in my infrastructure whe have JsonTemplateLayout configured iwth
```
{
"@timestamp": {
"$resolver": "timestamp",
"pattern": {
"format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
"timeZone": "UTC"
}
},
"ecs.version": "1.11.0",
"log.level": {
"$resolver": "level",
"field": "name"
},
"application": "${lower:${spring:spring.application.name}}",
"short_message": {
"$resolver": "message",
"stringified": true
},
"message": {
"$resolver": "pattern",
"pattern": "[%t] %X{requestId, sessionId, loginId, ipAddress} %C{1.}.%M:%L - %m%n"
},
"process.thread.name": {
"$resolver": "thread",
"field": "name"
},
"log.logger": {
"$resolver": "logger",
"field": "name"
},
"event.action": {
"$resolver": "marker",
"field": "name"
},
"event.data": {
"$resolver": "map",
"stringified": true
},
"labels": {
"$resolver": "mdc",
"flatten": true,
"stringified": true,
"pattern": "(?!(?i:token)).+"
},
"tags": {
"$resolver": "ndc"
},
"error.type": {
"$resolver": "exception",
"field": "className"
},
"error.message": {
"$resolver": "exception",
"field": "message"
},
"error.stack_trace": {
"$resolver": "exception",
"field": "stackTrace",
"stackTrace": {
"stringified": true
}
}
}
```
Note that with this configuration the "message" key will look exactly like it would in a log file. However all the context fields, except for the user's token, are included as distinct fields while all the message map items are under the event.data tag. However, these could have also been flattened into distinct fields as the context data is and you wouldn't know the difference when searching. If you have other requirements for how the data should be formatted I am sure JsonTemplateLayout could be enhanced to support it, if it can't do it already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2028027525
> OK. Since you are -1 on ScopedContext I will close this PR.
I would be OK with an implementation based on `ThreadContextMap` that does not introduce new `ThreadLocal`s.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544596128
##########
log4j-api/src/main/java/org/apache/logging/log4j/ScopedContext.java:
##########
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.internal.ScopedContextAnchor;
+import org.apache.logging.log4j.status.StatusLogger;
+
+/**
+ * Context that can be used for data to be logged in a block of code.
+ *
+ * While this is influenced by ScopedValues from Java 21 it does not share the same API. While it can perform a
+ * similar function as a set of ScopedValues it is really meant to allow a block of code to include a set of keys and
+ * values in all the log events within that block. The underlying implementation must provide support for
+ * logging the ScopedContext for that to happen.
+ *
+ * The ScopedContext will not be bound to the current thread until either a run or call method is invoked. The
+ * contexts are nested so creating and running or calling via a second ScopedContext will result in the first
+ * ScopedContext being hidden until the call is returned. Thus the values from the first ScopedContext need to
+ * be added to the second to be included.
+ *
+ * The ScopedContext can be passed to child threads by including the ExecutorService to be used to manage the
+ * run or call methods. The caller should interact with the ExecutorService as if they were submitting their
+ * run or call methods directly to it. The ScopedContext performs no error handling other than to ensure the
+ * ThreadContext and ScopedContext are cleaned up from the executed Thread.
+ *
+ * @since 2.24.0
+ */
+public class ScopedContext {
Review Comment:
@ppkarwasz I have no plans to merge this PR without ScopedContext. I am quite sure I will use it in my environment far more frequently than I will ever use ResourceLogger. I can easily envision Service 1 creating a ScopedContext that gets propogated to Service 2, which then adds its own ScopedContext data. This would be extremely valuable in the logs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1545235864
##########
log4j-api/src/main/java/org/apache/logging/log4j/ScopedContext.java:
##########
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.internal.ScopedContextAnchor;
+import org.apache.logging.log4j.status.StatusLogger;
+
+/**
+ * Context that can be used for data to be logged in a block of code.
+ *
+ * While this is influenced by ScopedValues from Java 21 it does not share the same API. While it can perform a
+ * similar function as a set of ScopedValues it is really meant to allow a block of code to include a set of keys and
+ * values in all the log events within that block. The underlying implementation must provide support for
+ * logging the ScopedContext for that to happen.
+ *
+ * The ScopedContext will not be bound to the current thread until either a run or call method is invoked. The
+ * contexts are nested so creating and running or calling via a second ScopedContext will result in the first
+ * ScopedContext being hidden until the call is returned. Thus the values from the first ScopedContext need to
+ * be added to the second to be included.
+ *
+ * The ScopedContext can be passed to child threads by including the ExecutorService to be used to manage the
+ * run or call methods. The caller should interact with the ExecutorService as if they were submitting their
+ * run or call methods directly to it. The ScopedContext performs no error handling other than to ensure the
+ * ThreadContext and ScopedContext are cleaned up from the executed Thread.
+ *
+ * @since 2.24.0
+ */
+public class ScopedContext {
Review Comment:
> I don't believe Log4j should provide the integrations to these technologies at least as part of core or the api.
On this we agree, but do we need to force those projects to write yet another integration?
> The ScopedContext can be propogated anywhere as well. We have a utility class at work that propagates the ThreadContext via RestTemplate, Kafka, and Amqp. Making it support ScopedContext is easy as it just needs to do what ScopedContextDataProvider does to get hold of the context entries.
Every context propagation is easy, but it is annoying. At my previous job we had to propagate Spring's request scope, our own scope, Vaadin's context and so on. I started resenting colleagues that used static context accessors, since their code was usually tested on the servlet working thread and we only noticed failures (NPEs), when accessed from an asynchronous thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers closed pull request #2419: Scoped context
URL: https://github.com/apache/logging-log4j2/pull/2419
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544600596
##########
log4j-api/src/main/java/org/apache/logging/log4j/message/ParameterizedMapMessage.java:
##########
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j.message;
+
+import java.util.Map;
+
+/**
+ * Class Description goes here.
+ */
+public class ParameterizedMapMessage extends StringMapMessage {
+
+ private static final long serialVersionUID = -7724723101786525409L;
+ private final Message baseMessage;
+
+ ParameterizedMapMessage(Message baseMessage, Map<String, String> resourceMap) {
+ super(resourceMap);
+ this.baseMessage = baseMessage;
+ }
+
+ @Override
+ public String getFormattedMessage() {
+ return baseMessage.getFormattedMessage();
+ }
Review Comment:
@ppkarwasz
I would suggest that Open Telemetry would be better served by adopting ParameterizedMapMessage than treating the message key as special. I admit, I have done the same thing in the past which is what motivated me to create this Message class as I have always found that irritating. You are correct the getFormat() should be overridden but it should be baseMessage.getFormat().
The purpose of ParameterizedMapMessage, as I have stated several times, is to have it format %m EXACTLY how ParameterizedMessage would but still have it be a MapMessage so other Lookups, Filters, and Layouts can extract the structured data from the message. That is it. No fancy tricks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2027684082
> @rocketraman I have modified the PR so that ResourceLogger uses ScopedContext and behaves the way you requested. @ppkarwasz I have no idea why the PR system lets me reply to some messages and not others.
Some of my messages are part of a "review", other are simple comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1545166863
##########
log4j-api/src/main/java/org/apache/logging/log4j/message/ParameterizedMapMessage.java:
##########
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j.message;
+
+import java.util.Map;
+
+/**
+ * Class Description goes here.
+ */
+public class ParameterizedMapMessage extends StringMapMessage {
+
+ private static final long serialVersionUID = -7724723101786525409L;
+ private final Message baseMessage;
+
+ ParameterizedMapMessage(Message baseMessage, Map<String, String> resourceMap) {
+ super(resourceMap);
+ this.baseMessage = baseMessage;
+ }
+
+ @Override
+ public String getFormattedMessage() {
+ return baseMessage.getFormattedMessage();
+ }
Review Comment:
@ppkarwasz
> Due to a "happens before" relationship they didn't have the chance to do it. 😁
They will be able to after this is merged.
> However I would like to have a well documented Java interface that ParameterizedMapMessage will implement. In time we can switch all the instanceof MapMessage to that class
Umm. No. There are multiple reasons to use a MapMessage of various kinds. Consider StructuredDataMessage. That supports RFC5414 which specifies how things are supposed to behave. I use MapMessages to create data for dashboards in Kibana. In that case you don't want or need the ParameterizedMessage support.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2028019689
OK. Since you are -1 on ScopedContext I will close this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544756837
##########
log4j-api/src/main/java/org/apache/logging/log4j/internal/ScopedContextAnchor.java:
##########
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j.internal;
+
+import java.util.ArrayDeque;
+import java.util.Deque;
+import java.util.Optional;
+import org.apache.logging.log4j.ScopedContext;
+
+/**
+ * Anchor for the ScopedContext. This class is private and not for public consumption.
+ */
+public class ScopedContextAnchor {
+ private static final ThreadLocal<Deque<ScopedContext.Instance>> scopedContext = new ThreadLocal<>();
Review Comment:
I explicitly don't want the ThreadLocal populated if nothing has been pushed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rocketraman (via GitHub)" <gi...@apache.org>.
rocketraman commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2024335371
> All events in a ResourceLogger use ParameterizedMapMessage. When logging it behaves like a ParameterizedMessage in that formatMessage only returns the message directly being logged. However, it is a MapMessage so all the attributes can be accessed via the normal Layout, Lookup, and Filter capabilities we have added.
In structured logging use cases (e.g. to Google Cloud Logging), the resource attributes should go to the "context" field. With this approach, the attributes will go to the "message" field.
It won't be possible to easily find all the log messages when the same attributes are used with `ScopedContext` in some places (because the resource is relevant to a Thread) and with `ResourceLogger` in others (because the resource logs internally).
I think we need an approach that will allow sending resource attributes to "context" so the two approaches are consistent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2027966002
> @ppkarwasz You have flagged this as "Changes Requested". It isn't really clear what you have requested. If it is to remove ScopedContext that isn't going to happen so we will have to work that out. If it is something else please let me know.
Yes, in general I am +10 on `ParameterizedMapMessage` and -1 on `ScopedContext`. What I would like to change in this PR mostly falls into two categories:
- size down the impact on the public API, now that we know that we can not make breaking changes in the API for a very long time,
- decouple the contract of the new API from its implementation, by using **well-documented** interfaces.
In particular:
1. Is there something wrong with adding new methods to `CloseableThreadContext` instead of creating a new `ScopedContext`?
2. You introduce another `ThreadLocal` containing context data: all the libraries that read context data for metrics or traces will need to be updated. Could we reuse `ThreadContextMap`? `log4j-to-slf4j` will also ignore the scoped context.
3. Multiple `ContextDataProvider`s currently don't work very well: you will not be able to use your scoped data in a `${ctx:…}` lookup (see #2331),
4. Can we replace the static factory methods in `ResourceLogger` with new instance methods of `Logger`?
5. `ResourceLoggerBuilder` could be an interface: in `log4j-to-slf4j` I might want to implement it differently.
6. `ResourceLogger` should not appear in the signature of `ResourceLoggerBuilder#build()`: from a user perspective, it is nothing more than a `Logger`.
7. `ParameterizedMapMessage` should have a matching interface:
* we might want to implement a reusable version of that interface,
* we want to hide from users all the irrelevant `MapMessage` methods: `MapMessage` implements `MultiFormatStringBuilderFormattable` and is mutable,
* the interface should **precisely** describe what is the effect a user will get if he uses it. The Javadoc of [`Message#getFormat`](https://logging.apache.org/log4j/2.x/javadoc/log4j-api/org/apache/logging/log4j/message/Message.html#getFormat()) is a counter-example of a precise description. We might as well have written "Undefined behavior".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "vy (via GitHub)" <gi...@apache.org>.
vy commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2034076685
**Disclaimer:** I did not completely read all the earlier discussions in this PR.
### On using `ThreadLocal`s
I am against implementing `ScopedContext` using `ThreadLocal`s.
For one, Java introduced a specialized type for this: `ScopedValue`. It is partially implemented in Java (in `Thread.java` and `ScopedValue.java`) and partially in the JVM. See the following snippet from `Thread.java` of Java 22 sources:
```
// ScopedValue support:
@IntrinsicCandidate
static native Object[] scopedValueCache();
@IntrinsicCandidate
static native void setScopedValueCache(Object[] cache);
```
Second, as @rgoers is aware of, `ScopedContext` doesn't support thread jumps out of the box, hence he introduced `ExecutorService`-specialized operators, e.g., `R ScopedContext.callWhere(String,Object,ExecutorService,Callable<R>)`. I can think of several examples where this won't be applicable:
```
ScopedContext.runWhere(food, "awesome", () -> {
aClassRunningCommandsInItsOwnExecutor.cook(() -> {
log.info("Food will not show up `awesome`.");
});
});
```
I find it unacceptable that we roll out our custom semi-working `ScopedValue` using `ThreadLocal`s because we want to support Java 8. This is exactly the reason we ended up having a `Supplier` in our code base.
### On the new `Renderable` interface
I am against adding yet another `String`-to-`Object` interface, I already [stated this in the earlier PR](https://github.com/apache/logging-log4j2/pull/2385/files#r1531746011):
> Please let's not introduce a yet another `Object`-to-`String` helper interface ecosystem. I think `StringBuilderFormattable` fits the bill here.
@rgoers, responded as follows:
> How? There are no StringBuilders at all in ScopedContext. It is used to convert an object into its String representation so it can be included in the LogEvent's ContextMap as a single String item. While the Object's render method certainly might use a StringBuilder internally the code storing the data into the Map certainly doesn't want one.
Layout rendering the MDC/`ScopedContext`/etc. can take care of this. For instance, `JsonTemplateLayout` already recursively walks over `Map`s and checks if the entry value supports `StringBuilderFormattable`. Hence, no need to introduce a new interface. Users who want to customize the output of their MDC/`ScopedContext`/etc. values just need to extend from `StringBuilderFormattable` and it will work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "jvz (via GitHub)" <gi...@apache.org>.
jvz commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544697051
##########
log4j-api/src/main/java/org/apache/logging/log4j/ScopedContext.java:
##########
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.internal.ScopedContextAnchor;
+import org.apache.logging.log4j.status.StatusLogger;
+
+/**
+ * Context that can be used for data to be logged in a block of code.
+ *
+ * While this is influenced by ScopedValues from Java 21 it does not share the same API. While it can perform a
+ * similar function as a set of ScopedValues it is really meant to allow a block of code to include a set of keys and
+ * values in all the log events within that block. The underlying implementation must provide support for
+ * logging the ScopedContext for that to happen.
+ *
+ * The ScopedContext will not be bound to the current thread until either a run or call method is invoked. The
+ * contexts are nested so creating and running or calling via a second ScopedContext will result in the first
+ * ScopedContext being hidden until the call is returned. Thus the values from the first ScopedContext need to
+ * be added to the second to be included.
+ *
+ * The ScopedContext can be passed to child threads by including the ExecutorService to be used to manage the
+ * run or call methods. The caller should interact with the ExecutorService as if they were submitting their
+ * run or call methods directly to it. The ScopedContext performs no error handling other than to ensure the
+ * ThreadContext and ScopedContext are cleaned up from the executed Thread.
+ *
+ * @since 2.24.0
+ */
+public class ScopedContext {
Review Comment:
I agree with Ralph here. We're at the bottom of the stack. While we can bridge our APIs over to other libraries, those don't belong in our API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2028028153
@ppkarwasz
As I said in my edited comment above, that can't be done without exposing ThreadContextMap to breakage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544808343
##########
log4j-api/src/main/java/org/apache/logging/log4j/ScopedContext.java:
##########
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.internal.ScopedContextAnchor;
+import org.apache.logging.log4j.status.StatusLogger;
+
+/**
+ * Context that can be used for data to be logged in a block of code.
+ *
+ * While this is influenced by ScopedValues from Java 21 it does not share the same API. While it can perform a
+ * similar function as a set of ScopedValues it is really meant to allow a block of code to include a set of keys and
+ * values in all the log events within that block. The underlying implementation must provide support for
+ * logging the ScopedContext for that to happen.
+ *
+ * The ScopedContext will not be bound to the current thread until either a run or call method is invoked. The
+ * contexts are nested so creating and running or calling via a second ScopedContext will result in the first
+ * ScopedContext being hidden until the call is returned. Thus the values from the first ScopedContext need to
+ * be added to the second to be included.
+ *
+ * The ScopedContext can be passed to child threads by including the ExecutorService to be used to manage the
+ * run or call methods. The caller should interact with the ExecutorService as if they were submitting their
+ * run or call methods directly to it. The ScopedContext performs no error handling other than to ensure the
+ * ThreadContext and ScopedContext are cleaned up from the executed Thread.
+ *
+ * @since 2.24.0
+ */
+public class ScopedContext {
Review Comment:
@rgoers,
> As you know, I am also against the API having dependencies on anything but the JDK. I am not inclined to have the API be dependent on someone elses context propagation API.
I am not suggesting to add an external context propagation API dependency to `log4j-api`, but to delegate context propagation to higher levels of the stack. For example let's look at the first example of ["Observability with Spring Boot 3"](https://spring.io/blog/2022/10/12/observability-with-spring-boot-3#how-does-micrometer-observation-work):
```java
// Create an Observation and observe your code!
Observation.createNotStarted("user.name", registry)
.contextualName("getting-user-name")
.lowCardinalityKeyValue("userType", "userType1") // let's assume that you can have 3 user types
.highCardinalityKeyValue("userId", "1234") // let's assume that this is an arbitrary number
.observe(() -> log.info("Hello")); // this is a shortcut for starting an observation, opening a scope,
```
The `userId` value will be present in the spans and metrics, but will it be present in the context data of the log event? If it is not present, how can I add it? I don't think that wrapping the lambda using `ScopedContext` is a practical solution.
> Looking at the `Context` object I see that it mostly adds different kinds of functions that can be called. I am not convinced that is necessary but if it is they can always be added later.
The interesting part for me is not the amount of wrapper methods it provides, but the promise that it will propagate everywhere using any kind of technology (Netty, HTTP, Spring Reactor, …). They already integrate with Log4j Core (see [`log4j-context-data-2.17`](https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/log4j/log4j-context-data/log4j-context-data-2.17/library-autoconfigure)), but even if they didn't, we could create a Maven module that integrates OpenTelemetry with Log4j API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2028898990
> 1. In our discussion yesterday we agreed `ThreadContext` (and presumably the `ThreadContextMap`s) should be deprecated. That would include `ObjectThreadContextMap`.
I would love that, but after looking at our SLF4J integrations I am not sure any more we can do that.
> 3. Even if I could find a `ThreadLocal` to reuse I still don't understand why you are so reticent about adding a new one. After all, if ContextScope isn't being used then the ThreadLocal won't be instantiated. Likewise, one would imagine that if ContextScope is being used then the ThreadContext won't be.
I am reticent about adding a new thread local, because propagating the `ThreadContextMap` is well supported by many tools, while the new `ThreadLocal` in `ScopedContext` will require many new integrations.
As far as I remember JBoss LogManager also has an MDC that can have object values. In this case reusing `ThreadContextMap` will allow you to have an out-of-the-box integration.
> 4. I am not sure what it would mean to "Take a snapshot" of a ScopedContext or why one would want to do it.
By "taking a snapshot" I mean retrieving the `Foo` from the `ThreadLocal<Foo>` if `Foo` is immutable or make a copy of `Foo` otherwise).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544606640
##########
log4j-api/src/main/java/org/apache/logging/log4j/ResourceLogger.java:
##########
@@ -0,0 +1,276 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.Map;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.message.Message;
+import org.apache.logging.log4j.message.ParameterizedMapMessageFactory;
+import org.apache.logging.log4j.spi.AbstractLogger;
+import org.apache.logging.log4j.spi.ExtendedLogger;
+import org.apache.logging.log4j.status.StatusLogger;
+import org.apache.logging.log4j.util.StackLocatorUtil;
+import org.apache.logging.log4j.util.Strings;
+
+/**
+ * Logger for resources. Formats all events using the ParameterizedMapMessageFactory along with the provided
+ * Supplier. The Supplier provides resource attributes that should be included in all log events generated
+ * from the current resource. Note that since the Supplier is called for every LogEvent being generated
+ * the values returned may change as necessary. Care should be taken to make the Supplier as efficient as
+ * possible to avoid performance issues.
+ *
+ * Unlike regular Loggers ResourceLoggers CANNOT be declared to be static. A ResourceLogger
+ * must be declared as a class member that will be garbage collected along with the instance of the resource.
+ */
+public final class ResourceLogger extends AbstractLogger {
Review Comment:
Yes. Thanks. Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2027253194
@rgoers,
Could you review my #2381 PR that introduces another `MessageFactory` to deal with Logback's throwable semantics? Maybe we should really consider creating a `log4j-message` artifact with a log of message-related features and leave in `log4j-api` just the strict necessary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544596128
##########
log4j-api/src/main/java/org/apache/logging/log4j/ScopedContext.java:
##########
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.internal.ScopedContextAnchor;
+import org.apache.logging.log4j.status.StatusLogger;
+
+/**
+ * Context that can be used for data to be logged in a block of code.
+ *
+ * While this is influenced by ScopedValues from Java 21 it does not share the same API. While it can perform a
+ * similar function as a set of ScopedValues it is really meant to allow a block of code to include a set of keys and
+ * values in all the log events within that block. The underlying implementation must provide support for
+ * logging the ScopedContext for that to happen.
+ *
+ * The ScopedContext will not be bound to the current thread until either a run or call method is invoked. The
+ * contexts are nested so creating and running or calling via a second ScopedContext will result in the first
+ * ScopedContext being hidden until the call is returned. Thus the values from the first ScopedContext need to
+ * be added to the second to be included.
+ *
+ * The ScopedContext can be passed to child threads by including the ExecutorService to be used to manage the
+ * run or call methods. The caller should interact with the ExecutorService as if they were submitting their
+ * run or call methods directly to it. The ScopedContext performs no error handling other than to ensure the
+ * ThreadContext and ScopedContext are cleaned up from the executed Thread.
+ *
+ * @since 2.24.0
+ */
+public class ScopedContext {
Review Comment:
@ppkarwasz I have no plans to merge this PR without ScopedContext. I am quite sure I will use it in my environment far more frequently than I will ever use ResourceLogger. I can easily envision Service 1 creating a ScopedContext that gets propogated to Service 2, which then adds its own ScopedContext data. This would be extremely valuable in the logs.
As you know, I am also against the API having dependencies on anything but the JDK. I am not inclined to have the API be dependent on someone elses context propagation API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1545166863
##########
log4j-api/src/main/java/org/apache/logging/log4j/message/ParameterizedMapMessage.java:
##########
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j.message;
+
+import java.util.Map;
+
+/**
+ * Class Description goes here.
+ */
+public class ParameterizedMapMessage extends StringMapMessage {
+
+ private static final long serialVersionUID = -7724723101786525409L;
+ private final Message baseMessage;
+
+ ParameterizedMapMessage(Message baseMessage, Map<String, String> resourceMap) {
+ super(resourceMap);
+ this.baseMessage = baseMessage;
+ }
+
+ @Override
+ public String getFormattedMessage() {
+ return baseMessage.getFormattedMessage();
+ }
Review Comment:
@ppkarwasz
> Due to a "happens before" relationship they didn't have the chance to do it. 😁
They will be able to after this is merged.
> However I would like to have a well documented Java interface that ParameterizedMapMessage will implement. In time we can switch all the instanceof MapMessage to that class
Umm. No. There are multiple reasons to use a MapMessage of various kinds. Consider StructuredDataMessage. That supports RFC5414 which specifies how things are supposed to behave. I use MapMessages to create data for dashboards in Kibana. In that case you don't want or need the ParameterizedMessage support.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2028831329
@ppkarwasz
I really don't see the point of using ObjectThreadContextMap.
1. In out discussion we agreed ThreadContext (and presumably the ThreadContextMaps) should be deprecated. That would include ObjectThreadContextMap.
2. The only implementations of that interface declare:
```
protected final ThreadLocal<StringMap> localMap;
```
which isn't particularly helpful.
3. Even if I could find a ThreadLocal to reuse I still don't understand why you are so reticent about adding a new one. After all, if ContextScope isn't being used then the ThreadLocal won't be instantiated. Likewise, one would imagine that if ContextScope is being used then the ThreadContext won't be.
4. I am not sure what it would mean to "Take a snapshot" of a ScopedContext or why one would want to do it. I suppose I could imagine saving the topmost map as JSON for transport, but since it can contain objects it would almost surely have to serialize only the redered String variation of the objects in the map as it would be impossible to deserialized the map without know ing what is in it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2028922882
OK. I think I understand your concerns WRT tooling and I will do what I can to address that. FWIW, I don't believe using the ThreadContext's ThreadLocal would help in that at all.
As for the snampshot, again I don't have any idea why you would want the whole stack of Maps for a ScopedContext. Usually you would only want the current map.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2028035321
@rgoers,
In your edited comment you make a lot of good points. Let us discuss in a video call after the holidays.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1545237304
##########
log4j-api/src/main/java/org/apache/logging/log4j/message/ParameterizedMapMessage.java:
##########
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j.message;
+
+import java.util.Map;
+
+/**
+ * Class Description goes here.
+ */
+public class ParameterizedMapMessage extends StringMapMessage {
+
+ private static final long serialVersionUID = -7724723101786525409L;
+ private final Message baseMessage;
+
+ ParameterizedMapMessage(Message baseMessage, Map<String, String> resourceMap) {
+ super(resourceMap);
+ this.baseMessage = baseMessage;
+ }
+
+ @Override
+ public String getFormattedMessage() {
+ return baseMessage.getFormattedMessage();
+ }
Review Comment:
`getFormattedMessage` can always return the empty string, if there is not message attached to the map.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544596128
##########
log4j-api/src/main/java/org/apache/logging/log4j/ScopedContext.java:
##########
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.internal.ScopedContextAnchor;
+import org.apache.logging.log4j.status.StatusLogger;
+
+/**
+ * Context that can be used for data to be logged in a block of code.
+ *
+ * While this is influenced by ScopedValues from Java 21 it does not share the same API. While it can perform a
+ * similar function as a set of ScopedValues it is really meant to allow a block of code to include a set of keys and
+ * values in all the log events within that block. The underlying implementation must provide support for
+ * logging the ScopedContext for that to happen.
+ *
+ * The ScopedContext will not be bound to the current thread until either a run or call method is invoked. The
+ * contexts are nested so creating and running or calling via a second ScopedContext will result in the first
+ * ScopedContext being hidden until the call is returned. Thus the values from the first ScopedContext need to
+ * be added to the second to be included.
+ *
+ * The ScopedContext can be passed to child threads by including the ExecutorService to be used to manage the
+ * run or call methods. The caller should interact with the ExecutorService as if they were submitting their
+ * run or call methods directly to it. The ScopedContext performs no error handling other than to ensure the
+ * ThreadContext and ScopedContext are cleaned up from the executed Thread.
+ *
+ * @since 2.24.0
+ */
+public class ScopedContext {
Review Comment:
@ppkarwasz I have no plans to merge this PR without ScopedContext. I am quite sure I will use it in my environment far more frequently than I will ever use ResourceLogger. I can easily envision Service 1 creating a ScopedContext that gets propogated to Service 2, which then adds its own ScopedContext data. This would be extremely valuable in the logs.
As you know, I am also against the API having dependencies on anything but the JDK. I am not inclined to have the API be dependent on someone elses context propagation API. Looking at the Context object I see that it mostly adds different kinds of functions that can be called. I am not convinced that is necessary but if it is they can always be added later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "jvz (via GitHub)" <gi...@apache.org>.
jvz commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544699049
##########
log4j-api-test/src/test/java/org/apache/logging/log4j/ResourceLoggerTest.java:
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.hasSize;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.test.TestLogger;
+import org.apache.logging.log4j.test.TestLoggerContextFactory;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+
+/**
+ * Class Description goes here.
+ */
+public class ResourceLoggerTest {
+ @BeforeAll
+ public static void beforeAll() {
+ System.setProperty("log4j2.loggerContextFactory", TestLoggerContextFactory.class.getName());
Review Comment:
Nit: good idea to clear this property afterwards.
##########
log4j-api/src/main/java/org/apache/logging/log4j/internal/ScopedContextAnchor.java:
##########
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j.internal;
+
+import java.util.ArrayDeque;
+import java.util.Deque;
+import java.util.Optional;
+import org.apache.logging.log4j.ScopedContext;
+
+/**
+ * Anchor for the ScopedContext. This class is private and not for public consumption.
+ */
+public class ScopedContextAnchor {
+ private static final ThreadLocal<Deque<ScopedContext.Instance>> scopedContext = new ThreadLocal<>();
Review Comment:
Could you use `ThreadLocal.withInitial(ArrayDeque::new)` here?
##########
log4j-api/src/main/java/org/apache/logging/log4j/message/ParameterizedMapMessage.java:
##########
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j.message;
+
+import java.util.Map;
+
+/**
+ * Class Description goes here.
+ */
+public class ParameterizedMapMessage extends StringMapMessage {
+
+ private static final long serialVersionUID = -7724723101786525409L;
Review Comment:
If we're continuing to support serialization in API classes, then it would be nice if we used explicit serialization proxies for them for safety reasons.
##########
log4j-api/src/main/java/org/apache/logging/log4j/ScopedContext.java:
##########
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.internal.ScopedContextAnchor;
+import org.apache.logging.log4j.status.StatusLogger;
+
+/**
+ * Context that can be used for data to be logged in a block of code.
+ *
+ * While this is influenced by ScopedValues from Java 21 it does not share the same API. While it can perform a
+ * similar function as a set of ScopedValues it is really meant to allow a block of code to include a set of keys and
+ * values in all the log events within that block. The underlying implementation must provide support for
+ * logging the ScopedContext for that to happen.
+ *
+ * The ScopedContext will not be bound to the current thread until either a run or call method is invoked. The
+ * contexts are nested so creating and running or calling via a second ScopedContext will result in the first
+ * ScopedContext being hidden until the call is returned. Thus the values from the first ScopedContext need to
+ * be added to the second to be included.
+ *
+ * The ScopedContext can be passed to child threads by including the ExecutorService to be used to manage the
+ * run or call methods. The caller should interact with the ExecutorService as if they were submitting their
+ * run or call methods directly to it. The ScopedContext performs no error handling other than to ensure the
+ * ThreadContext and ScopedContext are cleaned up from the executed Thread.
+ *
+ * @since 2.24.0
+ */
+public class ScopedContext {
+
+ public static final Logger LOGGER = StatusLogger.getLogger();
+
+ /**
+ * @hidden
+ * Returns an unmodifiable copy of the current ScopedContext Map. This method should
+ * only be used by implementations of Log4j API.
+ * @return the Map of Renderable objects.
+ */
+ public static Map<String, Renderable> getContextMap() {
+ Optional<Instance> context = ScopedContextAnchor.getContext();
+ if (context.isPresent()
+ && context.get().contextMap != null
+ && !context.get().contextMap.isEmpty()) {
+ return Collections.unmodifiableMap(context.get().contextMap);
+ }
+ return Collections.emptyMap();
+ }
+
+ /**
+ * Return the key from the current ScopedContext, if there is one and the key exists.
+ * @param key The key.
+ * @return The value of the key in the current ScopedContext.
+ */
+ @SuppressWarnings("unchecked")
+ public static <T> T get(String key) {
+ Optional<Instance> context = ScopedContextAnchor.getContext();
+ if (context.isPresent()) {
+ Renderable renderable = context.get().contextMap.get(key);
+ if (renderable != null) {
+ return (T) renderable.getObject();
+ }
+ }
+ return null;
+ }
+
+ /**
+ * Creates a ScopedContext Instance with a key/value pair.
+ *
+ * @param key the key to add.
+ * @param value the value associated with the key.
+ * @return the Instance constructed if a valid key and value were provided. Otherwise, either the
+ * current Instance is returned or a new Instance is created if there is no current Instance.
+ */
+ public static Instance where(String key, Object value) {
+ if (value != null) {
+ Renderable renderable = value instanceof Renderable ? (Renderable) value : new ObjectRenderable(value);
+ Instance parent = current().isPresent() ? current().get() : null;
+ return new Instance(parent, key, renderable);
+ } else {
+ if (current().isPresent()) {
+ Map<String, Renderable> map = getContextMap();
+ map.remove(key);
+ return new Instance(map);
+ }
+ }
+ return current().isPresent() ? current().get() : new Instance();
+ }
+
+ /**
+ * Adds a key/value pair to the ScopedContext being constructed.
+ *
+ * @param key the key to add.
+ * @param supplier the function to generate the value.
+ * @return the ScopedContext being constructed.
+ */
+ public static Instance where(String key, Supplier<Object> supplier) {
+ return where(key, supplier.get());
+ }
+
+ /**
+ * Creates a ScopedContext Instance with a Map of keys and values.
+ * @param map the Map.
+ * @return the ScopedContext Instance constructed.
+ */
+ public static Instance where(Map<String, ?> map) {
+ if (map != null && !map.isEmpty()) {
+ Map<String, Renderable> renderableMap = new HashMap<>();
+ if (current().isPresent()) {
+ renderableMap.putAll(current().get().contextMap);
+ }
+ map.forEach((key, value) -> {
+ if (value == null || (value instanceof String && ((String) value).isEmpty())) {
+ renderableMap.remove(key);
+ } else {
+ renderableMap.put(
+ key, value instanceof Renderable ? (Renderable) value : new ObjectRenderable(value));
+ }
+ });
+ return new Instance(renderableMap);
+ } else {
+ return current().isPresent() ? current().get() : new Instance();
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a single key/value pair and calls a method.
+ * @param key the key.
+ * @param obj the value associated with the key.
+ * @param op the Runnable to call.
+ */
+ public static void runWhere(String key, Object obj, Runnable op) {
+ if (obj != null) {
+ Renderable renderable = obj instanceof Renderable ? (Renderable) obj : new ObjectRenderable(obj);
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.put(key, renderable);
+ new Instance(map).run(op);
+ } else {
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.remove(key);
+ new Instance(map).run(op);
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a single key/value pair and calls a method on a separate Thread.
+ * @param key the key.
+ * @param obj the value associated with the key.
+ * @param executorService the ExecutorService to dispatch the work.
+ * @param op the Runnable to call.
+ */
+ public static Future<?> runWhere(String key, Object obj, ExecutorService executorService, Runnable op) {
+ if (obj != null) {
+ Renderable renderable = obj instanceof Renderable ? (Renderable) obj : new ObjectRenderable(obj);
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.put(key, renderable);
+ if (executorService != null) {
+ return executorService.submit(new Runner(
+ new Instance(map), ThreadContext.getContext(), ThreadContext.getImmutableStack(), op));
+ } else {
+ new Instance(map).run(op);
+ return CompletableFuture.completedFuture(0);
+ }
+ } else {
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.remove(key);
+ if (executorService != null) {
+ return executorService.submit(new Runner(
+ new Instance(map), ThreadContext.getContext(), ThreadContext.getImmutableStack(), op));
+ } else {
+ new Instance(map).run(op);
+ return CompletableFuture.completedFuture(0);
+ }
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a Map of keys and values and calls a method.
+ * @param map the Map.
+ * @param op the Runnable to call.
+ */
+ public static void runWhere(Map<String, ?> map, Runnable op) {
+ if (map != null && !map.isEmpty()) {
+ Map<String, Renderable> renderableMap = new HashMap<>();
+ if (current().isPresent()) {
+ renderableMap.putAll(current().get().contextMap);
+ }
+ map.forEach((key, value) -> {
+ renderableMap.put(key, value instanceof Renderable ? (Renderable) value : new ObjectRenderable(value));
+ });
+ new Instance(renderableMap).run(op);
+ } else {
+ op.run();
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a single key/value pair and calls a method.
+ * @param key the key.
+ * @param obj the value associated with the key.
+ * @param op the Runnable to call.
+ */
+ public static <R> R callWhere(String key, Object obj, Callable<R> op) throws Exception {
+ if (obj != null) {
+ Renderable renderable = obj instanceof Renderable ? (Renderable) obj : new ObjectRenderable(obj);
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.put(key, renderable);
+ return new Instance(map).call(op);
+ } else {
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.remove(key);
+ return new Instance(map).call(op);
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a single key/value pair and calls a method on a separate Thread.
+ * @param key the key.
+ * @param obj the value associated with the key.
+ * @param executorService the ExecutorService to dispatch the work.
+ * @param op the Callable to call.
+ */
+ public static <R> Future<R> callWhere(String key, Object obj, ExecutorService executorService, Callable<R> op)
Review Comment:
We've got Java 8 at minimum at our fingertips. Let's make sure to support `CompletableFuture` as well. If possible, it'd be great to support `Flow.Publisher` from Java 9, too, though context propagation with reactive streams tends to use a non-standard API such as Reactor's context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544757312
##########
log4j-api/src/main/java/org/apache/logging/log4j/message/ParameterizedMapMessage.java:
##########
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j.message;
+
+import java.util.Map;
+
+/**
+ * Class Description goes here.
+ */
+public class ParameterizedMapMessage extends StringMapMessage {
+
+ private static final long serialVersionUID = -7724723101786525409L;
Review Comment:
I don't want to continue to support Serializatoin but Intellij shows a warning without the uuid.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2027934747
@ppkarwasz
You have flagged this as "Changes Requested". It isn't really clear what you have requested. If it is to remove ScopedContext that isn't going to happen so we will have to work that out. If it is something else please let me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1545163907
##########
log4j-api/src/main/java/org/apache/logging/log4j/ScopedContext.java:
##########
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.internal.ScopedContextAnchor;
+import org.apache.logging.log4j.status.StatusLogger;
+
+/**
+ * Context that can be used for data to be logged in a block of code.
+ *
+ * While this is influenced by ScopedValues from Java 21 it does not share the same API. While it can perform a
+ * similar function as a set of ScopedValues it is really meant to allow a block of code to include a set of keys and
+ * values in all the log events within that block. The underlying implementation must provide support for
+ * logging the ScopedContext for that to happen.
+ *
+ * The ScopedContext will not be bound to the current thread until either a run or call method is invoked. The
+ * contexts are nested so creating and running or calling via a second ScopedContext will result in the first
+ * ScopedContext being hidden until the call is returned. Thus the values from the first ScopedContext need to
+ * be added to the second to be included.
+ *
+ * The ScopedContext can be passed to child threads by including the ExecutorService to be used to manage the
+ * run or call methods. The caller should interact with the ExecutorService as if they were submitting their
+ * run or call methods directly to it. The ScopedContext performs no error handling other than to ensure the
+ * ThreadContext and ScopedContext are cleaned up from the executed Thread.
+ *
+ * @since 2.24.0
+ */
+public class ScopedContext {
Review Comment:
@ppkarwasz
Regarding your Observation object, just please no. This is way, way too complicated and it is serving a completely different concept than just tracking keyed items you want included in the logs (or possibly quick access to in your application). The ScopedContext can be propogated anywhere as well. We have a utility class at work that propagates the ThreadContext via RestTemplate, Kafka, and Amqp. Making it support ScopedContext is easy as it just needs to do what ScopedContextDataProvider does to get hold of the context entries. Actually, a version for REST is in Log4j-Audit. I don't believe Log4j should provide the integrations to these technologies at least as part of core or the api.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2035025220
Rats. I can't reopen the PR. Guess I have to create a new one again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rocketraman (via GitHub)" <gi...@apache.org>.
rocketraman commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2025107929
@rgoers
> If you have other requirements for how the data should be formatted I am sure JsonTemplateLayout could be enhanced to support it, if it can't do it already.
I understand that if one has end-to-end control over layout, we can use the approach you've created here. However, we have a meaningful and clear separation between logging and layout. This means it is absolutely normal for developers to configure application logging, while some completely different of people (enterprise dev tooling group or perhaps upstream library such as Spring Boot) determines layout, and in many cases developers have no ability to affect layout at all -- all applications across the enterprise use the same layout in order to enforce logging consistency. My references to Google Cloud Logging are for just such a case at an enterprise client of mine in which the logs in their Google Cloud have a `context` key for all contextual data, and I have absolutely no control over that. With your solution, my resource context will sometimes be in `context` and sometimes be in `message`. Furthermore, my `message` field won't even be consistent -- sometimes it will be a `Str
ing`, and sometimes it will be a `Map` of fields (unless I control the layout). This is just completely surprising behavior for a user.
If we want to blur the separation between context and message, then lets go all the way. Put everything into message, and then come up with an appropriate solution in order to distinguish different types of fields for layout. Make it a breaking change for Log4j3.
> The problem is that Loggers are singletons and the code where the LogEvent is constructed and the ContextMap is populated is pretty far down the stack.
I'm not as familiar with Log4j internals as you and @ppkarwasz . However, from the perspective of a naiive user, I'm not sure what is so difficult? As a user I could wrap my logger in my own class that, for every method, did (in pseudo-code):
```
class ContextLogger {
private final Map<String, String> context;
private final Logger delegate;
void log(...) {
setContext(context)
try {
delegate.log(...)
} finally {
resetContext()
}
}
}
```
and (I believe) problem solved. However, this should be unnecessary for a user to do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2025463001
@ppkarwasz
> The fact that there is only one logger per logger name is not set in stone (unlike in Logback). [LoggerRegistry](https://logging.apache.org/log4j/2.x/javadoc/log4j-api/org/apache/logging/log4j/spi/LoggerRegistry) is a multi-key map with a (String, MessageFactory) key.
>
> Due to the architectural choices you did at the beginning, Loggers don't even need to form a hierarchy, only LoggerConfigs are hierarchical. Let's profit from that and remove the "singleton" restriction from [LogManager#getLogger](https://logging.apache.org/log4j/2.x/javadoc/log4j-api/org/apache/logging/log4j/LogManager.html#getLogger(java.lang.Class,org.apache.logging.log4j.message.MessageFactory)). It is not a breaking change for the users to allow them to use different MessageFactory instances for the same logger name.
While it is a multi-key map you will get warnings if you try to create the same logger with multiple message factories as usually that is going to cause problems. However, I don't see how that helps with what rocketraman is requesting.
It is worth repeating that resource Loggers should **NEVER** be added to the LoggerRegistry. Resource Loggers need to have the same lifetime as the resource. Loggers in the registry never go away and so would result in a memory leak. Note that them not being in the registry isn't really a problem as the underlying Logger they use is. That Logger will be reconfigured when needed.
>
> IMHO, the ParameterizedMapMessage class that you introduce is already half way to meet @rocketraman's expectations:
>
> by using a special [LogEventFactory](https://logging.apache.org/log4j/2.x/javadoc/log4j-core/org/apache/logging/log4j/core/impl/LogEventFactory) the map data can be added to the context map and the baseMessage can be used as message. Something similar already happens with [TimestampMessage](https://logging.apache.org/log4j/2.x/javadoc/log4j-api/org/apache/logging/log4j/message/TimestampMessage)s that add their own timestamp to the LogEvent,
> certainly the specification of [ContextDataInjector#injectContextData](https://logging.apache.org/log4j/2.x/javadoc/log4j-core/org/apache/logging/log4j/core/ContextDataInjector#injectContextData(java.util.List,org.apache.logging.log4j.util.StringMap)) must change: currently it already receives the current context data of the event as reusable parameter, but most implementation ignore it.
Sure, you _could_ do this, but I am not sure it is a good idea. There are other uses of MapMessage where you do not want the data merged into the context data. I would be hesitant to always merge it without some indication from the user since there are a lot of cases where that is not what you want.
> Is there really such a difference between [Message](https://logging.apache.org/log4j/2.x/javadoc/log4j-api/org/apache/logging/log4j/message/Message) and [LogEvent](https://logging.apache.org/log4j/2.x/javadoc/log4j-core/org/apache/logging/log4j/core/LogEvent)?
Did you just answer your own question? Of course there is a difference. A Message is really just a container for the format String and its parameters. The fact that you can make more complex messages like a MapMessage is a perk. But the main difference is that the user really controls the contents of the Message while Log4j controls the LogEvent. Now, we could enhance the transformer to replace logging calls that take parameters with Logging calls that accept LogEvents but a) I believe it would be tough to make that garbage free and b) I am not sure we could always get it right since a lot is determined at run time.
The bottom line here is Piotr, I am not sure what to do with your comment. You haven't actually asked for any changes.
@rocketraman
> in many cases developers have no ability to affect layout at all — all applications across the enterprise use the same layout in order to enforce logging consistency
I absolutely understand this. All the Spring Boot and Spring Cloud Config support in Log4j was added by me to support my employer. We have a common logging configuration stored in Spring Cloud Config that all applications share. However, every application can still provide an override that extends capabilities and we take advantage of that.
Now this is sort of interesting
```
class ContextLogger {
private final Map<String, String> context;
private final Logger delegate;
void log(...) {
setContext(context)
try {
delegate.log(...)
} finally {
resetContext()
}
}
}
```
Yes, one could modify ResourceLogger to have
```
public void logMessage(final String fqcn, final Level level, final Marker marker, final Message message, final Throwable t) {
ScopedContext.runWhere(this.supplier.get(), () -> logger.logMessage(fqcn, level, marker, message, t));
}
```
However, this has a small side effect that any logging calls that occur from objects passed into the logging call or logging that occurs from Appenders and other components while processing the logevent will also have this context data, despite them using a different logger.
If that makes sense I would actually consider this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544402118
##########
log4j-api/src/main/java/org/apache/logging/log4j/message/ParameterizedMapMessage.java:
##########
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j.message;
+
+import java.util.Map;
+
+/**
+ * Class Description goes here.
+ */
+public class ParameterizedMapMessage extends StringMapMessage {
+
+ private static final long serialVersionUID = -7724723101786525409L;
+ private final Message baseMessage;
+
+ ParameterizedMapMessage(Message baseMessage, Map<String, String> resourceMap) {
+ super(resourceMap);
+ this.baseMessage = baseMessage;
+ }
+
+ @Override
+ public String getFormattedMessage() {
+ return baseMessage.getFormattedMessage();
+ }
Review Comment:
Currently, Open Telemetry uses `MapMessage#getFormat()` and `MapMessage#get("message")` to extract the body of a map message (cf. [`LogEventMapper`](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/7b5b7584a0fb9e9b7920aa114a5ec979901329dd/instrumentation/log4j/log4j-appender-2.17/library/src/main/java/io/opentelemetry/instrumentation/log4j/appender/v2_17/internal/LogEventMapper.java#L120-L154)). Therefore I would override the `getFormat()` to return `baseMessage.getFormattedMessage()`.
Regarding how `getFormattedMessage` should work, I am not sure what a user would expect. My guess is that it should be something like:
```
key1="value" key2="value" Formatted base message
```
i.e. the "FULL" message. Note that many (even third-party) layouts know about `MultiformatMessage`, so they don't call this method.
**Remark**: could we create an interface (e.g. `StructuredMessage`, `AttributeMessage` or something similar) that codifies how the current sub-classes of `MapMessage` work? E.g.:
```java
public interface AttributeMessage<V> extends Message {
@Override
default String getFormat() {
Message message = getMessage();
return message != null ? message.getFormattedMessage() : "";
}
@Override
String getFormattedMessage();
@Nullable Message getMessage();
Map<String, V> getData();
}
```
Also it would be nice to add some well-known constants for `MultiformatMessage`. Integrators **do** use strings like `JSON` in practice, but the specification does not prevent the creation of message implementations that return some garbage instead of JSON.
##########
log4j-api/src/main/java/org/apache/logging/log4j/ResourceLogger.java:
##########
@@ -0,0 +1,276 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.Map;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.message.Message;
+import org.apache.logging.log4j.message.ParameterizedMapMessageFactory;
+import org.apache.logging.log4j.spi.AbstractLogger;
+import org.apache.logging.log4j.spi.ExtendedLogger;
+import org.apache.logging.log4j.status.StatusLogger;
+import org.apache.logging.log4j.util.StackLocatorUtil;
+import org.apache.logging.log4j.util.Strings;
+
+/**
+ * Logger for resources. Formats all events using the ParameterizedMapMessageFactory along with the provided
+ * Supplier. The Supplier provides resource attributes that should be included in all log events generated
+ * from the current resource. Note that since the Supplier is called for every LogEvent being generated
+ * the values returned may change as necessary. Care should be taken to make the Supplier as efficient as
+ * possible to avoid performance issues.
+ *
+ * Unlike regular Loggers ResourceLoggers CANNOT be declared to be static. A ResourceLogger
+ * must be declared as a class member that will be garbage collected along with the instance of the resource.
+ */
+public final class ResourceLogger extends AbstractLogger {
+ private static final long serialVersionUID = -5837924138744974513L;
+ private final ExtendedLogger logger;
+
+ public static ResourceLoggerBuilder newBuilder() {
+ return new ResourceLoggerBuilder();
+ }
+
+ /*
+ * Pass our MessageFactory with its Supplier to AbstractLogger. This will be used to create
+ * the Messages prior to them being passed to the "real" Logger.
+ */
+ private ResourceLogger(final ExtendedLogger logger, final Supplier<Map<String, String>> supplier) {
+ super(logger.getName(), new ParameterizedMapMessageFactory(supplier));
+ this.logger = logger;
+ }
+
+ @Override
+ public Level getLevel() {
+ return logger.getLevel();
+ }
+
+ @Override
+ public boolean isEnabled(Level level, Marker marker, Message message, Throwable t) {
+ return logger.isEnabled(level, marker, message, t);
+ }
+
+ @Override
+ public boolean isEnabled(Level level, Marker marker, CharSequence message, Throwable t) {
+ return logger.isEnabled(level, marker, message, t);
+ }
+
+ @Override
+ public boolean isEnabled(Level level, Marker marker, Object message, Throwable t) {
+ return logger.isEnabled(level, marker, message, t);
+ }
+
+ @Override
+ public boolean isEnabled(Level level, Marker marker, String message, Throwable t) {
+ return logger.isEnabled(level, marker, message, t);
+ }
+
+ @Override
+ public boolean isEnabled(Level level, Marker marker, String message) {
+ return logger.isEnabled(level, marker, message);
+ }
+
+ @Override
+ public boolean isEnabled(Level level, Marker marker, String message, Object... params) {
+ return logger.isEnabled(level, marker, message, params);
+ }
+
+ @Override
+ public boolean isEnabled(Level level, Marker marker, String message, Object p0) {
+ return logger.isEnabled(level, marker, message, p0);
+ }
+
+ @Override
+ public boolean isEnabled(Level level, Marker marker, String message, Object p0, Object p1) {
+ return logger.isEnabled(level, marker, message, p0, p1);
+ }
+
+ @Override
+ public boolean isEnabled(Level level, Marker marker, String message, Object p0, Object p1, Object p2) {
+ return logger.isEnabled(level, marker, message, p0, p1, p2);
+ }
+
+ @Override
+ public boolean isEnabled(Level level, Marker marker, String message, Object p0, Object p1, Object p2, Object p3) {
+ return logger.isEnabled(level, marker, message, p0, p1, p2, p3);
+ }
+
+ @Override
+ public boolean isEnabled(
+ Level level, Marker marker, String message, Object p0, Object p1, Object p2, Object p3, Object p4) {
+ return logger.isEnabled(level, marker, message, p0, p1, p2, p3, p4);
+ }
+
+ @Override
+ public boolean isEnabled(
+ Level level,
+ Marker marker,
+ String message,
+ Object p0,
+ Object p1,
+ Object p2,
+ Object p3,
+ Object p4,
+ Object p5) {
+ return logger.isEnabled(level, marker, message, p0, p1, p2, p3, p4, p5);
+ }
+
+ @Override
+ public boolean isEnabled(
+ Level level,
+ Marker marker,
+ String message,
+ Object p0,
+ Object p1,
+ Object p2,
+ Object p3,
+ Object p4,
+ Object p5,
+ Object p6) {
+ return logger.isEnabled(level, marker, message, p0, p1, p2, p3, p4, p5, p6);
+ }
+
+ @Override
+ public boolean isEnabled(
+ Level level,
+ Marker marker,
+ String message,
+ Object p0,
+ Object p1,
+ Object p2,
+ Object p3,
+ Object p4,
+ Object p5,
+ Object p6,
+ Object p7) {
+ return logger.isEnabled(level, marker, message, p0, p1, p2, p3, p4, p5, p6, p7);
+ }
+
+ @Override
+ public boolean isEnabled(
+ Level level,
+ Marker marker,
+ String message,
+ Object p0,
+ Object p1,
+ Object p2,
+ Object p3,
+ Object p4,
+ Object p5,
+ Object p6,
+ Object p7,
+ Object p8) {
+ return logger.isEnabled(level, marker, message, p0, p1, p2, p3, p4, p5, p6, p7, p8);
+ }
+
+ @Override
+ public boolean isEnabled(
+ Level level,
+ Marker marker,
+ String message,
+ Object p0,
+ Object p1,
+ Object p2,
+ Object p3,
+ Object p4,
+ Object p5,
+ Object p6,
+ Object p7,
+ Object p8,
+ Object p9) {
+ return logger.isEnabled(level, marker, message, p0, p1, p2, p3, p4, p5, p6, p7, p8, p9);
+ }
+
+ @Override
+ public void logMessage(String fqcn, Level level, Marker marker, Message message, Throwable t) {
+ logger.logMessage(fqcn, level, marker, message, t);
+ }
+
+ /**
+ * Constructs a ResourceLogger.
+ */
+ public static final class ResourceLoggerBuilder {
Review Comment:
Personally I would prefer to just have a `ResourceLoggerBuilder` (or `LoggerBuilder`) **interface** and access it through a new `Logger#newDetachedLoggerBuilder` method.
Creating "detached" loggers might be used for other purposes than adding key/value pairs to a message. For example someone might want to use a different `MessageFactory`.
##########
log4j-api/src/main/java/org/apache/logging/log4j/ScopedContext.java:
##########
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.internal.ScopedContextAnchor;
+import org.apache.logging.log4j.status.StatusLogger;
+
+/**
+ * Context that can be used for data to be logged in a block of code.
+ *
+ * While this is influenced by ScopedValues from Java 21 it does not share the same API. While it can perform a
+ * similar function as a set of ScopedValues it is really meant to allow a block of code to include a set of keys and
+ * values in all the log events within that block. The underlying implementation must provide support for
+ * logging the ScopedContext for that to happen.
+ *
+ * The ScopedContext will not be bound to the current thread until either a run or call method is invoked. The
+ * contexts are nested so creating and running or calling via a second ScopedContext will result in the first
+ * ScopedContext being hidden until the call is returned. Thus the values from the first ScopedContext need to
+ * be added to the second to be included.
+ *
+ * The ScopedContext can be passed to child threads by including the ExecutorService to be used to manage the
+ * run or call methods. The caller should interact with the ExecutorService as if they were submitting their
+ * run or call methods directly to it. The ScopedContext performs no error handling other than to ensure the
+ * ThreadContext and ScopedContext are cleaned up from the executed Thread.
+ *
+ * @since 2.24.0
+ */
+public class ScopedContext {
Review Comment:
[This part seems independent from the `ResourceLogger`]
While technically this class is very well written and could benefit some users, I am more and more convinced that we shouldn't offer this kind of context-propagation API.
`ThreadLocal` was a big hit and allowed users to store and retrieve key/value pairs for logging purposes (and not only :wink:), but nowadays there are APIs specialized in this kind of things. Except the [`context-propagation`](https://github.com/micrometer-metrics/context-propagation) project I cited on the mailing-list, there is an entire [Context Propagation](https://opentelemetry.io/docs/concepts/context-propagation/) service from OpenTelemetry.
If we were to offer users `ScopedContext`, I am afraid that sooner or later they would need to refactor it to use another API instead. The OTel [`Context`](https://javadoc.io/doc/io.opentelemetry/opentelemetry-context/latest/io/opentelemetry/context/Context.html) class offers many more wrappers than this class.
In my opinion, what we should do instead is to help application and libraries that instrumented their code with `ThreadContext` to migrate to a specialized context-propagation API. For example we could submit to [`open-telemetry/opentelemetry-instrumentation-java`] a `ThreadContextMap` implementation that uses the [Bagage API](https://opentelemetry.io/docs/specs/otel/baggage/api/)
##########
log4j-api/src/main/java/org/apache/logging/log4j/ResourceLogger.java:
##########
@@ -0,0 +1,276 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.Map;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.message.Message;
+import org.apache.logging.log4j.message.ParameterizedMapMessageFactory;
+import org.apache.logging.log4j.spi.AbstractLogger;
+import org.apache.logging.log4j.spi.ExtendedLogger;
+import org.apache.logging.log4j.status.StatusLogger;
+import org.apache.logging.log4j.util.StackLocatorUtil;
+import org.apache.logging.log4j.util.Strings;
+
+/**
+ * Logger for resources. Formats all events using the ParameterizedMapMessageFactory along with the provided
+ * Supplier. The Supplier provides resource attributes that should be included in all log events generated
+ * from the current resource. Note that since the Supplier is called for every LogEvent being generated
+ * the values returned may change as necessary. Care should be taken to make the Supplier as efficient as
+ * possible to avoid performance issues.
+ *
+ * Unlike regular Loggers ResourceLoggers CANNOT be declared to be static. A ResourceLogger
+ * must be declared as a class member that will be garbage collected along with the instance of the resource.
+ */
+public final class ResourceLogger extends AbstractLogger {
Review Comment:
Shouldn't this extend `ExtendedLoggerWrapper` instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2028616139
@rgoers,
> 1. We cannot reuse `ThreadContextMap` without breaking it. It only allows String values. Relaxing that allows users to store objects, which we know causes problems. Due to the way `ScopedContext` behaves storing objects is not a problem.
I think that we can reuse `ThreadContextMap`. Since Log4j API 2.8 there is a (very dangerous) [`ObjectThreadContextMap`](https://logging.apache.org/log4j/2.x/javadoc/log4j-api/org/apache/logging/log4j/spi/ObjectThreadContextMap) interface for which we have one implementation.
While I **totally** agree that exposing the functionality of that interface through `ThreadContext` would be simply insane, exposing it through `ScopedContext` is perfectly **safe**: your implementation guarantees that all `Object` values that a user might insert into the thread context will be removed when your `run` method ends.
Sure, implementing `ScopedContext` using `ThreadContextMap` is a little bit harder, since the former is `Deque`-based and the latter has a single value, but as I remarked in [this comment on #2330](https://github.com/apache/logging-log4j2/pull/2330#pullrequestreview-1924545044) we might need to add two static methods somewhere in `o.a.l.l.spi` that allow to rapidly take a snapshot of a context and restore it:
```java
/**
* Saves the current context map data.
* <p>
* The returned object is not affected by later changes to the context map.
* </p>
* @return An opaque thread-safe version of the internal context map representation.
* @see #restoreContextMap
* @since 2.24
*/
static Object saveContextMap();
/**
* Restores the context map data, from a saved version.
* @param contextMap An opaque thread-safe version of the internal context map representation.
* @see #saveContextMap
* @since 2.24
*/
static void restoreContextMap(Object contextMap);
```
These methods might be useful in `CloseableThreadContext` and for integrators to more easily transfer the thread context from one thread to another, without converting it from and to a `Map<String, String>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1545163907
##########
log4j-api/src/main/java/org/apache/logging/log4j/ScopedContext.java:
##########
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.internal.ScopedContextAnchor;
+import org.apache.logging.log4j.status.StatusLogger;
+
+/**
+ * Context that can be used for data to be logged in a block of code.
+ *
+ * While this is influenced by ScopedValues from Java 21 it does not share the same API. While it can perform a
+ * similar function as a set of ScopedValues it is really meant to allow a block of code to include a set of keys and
+ * values in all the log events within that block. The underlying implementation must provide support for
+ * logging the ScopedContext for that to happen.
+ *
+ * The ScopedContext will not be bound to the current thread until either a run or call method is invoked. The
+ * contexts are nested so creating and running or calling via a second ScopedContext will result in the first
+ * ScopedContext being hidden until the call is returned. Thus the values from the first ScopedContext need to
+ * be added to the second to be included.
+ *
+ * The ScopedContext can be passed to child threads by including the ExecutorService to be used to manage the
+ * run or call methods. The caller should interact with the ExecutorService as if they were submitting their
+ * run or call methods directly to it. The ScopedContext performs no error handling other than to ensure the
+ * ThreadContext and ScopedContext are cleaned up from the executed Thread.
+ *
+ * @since 2.24.0
+ */
+public class ScopedContext {
Review Comment:
@ppkarwasz
Regarding your Observation object, just please no. This is way, way too complicated. The ScopedContext can be propogated anywhere as well. We have a utility class at work that propagates the ThreadContext via RestTemplate, Kafka, and Amqp. Making it support ScopedContext is easy as it just needs to do what ScopedContextDataProvider does to get hold of the context entries. Actually, a version for REST is in Log4j-Audit. I don't believe Log4j should provide the integrations to these technologies at least as part of core or the api.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544820856
##########
log4j-api/src/main/java/org/apache/logging/log4j/message/ParameterizedMapMessage.java:
##########
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j.message;
+
+import java.util.Map;
+
+/**
+ * Class Description goes here.
+ */
+public class ParameterizedMapMessage extends StringMapMessage {
+
+ private static final long serialVersionUID = -7724723101786525409L;
+ private final Message baseMessage;
+
+ ParameterizedMapMessage(Message baseMessage, Map<String, String> resourceMap) {
+ super(resourceMap);
+ this.baseMessage = baseMessage;
+ }
+
+ @Override
+ public String getFormattedMessage() {
+ return baseMessage.getFormattedMessage();
+ }
Review Comment:
> I would suggest that Open Telemetry would be better served by adopting `ParameterizedMapMessage` than treating the message key as special.
Due to a "happens before" relationship they didn't have the chance to do it. :grin:
> The purpose of ParameterizedMapMessage, as I have stated several times, is to have it format %m EXACTLY how ParameterizedMessage would but still have it be a MapMessage so other Lookups, Filters, and Layouts can extract the structured data from the message. That is it. No fancy tricks.
Sure, extending `MapMessage` is a nice trick to maintain some sort of backward compatibility. However I would like to have a well documented Java interface that `ParameterizedMapMessage` will implement. In time we can switch all the `instanceof MapMessage` to that class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2024651863
@rgoers,
> @rocketraman I did look into that and what you are asking for just can't be implemented in a reasonable way without significant mods to the API and will likely impact performance. The problem is that Loggers are singletons …
The fact that there is only one **logger** per **logger name** is not set in stone (unlike in Logback). [`LoggerRegistry`](https://logging.apache.org/log4j/2.x/javadoc/log4j-api/org/apache/logging/log4j/spi/LoggerRegistry) is a multi-key map with a `(String, MessageFactory)` key.
Due to the architectural choices you did at the beginning, `Logger`s don't even need to form a hierarchy, only `LoggerConfig`s are hierarchical. Let's profit from that and remove the "singleton" restriction from [`LogManager#getLogger`](https://logging.apache.org/log4j/2.x/javadoc/log4j-api/org/apache/logging/log4j/LogManager.html#getLogger(java.lang.Class,org.apache.logging.log4j.message.MessageFactory)). It is not a breaking change for the users to allow them to use different `MessageFactory` instances for the same logger name.
> … and the code where the LogEvent is constructed and the ContextMap is populated is pretty far down the stack.
IMHO, the `ParameterizedMapMessage` class that you introduce is already half way to meet @rocketraman's expectations:
* by using a special [`LogEventFactory`](https://logging.apache.org/log4j/2.x/javadoc/log4j-core/org/apache/logging/log4j/core/impl/LogEventFactory) the map data can be added to the context map and the `baseMessage` can be used as message. Something similar already happens with [`TimestampMessage`](https://logging.apache.org/log4j/2.x/javadoc/log4j-api/org/apache/logging/log4j/message/TimestampMessage)s that add their own timestamp to the `LogEvent`,
* certainly the specification of [`ContextDataInjector#injectContextData`](https://logging.apache.org/log4j/2.x/javadoc/log4j-core/org/apache/logging/log4j/core/ContextDataInjector#injectContextData(java.util.List,org.apache.logging.log4j.util.StringMap)) must change: currently it already receives the current context data of the event as `reusable` parameter, but most implementation ignore it.
Another thing that I like about your approach is that is solves #1813: in SLF4J 2.x there is a [list of key/value pairs](https://www.slf4j.org/api/org/slf4j/event/LoggingEvent.html#getKeyValuePairs()) that is **distinct** from the context map. Currently I am mixing the two sets in `log4j-slf4j2-impl`, which is not what users expect.
> We defer creating LogEvents to avoid the performance overhead when they aren't going to be used. Messages, on the other hand, are constructed very early as they are fairly light-weight. So doing that in a per-instance Logger that wraps the singleton was fairly easy to do.
Is there really such a difference between [`Message`](https://logging.apache.org/log4j/2.x/javadoc/log4j-api/org/apache/logging/log4j/message/Message) and [`LogEvent`](https://logging.apache.org/log4j/2.x/javadoc/log4j-core/org/apache/logging/log4j/core/LogEvent)? I am very happy that there is a net separation (unlike in Logback) between:
* the data that the user **explicitly** specified in his code,
* the data that the logging backend added,
but an implementation could very well choose to use something like [`MutableLogEvent`](https://logging.apache.org/log4j/2.x/javadoc/log4j-core/org/apache/logging/log4j/core/impl/MutableLogEvent) all the way from the `Logger` to the `Appender`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "jvz (via GitHub)" <gi...@apache.org>.
jvz commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544765859
##########
log4j-api/src/main/java/org/apache/logging/log4j/ScopedContext.java:
##########
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.internal.ScopedContextAnchor;
+import org.apache.logging.log4j.status.StatusLogger;
+
+/**
+ * Context that can be used for data to be logged in a block of code.
+ *
+ * While this is influenced by ScopedValues from Java 21 it does not share the same API. While it can perform a
+ * similar function as a set of ScopedValues it is really meant to allow a block of code to include a set of keys and
+ * values in all the log events within that block. The underlying implementation must provide support for
+ * logging the ScopedContext for that to happen.
+ *
+ * The ScopedContext will not be bound to the current thread until either a run or call method is invoked. The
+ * contexts are nested so creating and running or calling via a second ScopedContext will result in the first
+ * ScopedContext being hidden until the call is returned. Thus the values from the first ScopedContext need to
+ * be added to the second to be included.
+ *
+ * The ScopedContext can be passed to child threads by including the ExecutorService to be used to manage the
+ * run or call methods. The caller should interact with the ExecutorService as if they were submitting their
+ * run or call methods directly to it. The ScopedContext performs no error handling other than to ensure the
+ * ThreadContext and ScopedContext are cleaned up from the executed Thread.
+ *
+ * @since 2.24.0
+ */
+public class ScopedContext {
+
+ public static final Logger LOGGER = StatusLogger.getLogger();
+
+ /**
+ * @hidden
+ * Returns an unmodifiable copy of the current ScopedContext Map. This method should
+ * only be used by implementations of Log4j API.
+ * @return the Map of Renderable objects.
+ */
+ public static Map<String, Renderable> getContextMap() {
+ Optional<Instance> context = ScopedContextAnchor.getContext();
+ if (context.isPresent()
+ && context.get().contextMap != null
+ && !context.get().contextMap.isEmpty()) {
+ return Collections.unmodifiableMap(context.get().contextMap);
+ }
+ return Collections.emptyMap();
+ }
+
+ /**
+ * Return the key from the current ScopedContext, if there is one and the key exists.
+ * @param key The key.
+ * @return The value of the key in the current ScopedContext.
+ */
+ @SuppressWarnings("unchecked")
+ public static <T> T get(String key) {
+ Optional<Instance> context = ScopedContextAnchor.getContext();
+ if (context.isPresent()) {
+ Renderable renderable = context.get().contextMap.get(key);
+ if (renderable != null) {
+ return (T) renderable.getObject();
+ }
+ }
+ return null;
+ }
+
+ /**
+ * Creates a ScopedContext Instance with a key/value pair.
+ *
+ * @param key the key to add.
+ * @param value the value associated with the key.
+ * @return the Instance constructed if a valid key and value were provided. Otherwise, either the
+ * current Instance is returned or a new Instance is created if there is no current Instance.
+ */
+ public static Instance where(String key, Object value) {
+ if (value != null) {
+ Renderable renderable = value instanceof Renderable ? (Renderable) value : new ObjectRenderable(value);
+ Instance parent = current().isPresent() ? current().get() : null;
+ return new Instance(parent, key, renderable);
+ } else {
+ if (current().isPresent()) {
+ Map<String, Renderable> map = getContextMap();
+ map.remove(key);
+ return new Instance(map);
+ }
+ }
+ return current().isPresent() ? current().get() : new Instance();
+ }
+
+ /**
+ * Adds a key/value pair to the ScopedContext being constructed.
+ *
+ * @param key the key to add.
+ * @param supplier the function to generate the value.
+ * @return the ScopedContext being constructed.
+ */
+ public static Instance where(String key, Supplier<Object> supplier) {
+ return where(key, supplier.get());
+ }
+
+ /**
+ * Creates a ScopedContext Instance with a Map of keys and values.
+ * @param map the Map.
+ * @return the ScopedContext Instance constructed.
+ */
+ public static Instance where(Map<String, ?> map) {
+ if (map != null && !map.isEmpty()) {
+ Map<String, Renderable> renderableMap = new HashMap<>();
+ if (current().isPresent()) {
+ renderableMap.putAll(current().get().contextMap);
+ }
+ map.forEach((key, value) -> {
+ if (value == null || (value instanceof String && ((String) value).isEmpty())) {
+ renderableMap.remove(key);
+ } else {
+ renderableMap.put(
+ key, value instanceof Renderable ? (Renderable) value : new ObjectRenderable(value));
+ }
+ });
+ return new Instance(renderableMap);
+ } else {
+ return current().isPresent() ? current().get() : new Instance();
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a single key/value pair and calls a method.
+ * @param key the key.
+ * @param obj the value associated with the key.
+ * @param op the Runnable to call.
+ */
+ public static void runWhere(String key, Object obj, Runnable op) {
+ if (obj != null) {
+ Renderable renderable = obj instanceof Renderable ? (Renderable) obj : new ObjectRenderable(obj);
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.put(key, renderable);
+ new Instance(map).run(op);
+ } else {
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.remove(key);
+ new Instance(map).run(op);
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a single key/value pair and calls a method on a separate Thread.
+ * @param key the key.
+ * @param obj the value associated with the key.
+ * @param executorService the ExecutorService to dispatch the work.
+ * @param op the Runnable to call.
+ */
+ public static Future<?> runWhere(String key, Object obj, ExecutorService executorService, Runnable op) {
+ if (obj != null) {
+ Renderable renderable = obj instanceof Renderable ? (Renderable) obj : new ObjectRenderable(obj);
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.put(key, renderable);
+ if (executorService != null) {
+ return executorService.submit(new Runner(
+ new Instance(map), ThreadContext.getContext(), ThreadContext.getImmutableStack(), op));
+ } else {
+ new Instance(map).run(op);
+ return CompletableFuture.completedFuture(0);
+ }
+ } else {
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.remove(key);
+ if (executorService != null) {
+ return executorService.submit(new Runner(
+ new Instance(map), ThreadContext.getContext(), ThreadContext.getImmutableStack(), op));
+ } else {
+ new Instance(map).run(op);
+ return CompletableFuture.completedFuture(0);
+ }
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a Map of keys and values and calls a method.
+ * @param map the Map.
+ * @param op the Runnable to call.
+ */
+ public static void runWhere(Map<String, ?> map, Runnable op) {
+ if (map != null && !map.isEmpty()) {
+ Map<String, Renderable> renderableMap = new HashMap<>();
+ if (current().isPresent()) {
+ renderableMap.putAll(current().get().contextMap);
+ }
+ map.forEach((key, value) -> {
+ renderableMap.put(key, value instanceof Renderable ? (Renderable) value : new ObjectRenderable(value));
+ });
+ new Instance(renderableMap).run(op);
+ } else {
+ op.run();
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a single key/value pair and calls a method.
+ * @param key the key.
+ * @param obj the value associated with the key.
+ * @param op the Runnable to call.
+ */
+ public static <R> R callWhere(String key, Object obj, Callable<R> op) throws Exception {
+ if (obj != null) {
+ Renderable renderable = obj instanceof Renderable ? (Renderable) obj : new ObjectRenderable(obj);
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.put(key, renderable);
+ return new Instance(map).call(op);
+ } else {
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.remove(key);
+ return new Instance(map).call(op);
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a single key/value pair and calls a method on a separate Thread.
+ * @param key the key.
+ * @param obj the value associated with the key.
+ * @param executorService the ExecutorService to dispatch the work.
+ * @param op the Callable to call.
+ */
+ public static <R> Future<R> callWhere(String key, Object obj, ExecutorService executorService, Callable<R> op)
Review Comment:
It implements Future, yes, but it's also a CompletionStage which is the part of the API I'm more interested in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2027242392
@rgoers,
> It is worth repeating that resource Loggers should **NEVER** be added to the LoggerRegistry. Resource Loggers need to have the same lifetime as the resource. Loggers in the registry never go away and so would result in a memory leak. Note that them not being in the registry isn't really a problem as the underlying Logger they use is. That Logger will be reconfigured when needed.
Good point, so maybe `LogManager#getLogger(String name, MessageFactory factory)` could be reimplemented to:
1. create a logger `logger` with the **default** message factory,
2. if `factory` is not `null`, return a wrapper of `logger`.
> > IMHO, the ParameterizedMapMessage class that you introduce is already half way to meet @rocketraman's expectations:
> > * by using a special [LogEventFactory](https://logging.apache.org/log4j/2.x/javadoc/log4j-core/org/apache/logging/log4j/core/impl/LogEventFactory) the map data can be added to the context map and the baseMessage can be used as message. Something similar already happens with [TimestampMessage](https://logging.apache.org/log4j/2.x/javadoc/log4j-api/org/apache/logging/log4j/message/TimestampMessage)s that add their own timestamp to the LogEvent,
> > * certainly the specification of [ContextDataInjector#injectContextData](https://logging.apache.org/log4j/2.x/javadoc/log4j-core/org/apache/logging/log4j/core/ContextDataInjector#injectContextData(java.util.List,org.apache.logging.log4j.util.StringMap)) must change: currently it already receives the current context data of the event as reusable parameter, but most implementation ignore it.
>
> Sure, you _could_ do this, but I am not sure it is a good idea. There are other uses of `MapMessage` where you do not want the data merged into the context data. I would be hesitant to always merge it without some indication from the user since there are a lot of cases where that is not what you want.
Sure, the **default** event factory should not merge the data, but we can propose an alternative one that does merge it. The crucial point for this to work is that `ContextDataInjector` implementations should **not** ignore the `reusable` parameter they receive, but always test if it is empty or not and act accordingly. I don't think the an `isEmpty()` test will have any impact on the performance.
> Did you just answer your own question? Of course there is a difference. A Message is really just a container for the format String and its parameters. The fact that you can make more complex messages like a MapMessage is a perk. But the main difference is that the user really controls the contents of the Message while Log4j controls the LogEvent.
Sure, there is a semantic difference, but a `MutableLogEvent` with all the "message" fields filled and all the "event" fields empty, looks pretty much as a `Message` to me. As I previously mentioned, for Log4j 3.x I would like to create a factory that implements both `MessageFactory` and `LogEventFactory`. The `createMessage` methods will pull `MutableLogEvent`s from a recycler and fill the "message" fields, while the `createEvent` methods will fill the remaining fields.
> The bottom line here is Piotr, I am not sure what to do with your comment. You haven't actually asked for any changes.
I needed some time to digest your proposed changes, :wink:. While it doesn't touch a lot files, the amount of API changes in this PR has not been seen since the introduction of `LogBuilder`. Having done some questionable "improvements" to the `o.a.l.l` package in the past (e.g. the `BridgeAware` interface that I should have put in `o.a.l.l.spi`), I had to do some research to decide which changes are required and which we'd better drop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on code in PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#discussion_r1544753833
##########
log4j-api/src/main/java/org/apache/logging/log4j/ScopedContext.java:
##########
@@ -0,0 +1,558 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.logging.log4j;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.function.Supplier;
+import org.apache.logging.log4j.internal.ScopedContextAnchor;
+import org.apache.logging.log4j.status.StatusLogger;
+
+/**
+ * Context that can be used for data to be logged in a block of code.
+ *
+ * While this is influenced by ScopedValues from Java 21 it does not share the same API. While it can perform a
+ * similar function as a set of ScopedValues it is really meant to allow a block of code to include a set of keys and
+ * values in all the log events within that block. The underlying implementation must provide support for
+ * logging the ScopedContext for that to happen.
+ *
+ * The ScopedContext will not be bound to the current thread until either a run or call method is invoked. The
+ * contexts are nested so creating and running or calling via a second ScopedContext will result in the first
+ * ScopedContext being hidden until the call is returned. Thus the values from the first ScopedContext need to
+ * be added to the second to be included.
+ *
+ * The ScopedContext can be passed to child threads by including the ExecutorService to be used to manage the
+ * run or call methods. The caller should interact with the ExecutorService as if they were submitting their
+ * run or call methods directly to it. The ScopedContext performs no error handling other than to ensure the
+ * ThreadContext and ScopedContext are cleaned up from the executed Thread.
+ *
+ * @since 2.24.0
+ */
+public class ScopedContext {
+
+ public static final Logger LOGGER = StatusLogger.getLogger();
+
+ /**
+ * @hidden
+ * Returns an unmodifiable copy of the current ScopedContext Map. This method should
+ * only be used by implementations of Log4j API.
+ * @return the Map of Renderable objects.
+ */
+ public static Map<String, Renderable> getContextMap() {
+ Optional<Instance> context = ScopedContextAnchor.getContext();
+ if (context.isPresent()
+ && context.get().contextMap != null
+ && !context.get().contextMap.isEmpty()) {
+ return Collections.unmodifiableMap(context.get().contextMap);
+ }
+ return Collections.emptyMap();
+ }
+
+ /**
+ * Return the key from the current ScopedContext, if there is one and the key exists.
+ * @param key The key.
+ * @return The value of the key in the current ScopedContext.
+ */
+ @SuppressWarnings("unchecked")
+ public static <T> T get(String key) {
+ Optional<Instance> context = ScopedContextAnchor.getContext();
+ if (context.isPresent()) {
+ Renderable renderable = context.get().contextMap.get(key);
+ if (renderable != null) {
+ return (T) renderable.getObject();
+ }
+ }
+ return null;
+ }
+
+ /**
+ * Creates a ScopedContext Instance with a key/value pair.
+ *
+ * @param key the key to add.
+ * @param value the value associated with the key.
+ * @return the Instance constructed if a valid key and value were provided. Otherwise, either the
+ * current Instance is returned or a new Instance is created if there is no current Instance.
+ */
+ public static Instance where(String key, Object value) {
+ if (value != null) {
+ Renderable renderable = value instanceof Renderable ? (Renderable) value : new ObjectRenderable(value);
+ Instance parent = current().isPresent() ? current().get() : null;
+ return new Instance(parent, key, renderable);
+ } else {
+ if (current().isPresent()) {
+ Map<String, Renderable> map = getContextMap();
+ map.remove(key);
+ return new Instance(map);
+ }
+ }
+ return current().isPresent() ? current().get() : new Instance();
+ }
+
+ /**
+ * Adds a key/value pair to the ScopedContext being constructed.
+ *
+ * @param key the key to add.
+ * @param supplier the function to generate the value.
+ * @return the ScopedContext being constructed.
+ */
+ public static Instance where(String key, Supplier<Object> supplier) {
+ return where(key, supplier.get());
+ }
+
+ /**
+ * Creates a ScopedContext Instance with a Map of keys and values.
+ * @param map the Map.
+ * @return the ScopedContext Instance constructed.
+ */
+ public static Instance where(Map<String, ?> map) {
+ if (map != null && !map.isEmpty()) {
+ Map<String, Renderable> renderableMap = new HashMap<>();
+ if (current().isPresent()) {
+ renderableMap.putAll(current().get().contextMap);
+ }
+ map.forEach((key, value) -> {
+ if (value == null || (value instanceof String && ((String) value).isEmpty())) {
+ renderableMap.remove(key);
+ } else {
+ renderableMap.put(
+ key, value instanceof Renderable ? (Renderable) value : new ObjectRenderable(value));
+ }
+ });
+ return new Instance(renderableMap);
+ } else {
+ return current().isPresent() ? current().get() : new Instance();
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a single key/value pair and calls a method.
+ * @param key the key.
+ * @param obj the value associated with the key.
+ * @param op the Runnable to call.
+ */
+ public static void runWhere(String key, Object obj, Runnable op) {
+ if (obj != null) {
+ Renderable renderable = obj instanceof Renderable ? (Renderable) obj : new ObjectRenderable(obj);
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.put(key, renderable);
+ new Instance(map).run(op);
+ } else {
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.remove(key);
+ new Instance(map).run(op);
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a single key/value pair and calls a method on a separate Thread.
+ * @param key the key.
+ * @param obj the value associated with the key.
+ * @param executorService the ExecutorService to dispatch the work.
+ * @param op the Runnable to call.
+ */
+ public static Future<?> runWhere(String key, Object obj, ExecutorService executorService, Runnable op) {
+ if (obj != null) {
+ Renderable renderable = obj instanceof Renderable ? (Renderable) obj : new ObjectRenderable(obj);
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.put(key, renderable);
+ if (executorService != null) {
+ return executorService.submit(new Runner(
+ new Instance(map), ThreadContext.getContext(), ThreadContext.getImmutableStack(), op));
+ } else {
+ new Instance(map).run(op);
+ return CompletableFuture.completedFuture(0);
+ }
+ } else {
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.remove(key);
+ if (executorService != null) {
+ return executorService.submit(new Runner(
+ new Instance(map), ThreadContext.getContext(), ThreadContext.getImmutableStack(), op));
+ } else {
+ new Instance(map).run(op);
+ return CompletableFuture.completedFuture(0);
+ }
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a Map of keys and values and calls a method.
+ * @param map the Map.
+ * @param op the Runnable to call.
+ */
+ public static void runWhere(Map<String, ?> map, Runnable op) {
+ if (map != null && !map.isEmpty()) {
+ Map<String, Renderable> renderableMap = new HashMap<>();
+ if (current().isPresent()) {
+ renderableMap.putAll(current().get().contextMap);
+ }
+ map.forEach((key, value) -> {
+ renderableMap.put(key, value instanceof Renderable ? (Renderable) value : new ObjectRenderable(value));
+ });
+ new Instance(renderableMap).run(op);
+ } else {
+ op.run();
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a single key/value pair and calls a method.
+ * @param key the key.
+ * @param obj the value associated with the key.
+ * @param op the Runnable to call.
+ */
+ public static <R> R callWhere(String key, Object obj, Callable<R> op) throws Exception {
+ if (obj != null) {
+ Renderable renderable = obj instanceof Renderable ? (Renderable) obj : new ObjectRenderable(obj);
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.put(key, renderable);
+ return new Instance(map).call(op);
+ } else {
+ Map<String, Renderable> map = new HashMap<>();
+ if (current().isPresent()) {
+ map.putAll(current().get().contextMap);
+ }
+ map.remove(key);
+ return new Instance(map).call(op);
+ }
+ }
+
+ /**
+ * Creates a ScopedContext with a single key/value pair and calls a method on a separate Thread.
+ * @param key the key.
+ * @param obj the value associated with the key.
+ * @param executorService the ExecutorService to dispatch the work.
+ * @param op the Callable to call.
+ */
+ public static <R> Future<R> callWhere(String key, Object obj, ExecutorService executorService, Callable<R> op)
Review Comment:
@jvz Isn't CompletableFuture a Future? How is it not supported now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2027370520
@rocketraman I have modified the PR so that ResourceLogger uses ScopedContext and behaves the way you requested.
@ppkarwasz I have no idea why the PR system lets me reply to some messages and not others. It means I have to quote stuff which is annoying but whatever.
With regards to your last comment. I have changed ResourceLogger to no longer add the map data to the Message so I am assuming most of that is no longer relevant.
As things stand, I am quite happy where this is and personally would start using it (well ScopedContext anyway) right away.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Scoped context (logging-log4j2)
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #2419:
URL: https://github.com/apache/logging-log4j2/pull/2419#issuecomment-2035006189
I have updated the branch with some significant changes:
1. ContextDataProvider is now moved to the API. This allows all ContextDataProviders to be used in Lookups, Filters, etc. For example, the $ctx lookup will now include fields from both the ScopedContext, ThreadContext and custom providers.
2. ContextData has been introduced as a way to access the unified context data. So ContextData.getValue(key) will find the value for a key regardless of which ContextDataProvider it was specified in.
3. All access to the Contexts is garbage free. Multiple versions of ThreadContextMap are no longer necessary.
4. ThreadContextDataInjector and related classes are deprecated in 2.24.0 and scheduled for removal in 3.0.0. This will eliminate a lot of noise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org