You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@logging.apache.org by "Piotr P. Karwasz" <pi...@gmail.com> on 2022/12/29 06:27:15 UTC
[log4j] New FlowMessageFactory and LogBuilder methods
Hi all,
Since a little more than a month I have been working on a bytecode
conversion tool that converts all logging API calls to chains of
`LogBuilder` calls with integrated location information.
The most complex API to convert was obviously Log4j2 API, with almost
400 methods to support. This is practically done[1], except method
references like `list.forEach(logger::info)` and two families of
methods: `traceEntry` and `traceExit`.
While it is possible to deal with them without modification to the
APIs, it would simplify the logic (especially for `traceEntry`) if we
add:
* a method `LogBuilder#logAndGet(Supplier<Message>)` that returns the
logged message or null if nothing was logged. This is the semantics of
`traceEntry`.
* a method `Logger#getFlowMessageFactory()` to return the flow message
factory used by the logger,
* move the logic that creates EntryMessage/ExitMessage from
AbstractLogger to FlowMessageFactory (3 new methods).
A concrete implementation of the proposal is in
https://github.com/apache/logging-log4j2/pull/1171. What do you think
about it?
Since the `LogBuilder` equivalent of `traceEntry` and `traceExit` is
much longer, I don't expect anyone to employ these methods in their
source code.
Piotr
[1] https://github.com/apache/logging-log4j2/pull/1147
Re: Log4j Maven plugin
Posted by "Piotr P. Karwasz" <pi...@gmail.com>.
Hi Ralph,
On Wed, 4 Jan 2023 at 01:42, Ralph Goers <ra...@dslextreme.com> wrote:
> I do like this as a separate repo. If I understand what you are intending I expect that releasing this tool will be big news:
At least I hope it will be big news, previous attempts like SLF4J
Migrator[1] do not seem to be very popular, probably because they lack
binary support.
> 1. It automatically converts calls to SLF4J to Log4j, eliminating the need to include the SLF4J and log4j-slf4j-impl jars at runtime.
Yes, I believe this might be popular with projects that mostly use
Log4j2 API, but have that pesky little dependency that uses SLF4J.
A practical example: at work we have a
'com.vaadin.external.atmosphere:atmosphere:runtime'[2]. This modified
version of Atmosphere uses a rebased SLF4J, that only offers JUL as a
backend. It will the first to go under the knife.
> 2. It adds location information for free for both SLF4J and Log4j API calls, which should be a significant performance improvement.
According to our own benchmarks[3] using `LogBuilder` is about 2.5
times slower than using `Logger` *without* location. We probably need
to fill this gap.
> One gotcha we will need to make sure is accounted for is the use of Lombok. Will the tool be able to find the Logger created by Lombok?
Yes, Lombok runs at compilation time. When the tool post-processes the
bytecode, it will find everything it needs.
BTW, I have found an OpenRewrite plugin[4] that rewrites logging
frameworks. So I'll just need to send them a big PR with an
"everything to Log4j2" rule.
Piotr
[1] https://www.slf4j.org/migrator.html
[2] https://mvnrepository.com/artifact/com.vaadin.external.atmosphere/atmosphere-runtime
[3] https://logging.apache.org/log4j/2.x/manual/logbuilder.html
[4] https://github.com/openrewrite/rewrite-logging-frameworks
Re: Log4j Maven plugin
Posted by Ralph Goers <ra...@dslextreme.com>.
I do like this as a separate repo. If I understand what you are intending I expect that releasing this tool will be big news:
1. It automatically converts calls to SLF4J to Log4j, eliminating the need to include the SLF4J and log4j-slf4j-impl jars at runtime.
2. It adds location information for free for both SLF4J and Log4j API calls, which should be a significant performance improvement.
One gotcha we will need to make sure is accounted for is the use of Lombok. Will the tool be able to find the Logger created by Lombok?
Ralph
> On Jan 3, 2023, at 12:35 PM, Volkan Yazıcı <vo...@yazi.ci> wrote:
>
> `logging-log4j-transform` and `org.apache.logging.log4j` are fine by me.
>
> On Tue, Jan 3, 2023 at 6:43 PM Piotr P. Karwasz <pi...@gmail.com>
> wrote:
>
>> Hi Volkan,
>>
>> On Tue, 3 Jan 2023 at 09:45, Volkan Yazıcı <vo...@yazi.ci> wrote:
>>> 1. Create a new `logging-log4j-infra` (groupId: `o.a.l.l.infra`)
>>> repository and move the `log4j-changelog` module there and put Piotr's
>>> stuff into `logging-log4j-tools` (groupId: `o.a.l.l.tools`)
>>> 2. Move everybody to their own repo: `logging-log4j-changelog`,
>>> `logging-log4j-instrument`, `logging-log4j-errorprone`, etc. This
>> might
>>> turn out to be a better approach given aforementioned projects will
>> most
>>> probably have totally different development cycles.
>>
>> What about `logging-log4j-transform`? The Maven plugin could be called
>> `log4j-transform-maven-plugin` in case we publish other plugins.
>>
>> For the groupId I think `org.apache.logging.log4j` that Ralph proposes
>> will be fine.
>>
>> Piotr
>>
Re: Log4j Maven plugin
Posted by Volkan Yazıcı <vo...@yazi.ci>.
`logging-log4j-transform` and `org.apache.logging.log4j` are fine by me.
On Tue, Jan 3, 2023 at 6:43 PM Piotr P. Karwasz <pi...@gmail.com>
wrote:
> Hi Volkan,
>
> On Tue, 3 Jan 2023 at 09:45, Volkan Yazıcı <vo...@yazi.ci> wrote:
> > 1. Create a new `logging-log4j-infra` (groupId: `o.a.l.l.infra`)
> > repository and move the `log4j-changelog` module there and put Piotr's
> > stuff into `logging-log4j-tools` (groupId: `o.a.l.l.tools`)
> > 2. Move everybody to their own repo: `logging-log4j-changelog`,
> > `logging-log4j-instrument`, `logging-log4j-errorprone`, etc. This
> might
> > turn out to be a better approach given aforementioned projects will
> most
> > probably have totally different development cycles.
>
> What about `logging-log4j-transform`? The Maven plugin could be called
> `log4j-transform-maven-plugin` in case we publish other plugins.
>
> For the groupId I think `org.apache.logging.log4j` that Ralph proposes
> will be fine.
>
> Piotr
>
Re: Log4j Maven plugin
Posted by "Piotr P. Karwasz" <pi...@gmail.com>.
Hi Volkan,
On Tue, 3 Jan 2023 at 09:45, Volkan Yazıcı <vo...@yazi.ci> wrote:
> 1. Create a new `logging-log4j-infra` (groupId: `o.a.l.l.infra`)
> repository and move the `log4j-changelog` module there and put Piotr's
> stuff into `logging-log4j-tools` (groupId: `o.a.l.l.tools`)
> 2. Move everybody to their own repo: `logging-log4j-changelog`,
> `logging-log4j-instrument`, `logging-log4j-errorprone`, etc. This might
> turn out to be a better approach given aforementioned projects will most
> probably have totally different development cycles.
What about `logging-log4j-transform`? The Maven plugin could be called
`log4j-transform-maven-plugin` in case we publish other plugins.
For the groupId I think `org.apache.logging.log4j` that Ralph proposes
will be fine.
Piotr
Re: Log4j Maven plugin
Posted by Volkan Yazıcı <vo...@yazi.ci>.
I disagree with putting these into `logging-log4j-tools` with the current
state of the project: It is intended for *internal* usage. For
`log4j-changelog`, I want to make it explicit that it is not intended for
usage outside Log4j itself. I have two proposals:
1. Create a new `logging-log4j-infra` (groupId: `o.a.l.l.infra`)
repository and move the `log4j-changelog` module there and put Piotr's
stuff into `logging-log4j-tools` (groupId: `o.a.l.l.tools`)
2. Move everybody to their own repo: `logging-log4j-changelog`,
`logging-log4j-instrument`, `logging-log4j-errorprone`, etc. This might
turn out to be a better approach given aforementioned projects will most
probably have totally different development cycles.
What do you think?
Regarding code rewrites, i.e., `log4j-errorprone`, Spring Boot 3 has
introduced Spring Boot Migrator
<https://github.com/spring-projects-experimental/spring-boot-migrator>,
which uses OpenRewrite <https://docs.openrewrite.org/> under the hood.
Judging from my personal experience, code rewrite functionality in Error
Prone never got mass adoption and Spring is really influential in the Java
domain. I expect OpenRewrite to get quite some steam in the near future.
On Tue, Jan 3, 2023 at 1:50 AM Ralph Goers <ra...@dslextreme.com>
wrote:
> I think putting it in the tools repo would be fine.
>
> Ralph
>
> > On Jan 2, 2023, at 2:34 PM, Piotr P. Karwasz <pi...@gmail.com>
> wrote:
> >
> > Hi,
> >
> >> On Mon, 2 Jan 2023 at 10:08, Volkan Yazıcı <vo...@yazi.ci> wrote:
> >> Agreed with the separate repository idea. I suggest moving it to the
> >> `logging-log4j-maven-plugins` repository – you need an INFRA ticket to
> >> create the repository.
> >
> > The Maven plugin will not be the only use-case of the code after I
> > finish up the project.
> >
> > Since I am really bad at naming things, so maybe I'll describe the
> > feature-set and someone will come with the ideal name. :-D
> >
> > I imagine one module that will weave bytecode (`log4j-instrument`) to:
> > * convert classical Log4j2 API calls to LogBuilder calls with location,
> > * convert other API calls (Log4j 1.2, JCL, SLF4J and especially
> > `java.util.logging`) to Log4j2 API calls (LogBuilder with location for
> > now),
> > * implement `java.lang.instrument.ClassFileTransformer` so that we
> > can use it to weave at runtime.
> >
> > A second module would be based on a refactoring library (Error Prone?
> > called `log4j-errorprone`?) to:
> > * convert classical Log4j2 API calls to LogBuilder (this simplifies
> > weaving a lot),
> > * convert other API calls to Log4j2 API,
> > * check for common mistakes of new Log4j2 API adopters: e.g. `if
> > (logger.isDebugEnabled() logger.debug(...);` or `logger.debug("Hello
> > name " + name + ".")`.
> >
> > A Maven module would use `log4j-instrument` to:
> > * weave a project's classes,
> > * weave all JAR dependencies (well, with some default exclusion
> > patterns such as `log4j-*.jar` of course),
> > * create a POM that only references Log4j2 API.
> >
> > And of course Maven is not the only build tool on the market.
> > Contributors could submit plugins for other build tools.
> >
> > I think that `logging-log4j-tools` would be a nice name for a
> > repository, but it is taken. :-) Unless `log4j-changelog` is targeted
> > at a larger audience, in which case we can share. :-)
> >
> > Piotr
>
>
Re: Log4j Maven plugin
Posted by Ralph Goers <ra...@dslextreme.com>.
I think putting it in the tools repo would be fine.
Ralph
> On Jan 2, 2023, at 2:34 PM, Piotr P. Karwasz <pi...@gmail.com> wrote:
>
> Hi,
>
>> On Mon, 2 Jan 2023 at 10:08, Volkan Yazıcı <vo...@yazi.ci> wrote:
>> Agreed with the separate repository idea. I suggest moving it to the
>> `logging-log4j-maven-plugins` repository – you need an INFRA ticket to
>> create the repository.
>
> The Maven plugin will not be the only use-case of the code after I
> finish up the project.
>
> Since I am really bad at naming things, so maybe I'll describe the
> feature-set and someone will come with the ideal name. :-D
>
> I imagine one module that will weave bytecode (`log4j-instrument`) to:
> * convert classical Log4j2 API calls to LogBuilder calls with location,
> * convert other API calls (Log4j 1.2, JCL, SLF4J and especially
> `java.util.logging`) to Log4j2 API calls (LogBuilder with location for
> now),
> * implement `java.lang.instrument.ClassFileTransformer` so that we
> can use it to weave at runtime.
>
> A second module would be based on a refactoring library (Error Prone?
> called `log4j-errorprone`?) to:
> * convert classical Log4j2 API calls to LogBuilder (this simplifies
> weaving a lot),
> * convert other API calls to Log4j2 API,
> * check for common mistakes of new Log4j2 API adopters: e.g. `if
> (logger.isDebugEnabled() logger.debug(...);` or `logger.debug("Hello
> name " + name + ".")`.
>
> A Maven module would use `log4j-instrument` to:
> * weave a project's classes,
> * weave all JAR dependencies (well, with some default exclusion
> patterns such as `log4j-*.jar` of course),
> * create a POM that only references Log4j2 API.
>
> And of course Maven is not the only build tool on the market.
> Contributors could submit plugins for other build tools.
>
> I think that `logging-log4j-tools` would be a nice name for a
> repository, but it is taken. :-) Unless `log4j-changelog` is targeted
> at a larger audience, in which case we can share. :-)
>
> Piotr
Re: Log4j Maven plugin
Posted by "Piotr P. Karwasz" <pi...@gmail.com>.
Hi,
On Mon, 2 Jan 2023 at 10:08, Volkan Yazıcı <vo...@yazi.ci> wrote:
> Agreed with the separate repository idea. I suggest moving it to the
> `logging-log4j-maven-plugins` repository – you need an INFRA ticket to
> create the repository.
The Maven plugin will not be the only use-case of the code after I
finish up the project.
Since I am really bad at naming things, so maybe I'll describe the
feature-set and someone will come with the ideal name. :-D
I imagine one module that will weave bytecode (`log4j-instrument`) to:
* convert classical Log4j2 API calls to LogBuilder calls with location,
* convert other API calls (Log4j 1.2, JCL, SLF4J and especially
`java.util.logging`) to Log4j2 API calls (LogBuilder with location for
now),
* implement `java.lang.instrument.ClassFileTransformer` so that we
can use it to weave at runtime.
A second module would be based on a refactoring library (Error Prone?
called `log4j-errorprone`?) to:
* convert classical Log4j2 API calls to LogBuilder (this simplifies
weaving a lot),
* convert other API calls to Log4j2 API,
* check for common mistakes of new Log4j2 API adopters: e.g. `if
(logger.isDebugEnabled() logger.debug(...);` or `logger.debug("Hello
name " + name + ".")`.
A Maven module would use `log4j-instrument` to:
* weave a project's classes,
* weave all JAR dependencies (well, with some default exclusion
patterns such as `log4j-*.jar` of course),
* create a POM that only references Log4j2 API.
And of course Maven is not the only build tool on the market.
Contributors could submit plugins for other build tools.
I think that `logging-log4j-tools` would be a nice name for a
repository, but it is taken. :-) Unless `log4j-changelog` is targeted
at a larger audience, in which case we can share. :-)
Piotr
Re: Log4j Maven plugin
Posted by Volkan Yazıcı <vo...@yazi.ci>.
Great job, Piotr! 👏
Agreed with the separate repository idea. I suggest moving it to the
`logging-log4j-maven-plugins` repository – you need an INFRA ticket to
create the repository.
If we go that route, I strongly suggest incorporating following goodies
from the `logging-log4j-tools` repository:
- GitHub Actions `build.yml` (the repository path there needs to be
adapted)
- `/pom.xml` + `log4j-tools-parent/pom.xml` (Note that you need neither
a `-bom`, nor a `-parent` module! Adding sources to the root module should
suffice.)
- `CHANGELOG.adoc`
- `LICENSE.txt`
- `SECURITY.adoc`
- `spotless-license-header.txt`
- `.editorconfig`
- `.asf.yaml` (`github.description` needs to be updated here)
- `.gitignore`
- `.java-version`
I put a great deal of effort into their correctness and compactness, hence
my insistence on reusing them.
On Sun, Jan 1, 2023 at 10:00 PM Piotr P. Karwasz <pi...@gmail.com>
wrote:
> Hi all,
>
> After successfully merging my `LogBuilder` enhancement proposals, I
> have a Maven plugin ready[#], which hardcodes location information in
> a class's bytecode.
>
> As Ralph remarks in the PR, the right place for this module is not
> necessarily the `loggjing-log4j2` repository. I tend to agree with
> this: the lifecycle of this plugin is not necessarily linked with
> Log4j2. I have a long list of additional features I want to implement
> in the near future (like migrating SLF4J, JCL and JUL bytecode to
> Log4j2 API), but after that development will probably stop (apart from
> some bug fixes).
>
> Where do you think I should upload this code? What Maven groupId should I
> use?
>
> Piotr
>
> [#] https://github.com/apache/logging-log4j2/pull/1147
>
Re: Log4j Maven plugin
Posted by Matt Sicker <ma...@musigma.org>.
Link to the self-service page: https://selfserve.apache.org/
—
Matt Sicker
> On Jan 2, 2023, at 09:31, Ralph Goers <ra...@dslextreme.com> wrote:
>
> The groupId will almost always be “org.apache.logging.log4j” unless it isn’t specifically for log4j. In that case the last token would be the “category/subproject” it belongs to.
>
> As far as where to put it you have a few choices. Any PMC member can create a new repo so the only question is the name. All Logging Services repos must start with “logging-“. So you could use “logging-maven-plugins” under the assumption there may be multiple Maven plugins. More normal for plugins though is for each to be in its own repo. So “logging-location-maven-plugin” might be a decent repo name (although a bit long).
>
> Ralph
>
>> On Jan 1, 2023, at 1:59 PM, Piotr P. Karwasz <pi...@gmail.com> wrote:
>>
>> Hi all,
>>
>> After successfully merging my `LogBuilder` enhancement proposals, I
>> have a Maven plugin ready[#], which hardcodes location information in
>> a class's bytecode.
>>
>> As Ralph remarks in the PR, the right place for this module is not
>> necessarily the `loggjing-log4j2` repository. I tend to agree with
>> this: the lifecycle of this plugin is not necessarily linked with
>> Log4j2. I have a long list of additional features I want to implement
>> in the near future (like migrating SLF4J, JCL and JUL bytecode to
>> Log4j2 API), but after that development will probably stop (apart from
>> some bug fixes).
>>
>> Where do you think I should upload this code? What Maven groupId should I use?
>>
>> Piotr
>>
>> [#] https://github.com/apache/logging-log4j2/pull/1147
>
Re: Log4j Maven plugin
Posted by Ralph Goers <ra...@dslextreme.com>.
The groupId will almost always be “org.apache.logging.log4j” unless it isn’t specifically for log4j. In that case the last token would be the “category/subproject” it belongs to.
As far as where to put it you have a few choices. Any PMC member can create a new repo so the only question is the name. All Logging Services repos must start with “logging-“. So you could use “logging-maven-plugins” under the assumption there may be multiple Maven plugins. More normal for plugins though is for each to be in its own repo. So “logging-location-maven-plugin” might be a decent repo name (although a bit long).
Ralph
> On Jan 1, 2023, at 1:59 PM, Piotr P. Karwasz <pi...@gmail.com> wrote:
>
> Hi all,
>
> After successfully merging my `LogBuilder` enhancement proposals, I
> have a Maven plugin ready[#], which hardcodes location information in
> a class's bytecode.
>
> As Ralph remarks in the PR, the right place for this module is not
> necessarily the `loggjing-log4j2` repository. I tend to agree with
> this: the lifecycle of this plugin is not necessarily linked with
> Log4j2. I have a long list of additional features I want to implement
> in the near future (like migrating SLF4J, JCL and JUL bytecode to
> Log4j2 API), but after that development will probably stop (apart from
> some bug fixes).
>
> Where do you think I should upload this code? What Maven groupId should I use?
>
> Piotr
>
> [#] https://github.com/apache/logging-log4j2/pull/1147
Log4j Maven plugin
Posted by "Piotr P. Karwasz" <pi...@gmail.com>.
Hi all,
After successfully merging my `LogBuilder` enhancement proposals, I
have a Maven plugin ready[#], which hardcodes location information in
a class's bytecode.
As Ralph remarks in the PR, the right place for this module is not
necessarily the `loggjing-log4j2` repository. I tend to agree with
this: the lifecycle of this plugin is not necessarily linked with
Log4j2. I have a long list of additional features I want to implement
in the near future (like migrating SLF4J, JCL and JUL bytecode to
Log4j2 API), but after that development will probably stop (apart from
some bug fixes).
Where do you think I should upload this code? What Maven groupId should I use?
Piotr
[#] https://github.com/apache/logging-log4j2/pull/1147