You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@logging.apache.org by Gary Gregory <ga...@gmail.com> on 2018/01/21 15:55:01 UTC

[log4j] The shape of Log4j

Hi All,

I am writing this to in an effort to understand how we can manage and grow
Log4j. I use the term 'grow' not in the 'bigger is better' sense but more
in the maturing sense.

I am prompted to write this by Ralph's comment that log4j-core and the main
repo is too big and that releases take too long make, partially prompted by
my addition of a new module called log4j-jdbc-dbcp2 which currently
contains one small class with a dependency on Apache Commons DBCP 2.

I would like to express my gratitude at Ralph's efforts to revitalized
Log4j and performing most release management duties. Thank you Ralph!

I can see two main orthogonal issues:
- The size of the git repo.
- The size of the log4j-core module.

Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
another 'plugin' repo.

I really dislike this idea:
- The plugin repo has never been released. Not that one releases a repo but
you get the point.
- How do you keep things in sync between repos and code when we have no
official 'core' SPI.

For my money, we should keep _everything_ in one repo. Good enough for
Google, so good enough for me. What we release out of that repo is a
different story and what I would like to discuss next.

This is not the same as Maven plugins IMO but the case could be made for it
I suppose where a lot/most plugins live in their own repos. It is not the
same as with Maven IMO because our plugins rely on log4j-core and it's
guts, for which we only make loose compatibility guarantees -- as opposed
to log4j-core where we are strict(er). Maven OTOH, has a API for plugin
auteurs.

For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we have
no 'official' core SPI, so splitting it off into a separate repo would
greatly increase the risk of it falling out of sync. It is just so much
more easier to maintain when it is all in one repo.

My proposal is to:

- Put everything back into one repo (Chainsaw too?)
- Define a core SPI for plugin writers where we make some statement about
BC, more than the casual 'we try not break stuff.'
- Defining what Log4j project 'components' we have and release based on
that. For example, today, all of the main Log4j repo is one component with
many modules. Chainsaw would be another component of the Log4j project.
Maybe we need to redefine components: API, File, JDBC and so on. A
component can have one of more module
- Got for there.

Thoughts? Help me flush this out?

Thank you for reading!
Gary

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Thu, Jan 25, 2018 at 9:05 AM, Mikael Ståldal <mi...@apache.org> wrote:

> Yes, but it is unfortunate that the worry about the main repo size make us
> hesitate splitting out stuff from log4j-core into own modules.


And, with perfect timing:
https://issues.apache.org/jira/browse/LOG4J2-1863?focusedCommentId=16339319&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16339319

Gary


>
>
>
> On 2018-01-24 22:37, Ralph Goers wrote:
>
>> They are not really orthogonal.
>>
>> 1. Log4j core contains some stuff that isn’t really “core”
>> functionality., such as Kafka or JMS. These cause the core tests to take
>> longer than necessary.
>> 2. The size of the repo directly effects how long a build takes to run,
>> as does the number of components in core.
>>
>>
>>
>> On Jan 24, 2018, at 2:24 PM, Mikael Ståldal <mi...@apache.org> wrote:
>>>
>>> Yes, and it's unfortunate that they get conflated.
>>>
>>> On 2018-01-21 16:55, Gary Gregory wrote:
>>>
>>>> I can see two main orthogonal issues:
>>>> - The size of the git repo.
>>>> - The size of the log4j-core module.
>>>>
>>>
>>>
>>
>>
>

Re: [log4j] The shape of Log4j

Posted by Mikael Ståldal <mi...@apache.org>.
Yes, but it is unfortunate that the worry about the main repo size make 
us hesitate splitting out stuff from log4j-core into own modules.


On 2018-01-24 22:37, Ralph Goers wrote:
> They are not really orthogonal.
> 
> 1. Log4j core contains some stuff that isn’t really “core” functionality., such as Kafka or JMS. These cause the core tests to take longer than necessary.
> 2. The size of the repo directly effects how long a build takes to run, as does the number of components in core.
> 
> 
> 
>> On Jan 24, 2018, at 2:24 PM, Mikael Ståldal <mi...@apache.org> wrote:
>>
>> Yes, and it's unfortunate that they get conflated.
>>
>> On 2018-01-21 16:55, Gary Gregory wrote:
>>> I can see two main orthogonal issues:
>>> - The size of the git repo.
>>> - The size of the log4j-core module.
>>
> 
> 


Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
They are not really orthogonal.

1. Log4j core contains some stuff that isn’t really “core” functionality., such as Kafka or JMS. These cause the core tests to take longer than necessary.
2. The size of the repo directly effects how long a build takes to run, as does the number of components in core.



> On Jan 24, 2018, at 2:24 PM, Mikael Ståldal <mi...@apache.org> wrote:
> 
> Yes, and it's unfortunate that they get conflated.
> 
> On 2018-01-21 16:55, Gary Gregory wrote:
>> I can see two main orthogonal issues:
>> - The size of the git repo.
>> - The size of the log4j-core module.
> 



Re: [log4j] The shape of Log4j

Posted by Mikael Ståldal <mi...@apache.org>.
Yes, and it's unfortunate that they get conflated.

On 2018-01-21 16:55, Gary Gregory wrote:
> I can see two main orthogonal issues:
> - The size of the git repo.
> - The size of the log4j-core module.

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
I am always in favor of improving the documentation. The general rule is you never have enough and what you have is never good enough.

Ralph

> On Jan 22, 2018, at 12:49 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> On Mon, Jan 22, 2018 at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:
> 
>> This whole conversation just reminds me of all the times in the past I
>> suggested full modularization of the core API/SPI. I think such an effort
>> would make sense in a 3.0 release.
>> 
>> In 2.x, perhaps what we can do is define a stable plugin API based on what
>> we already have. Since we're still supporting Java 7, I'd suggest we make
>> abstract classes be the stable part of the API since we don't have default
>> methods on interfaces here.
>> 
> 
> At the very least and as a first step, we could make official the
> annotations required to build plugins. This would mean a serious effort at
> providing better Javadocs. For
> example, org.apache.logging.log4j.core.config.plugins.Plugin.deferChildren()
> has NO Javadoc. Actually, the first step would be to Javadoc all of the
> annotations as best we can, ideally to the point where folks would not need
> to go to the site; having examples in the Javadoc you can cut and paste for
> example.
> 
> Gary
> 
> 
>> 
>> And yes, the GitBox integration thing sounds neat, particularly for code
>> review, though that's a separate topic entirely.
>> 
>> On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com> wrote:
>> 
>>> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <ralph.goers@dslextreme.com
>>> 
>>> wrote:
>>> 
>>>> I am very much against the idea of a single repo. Yes, you can have
>>>> multiple projects in the repo but I am not sure how that can be sanely
>>>> released. I much prefer the model that Maven has taken. They are now
>>> using
>>>> gitbox [1] which seems to allow GitHub to be the primary repo.
>>> 
>>> 
>>> GitHub is a distracting tangent here IMO. I really like GitHub's code
>>> commenting feature, but it should not matter if we use Apache's Git
>> hosted
>>> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
>>> conversation... ;-)
>>> 
>>> Gary
>>> 
>>> 
>>>> Every Maven plugin is individually released. Scroll down the link below
>>> to
>>>> the Maven section and you can see all the plugin repos.
>>>> 
>>>> The upside to this is that it are:
>>>>        1. It is far easier to perform releases of the individual
>>>> components.
>>>>        2. It is much easier to accept plugin contributions.
>>>> The downsides are:
>>>>        1. A page like https://maven.apache.org/plugins/ <
>>>> https://maven.apache.org/plugins/> is needed to keep track of the
>> plugin
>>>> versions.
>>>>        2. It could make sense to have log4j-parent and log4j-bom
>>>> projects. The first to help keep the builds similar and the second to
>>> help
>>>> customers pick up the latest versions.
>>>> 
>>>> Ralph
>>>> 
>>>> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
>>>> repos/asf>
>>>> 
>>>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
>>>> wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> I am writing this to in an effort to understand how we can manage and
>>>> grow
>>>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense but
>>> more
>>>>> in the maturing sense.
>>>>> 
>>>>> I am prompted to write this by Ralph's comment that log4j-core and
>> the
>>>> main
>>>>> repo is too big and that releases take too long make, partially
>>> prompted
>>>> by
>>>>> my addition of a new module called log4j-jdbc-dbcp2 which currently
>>>>> contains one small class with a dependency on Apache Commons DBCP 2.
>>>>> 
>>>>> I would like to express my gratitude at Ralph's efforts to
>> revitalized
>>>>> Log4j and performing most release management duties. Thank you Ralph!
>>>>> 
>>>>> I can see two main orthogonal issues:
>>>>> - The size of the git repo.
>>>>> - The size of the log4j-core module.
>>>>> 
>>>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
>>>>> another 'plugin' repo.
>>>>> 
>>>>> I really dislike this idea:
>>>>> - The plugin repo has never been released. Not that one releases a
>> repo
>>>> but
>>>>> you get the point.
>>>>> - How do you keep things in sync between repos and code when we have
>> no
>>>>> official 'core' SPI.
>>>>> 
>>>>> For my money, we should keep _everything_ in one repo. Good enough
>> for
>>>>> Google, so good enough for me. What we release out of that repo is a
>>>>> different story and what I would like to discuss next.
>>>>> 
>>>>> This is not the same as Maven plugins IMO but the case could be made
>>> for
>>>> it
>>>>> I suppose where a lot/most plugins live in their own repos. It is not
>>> the
>>>>> same as with Maven IMO because our plugins rely on log4j-core and
>> it's
>>>>> guts, for which we only make loose compatibility guarantees -- as
>>> opposed
>>>>> to log4j-core where we are strict(er). Maven OTOH, has a API for
>> plugin
>>>>> auteurs.
>>>>> 
>>>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and
>> we
>>>> have
>>>>> no 'official' core SPI, so splitting it off into a separate repo
>> would
>>>>> greatly increase the risk of it falling out of sync. It is just so
>> much
>>>>> more easier to maintain when it is all in one repo.
>>>>> 
>>>>> My proposal is to:
>>>>> 
>>>>> - Put everything back into one repo (Chainsaw too?)
>>>>> - Define a core SPI for plugin writers where we make some statement
>>> about
>>>>> BC, more than the casual 'we try not break stuff.'
>>>>> - Defining what Log4j project 'components' we have and release based
>> on
>>>>> that. For example, today, all of the main Log4j repo is one component
>>>> with
>>>>> many modules. Chainsaw would be another component of the Log4j
>> project.
>>>>> Maybe we need to redefine components: API, File, JDBC and so on. A
>>>>> component can have one of more module
>>>>> - Got for there.
>>>>> 
>>>>> Thoughts? Help me flush this out?
>>>>> 
>>>>> Thank you for reading!
>>>>> Gary
>>>> 
>>>> 
>>> 
>> 
>> 
>> 
>> --
>> Matt Sicker <bo...@gmail.com>
>> 



Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 22, 2018 at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:

> This whole conversation just reminds me of all the times in the past I
> suggested full modularization of the core API/SPI. I think such an effort
> would make sense in a 3.0 release.
>
> In 2.x, perhaps what we can do is define a stable plugin API based on what
> we already have. Since we're still supporting Java 7, I'd suggest we make
> abstract classes be the stable part of the API since we don't have default
> methods on interfaces here.
>

At the very least and as a first step, we could make official the
annotations required to build plugins. This would mean a serious effort at
providing better Javadocs. For
example, org.apache.logging.log4j.core.config.plugins.Plugin.deferChildren()
has NO Javadoc. Actually, the first step would be to Javadoc all of the
annotations as best we can, ideally to the point where folks would not need
to go to the site; having examples in the Javadoc you can cut and paste for
example.

Gary


>
> And yes, the GitBox integration thing sounds neat, particularly for code
> review, though that's a separate topic entirely.
>
> On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com> wrote:
>
> > On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <ralph.goers@dslextreme.com
> >
> > wrote:
> >
> > > I am very much against the idea of a single repo. Yes, you can have
> > > multiple projects in the repo but I am not sure how that can be sanely
> > > released. I much prefer the model that Maven has taken. They are now
> > using
> > > gitbox [1] which seems to allow GitHub to be the primary repo.
> >
> >
> > GitHub is a distracting tangent here IMO. I really like GitHub's code
> > commenting feature, but it should not matter if we use Apache's Git
> hosted
> > repos or GitHub's. So I'm not sure why we are mixing GitHub in the
> > conversation... ;-)
> >
> > Gary
> >
> >
> > > Every Maven plugin is individually released. Scroll down the link below
> > to
> > > the Maven section and you can see all the plugin repos.
> > >
> > > The upside to this is that it are:
> > >         1. It is far easier to perform releases of the individual
> > > components.
> > >         2. It is much easier to accept plugin contributions.
> > > The downsides are:
> > >         1. A page like https://maven.apache.org/plugins/ <
> > > https://maven.apache.org/plugins/> is needed to keep track of the
> plugin
> > > versions.
> > >         2. It could make sense to have log4j-parent and log4j-bom
> > > projects. The first to help keep the builds similar and the second to
> > help
> > > customers pick up the latest versions.
> > >
> > > Ralph
> > >
> > > [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
> > > repos/asf>
> > >
> > > > On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
> > > wrote:
> > > >
> > > > Hi All,
> > > >
> > > > I am writing this to in an effort to understand how we can manage and
> > > grow
> > > > Log4j. I use the term 'grow' not in the 'bigger is better' sense but
> > more
> > > > in the maturing sense.
> > > >
> > > > I am prompted to write this by Ralph's comment that log4j-core and
> the
> > > main
> > > > repo is too big and that releases take too long make, partially
> > prompted
> > > by
> > > > my addition of a new module called log4j-jdbc-dbcp2 which currently
> > > > contains one small class with a dependency on Apache Commons DBCP 2.
> > > >
> > > > I would like to express my gratitude at Ralph's efforts to
> revitalized
> > > > Log4j and performing most release management duties. Thank you Ralph!
> > > >
> > > > I can see two main orthogonal issues:
> > > > - The size of the git repo.
> > > > - The size of the log4j-core module.
> > > >
> > > > Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
> > > > another 'plugin' repo.
> > > >
> > > > I really dislike this idea:
> > > > - The plugin repo has never been released. Not that one releases a
> repo
> > > but
> > > > you get the point.
> > > > - How do you keep things in sync between repos and code when we have
> no
> > > > official 'core' SPI.
> > > >
> > > > For my money, we should keep _everything_ in one repo. Good enough
> for
> > > > Google, so good enough for me. What we release out of that repo is a
> > > > different story and what I would like to discuss next.
> > > >
> > > > This is not the same as Maven plugins IMO but the case could be made
> > for
> > > it
> > > > I suppose where a lot/most plugins live in their own repos. It is not
> > the
> > > > same as with Maven IMO because our plugins rely on log4j-core and
> it's
> > > > guts, for which we only make loose compatibility guarantees -- as
> > opposed
> > > > to log4j-core where we are strict(er). Maven OTOH, has a API for
> plugin
> > > > auteurs.
> > > >
> > > > For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and
> we
> > > have
> > > > no 'official' core SPI, so splitting it off into a separate repo
> would
> > > > greatly increase the risk of it falling out of sync. It is just so
> much
> > > > more easier to maintain when it is all in one repo.
> > > >
> > > > My proposal is to:
> > > >
> > > > - Put everything back into one repo (Chainsaw too?)
> > > > - Define a core SPI for plugin writers where we make some statement
> > about
> > > > BC, more than the casual 'we try not break stuff.'
> > > > - Defining what Log4j project 'components' we have and release based
> on
> > > > that. For example, today, all of the main Log4j repo is one component
> > > with
> > > > many modules. Chainsaw would be another component of the Log4j
> project.
> > > > Maybe we need to redefine components: API, File, JDBC and so on. A
> > > > component can have one of more module
> > > > - Got for there.
> > > >
> > > > Thoughts? Help me flush this out?
> > > >
> > > > Thank you for reading!
> > > > Gary
> > >
> > >
> >
>
>
>
> --
> Matt Sicker <bo...@gmail.com>
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.

> On Jan 22, 2018, at 3:23 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> On Mon, Jan 22, 2018 at 2:58 PM, Matt Sicker <bo...@gmail.com> wrote:
> 
>> Using Maven profiles could help, though when tagging a release, if we
>> didn't release every module within, then it'd get out of sync anyways. Then
>> it would still require building every single component in each release,
>> even when said components haven't changed at all.
>> 
> 
> I just tried using profiles to define the main modules and extra ones. It
> works like a charm. This allows us to move modules around very easily,
> except that nothing is 'moving'. Further thoughts?

While you can create profiles to include certain modules for certain purposes (we currently only build the log4j-distribution when the “apache-release” profile is active) that definitely will not work for releases. The problem is the version numbers get incremented by the release plugin and trying to release only parts of the tree will get the version numbers all out of whack.

Ralph



Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
On 22 January 2018 at 16:44, Gary Gregory <ga...@gmail.com> wrote:

> Ah... but you COULD make it help in the release just as well but putting
> what you are releasing into the main profile! That's the beauty of it.
> Sure, you'd tag the whole thing as one. Presumably, you'd think the RM
> would make sure at least everything compiles ;-)
>

The problem here would be version numbers going out of sync. Unless we use
a tag name strategy like component-version instead of just version (and
include multiple tags of the same commit when multiple components are
released concurrently).

-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Scott Deboy <sc...@gmail.com>.
My recollection is vague, tied to pom machinations supporting a
similar separation for the 'log4j companions': outside of core log4j -
chainsaw, component, extras, receivers and zeroconf'

Some of the issues I recall having to deal with when we had the
generic 'component' module:

 - Some classes start out in one module, like the 'component' module,
but then it turns out something in core could benefit from that same
class, so it's pulled in to core.  This could result in a class rename
or package move in the process in order to match the structure of
core, naming conventions, etc.

If we did this only at a 'backward-compatibility breaking' boundary
that would be ok, it can just be confusing and unnecessary if we have
one module.

We also had the entire 'component' module sucked in to the 'receivers'
module when we felt that the separation wasn't useful, with Receivers
eventually getting pulled in to Chainsaw, where it didn't really
belong IMO..

So my worry is about things getting 'promoted and moved' because they
are now useful somewhere else, and the confusion that can ensue.


On 1/22/18, Gary Gregory <ga...@gmail.com> wrote:
> On Mon, Jan 22, 2018 at 3:54 PM, Ralph Goers <ra...@dslextreme.com>
> wrote:
>
>>
>>
>> > On Jan 22, 2018, at 3:44 PM, Gary Gregory <ga...@gmail.com>
>> wrote:
>> >
>> > On Mon, Jan 22, 2018 at 3:25 PM, Matt Sicker <bo...@gmail.com> wrote:
>> >
>> >> The profiles idea would help a lot for local development and making it
>> >> easier to run the relevant unit tests when modifying code. This
>> >> wouldn't
>> >> really help the release process, though, as we'd either need to
>> >> release
>> >> everything in the repo at once, or we'd end up with unreleased modules
>> >> being tagged on release of anything else in the same repo.
>> >>
>> >
>> > Ah... but you COULD make it help in the release just as well but
>> > putting
>> > what you are releasing into the main profile! That's the beauty of it.
>> > Sure, you'd tag the whole thing as one. Presumably, you'd think the RM
>> > would make sure at least everything compiles ;-)
>>
>> I don’t understand. If I perform a release for 2.11.0 and skip certain
>> modules they are not going to have their versions updated. They will be
>> pointing at 2.11.0-SNAPSHOT as a parent even though the parent has been
>> changed to 2.11.1-SNAPSHOT.
>>
>
> The way I see it is that you'd update the POM versions _as if_ you were to
> release and had released the modules that in fact are skipped. For example,
> if log4j-foo is not in the main-modules profile, it's versions would go
> from 2.11.0-SNAPSHOT to 2.11.1-SNAPSHOT. I suppose that might means an
> adjustment to a current step or an extra step.
>
> Gary
>
>
>>
>> If I assume all the pom versions are the same at 2.11.0-SNAPSHOT and only
>> want to release the mongodb plugin it will update the parent pom and
>> mongodb pom to the next version - let’s say 2.11.1-SNAPSHOT. I now cannot
>> release log4j as 2.11.0 since the parent pom has already released 2.11.0.
>> So you are going to have version numbers skipping all around.
>>
>> Ralph
>>
>>
>>
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
You are making a mountain out of a mole hill. If a plugin released for 2.8.2 doesn’t work with 2.11.0 we are doing something wrong. Users shouldn’t need to inspect anything. 

The problem with your premise is assuming these components require some specific version of log4j-core.

Ralph

> On Jan 22, 2018, at 5:10 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> On Mon, Jan 22, 2018 at 4:45 PM, Ralph Goers <ra...@dslextreme.com>
> wrote:
> 
>> 
>> 
>>> On Jan 22, 2018, at 4:26 PM, Gary Gregory <ga...@gmail.com>
>> wrote:
>>> 
>>> On Mon, Jan 22, 2018 at 4:22 PM, Ralph Goers <ralph.goers@dslextreme.com
>> <ma...@dslextreme.com>>
>>> wrote:
>>>> 
>>>> That process would be guaranteed to have things messed up and would make
>>>> users wonder why versions of jars were skipped.
>>>> 
>>> 
>>> How is that different than having some jars produced out of another repo
>>> and not releasing from that repo?
>> 
>> Every repo follows its own versioning scheme and is consistent with
>> itself. When you have a chart like Maven does users can easily deal with
>> figuring out what versions they want to use.
>> 
> 
> I'm sorry but this does not make sense to me
> 
> "Every repo follows its own versioning scheme."
> 
> That's part of the problem. When I produce jar version 2.0 out of some
> other repo, how do I know what version of Log4j it matches? By its POM
> dependency, right? So let's say I have a customer's stack that looks like
> this:
> - log4j-foo 1.2.3 depends on log4j-core 2.8.2
> - log4j-bar 3.2.1 depends on log4j-core 2.11.0
> 
> Result: You end up with multiple log4-core jar files on your class path
> a.k.a Jar hell. The top level app build is now required to force a specific
> version, and exclude the rest, clearly not a selling point for Log4j.
> 
> Because all of our 'extra' plugins (JMS, JDBC, MongoDB, Flume, and so)
> depend deeply on a specific log4j-core version, I am now at great risk of
> not being able to assemble a working stack.
> 
> The chance of jar hell is greatly reduced if all the plugins come out with
> the same version: you know they all match up.
> 
> "When you have a chart like Maven does users can easily deal with figuring
> out what versions they want to use."
> 
> That makes no sense to me either: The page https://maven.apache.org/plugins/
> is just a list of plugins and their latest version. That basically tells me
> the same thing as mvn
> versions:display-dependency-updates|display-plugin-updates. How does that
> help picking version x vs. y?
> 
> Sorry to be so negative but I do not see a workable plan that is better
> than what we have now. By a long shot :-( But I like that we are hashing
> things out. Thank you for that :-)
> 
> Gary
> 
> Gary
> 
> 
>> 
>> Ralph
>> 
>> 



Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.

> On Jan 22, 2018, at 5:10 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> On Mon, Jan 22, 2018 at 4:45 PM, Ralph Goers <ra...@dslextreme.com>
> wrote:
>> 
> "When you have a chart like Maven does users can easily deal with figuring
> out what versions they want to use."
> 
> That makes no sense to me either: The page https://maven.apache.org/plugins/
> is just a list of plugins and their latest version. That basically tells me
> the same thing as mvn
> versions:display-dependency-updates|display-plugin-updates. How does that
> help picking version x vs. y?

You pick the plugin version based on what you need from it. Maven plugins generally only require 3.x, not a specific Maven version. The only times I have had problems is when the plugin itself had problems.

> 
> Sorry to be so negative but I do not see a workable plan that is better
> than what we have now. By a long shot :-( But I like that we are hashing
> things out. Thank you for that :-)
> 

What we have now isn’t working for me, so almost any alternative would be better.

Ralph


Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 22, 2018 at 4:45 PM, Ralph Goers <ra...@dslextreme.com>
wrote:

>
>
> > On Jan 22, 2018, at 4:26 PM, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > On Mon, Jan 22, 2018 at 4:22 PM, Ralph Goers <ralph.goers@dslextreme.com
> <ma...@dslextreme.com>>
> > wrote:
> >>
> >> That process would be guaranteed to have things messed up and would make
> >> users wonder why versions of jars were skipped.
> >>
> >
> > How is that different than having some jars produced out of another repo
> > and not releasing from that repo?
>
> Every repo follows its own versioning scheme and is consistent with
> itself. When you have a chart like Maven does users can easily deal with
> figuring out what versions they want to use.
>

I'm sorry but this does not make sense to me

"Every repo follows its own versioning scheme."

That's part of the problem. When I produce jar version 2.0 out of some
other repo, how do I know what version of Log4j it matches? By its POM
dependency, right? So let's say I have a customer's stack that looks like
this:
- log4j-foo 1.2.3 depends on log4j-core 2.8.2
- log4j-bar 3.2.1 depends on log4j-core 2.11.0

Result: You end up with multiple log4-core jar files on your class path
a.k.a Jar hell. The top level app build is now required to force a specific
version, and exclude the rest, clearly not a selling point for Log4j.

Because all of our 'extra' plugins (JMS, JDBC, MongoDB, Flume, and so)
depend deeply on a specific log4j-core version, I am now at great risk of
not being able to assemble a working stack.

The chance of jar hell is greatly reduced if all the plugins come out with
the same version: you know they all match up.

"When you have a chart like Maven does users can easily deal with figuring
out what versions they want to use."

That makes no sense to me either: The page https://maven.apache.org/plugins/
is just a list of plugins and their latest version. That basically tells me
the same thing as mvn
versions:display-dependency-updates|display-plugin-updates. How does that
help picking version x vs. y?

Sorry to be so negative but I do not see a workable plan that is better
than what we have now. By a long shot :-( But I like that we are hashing
things out. Thank you for that :-)

Gary

Gary


>
> Ralph
>
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.

> On Jan 22, 2018, at 4:26 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> On Mon, Jan 22, 2018 at 4:22 PM, Ralph Goers <ralph.goers@dslextreme.com <ma...@dslextreme.com>>
> wrote:
>> 
>> That process would be guaranteed to have things messed up and would make
>> users wonder why versions of jars were skipped.
>> 
> 
> How is that different than having some jars produced out of another repo
> and not releasing from that repo?

Every repo follows its own versioning scheme and is consistent with itself. When you have a chart like Maven does users can easily deal with figuring out what versions they want to use.

Ralph


Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 22, 2018 at 4:22 PM, Ralph Goers <ra...@dslextreme.com>
wrote:

>
>
> > On Jan 22, 2018, at 4:02 PM, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > On Mon, Jan 22, 2018 at 3:54 PM, Ralph Goers <ralph.goers@dslextreme.com
> >
> > wrote:
> >
> >>
> >>
> >>> On Jan 22, 2018, at 3:44 PM, Gary Gregory <ga...@gmail.com>
> >> wrote:
> >>>
> >>> On Mon, Jan 22, 2018 at 3:25 PM, Matt Sicker <bo...@gmail.com> wrote:
> >>>
> >>>> The profiles idea would help a lot for local development and making it
> >>>> easier to run the relevant unit tests when modifying code. This
> wouldn't
> >>>> really help the release process, though, as we'd either need to
> release
> >>>> everything in the repo at once, or we'd end up with unreleased modules
> >>>> being tagged on release of anything else in the same repo.
> >>>>
> >>>
> >>> Ah... but you COULD make it help in the release just as well but
> putting
> >>> what you are releasing into the main profile! That's the beauty of it.
> >>> Sure, you'd tag the whole thing as one. Presumably, you'd think the RM
> >>> would make sure at least everything compiles ;-)
> >>
> >> I don’t understand. If I perform a release for 2.11.0 and skip certain
> >> modules they are not going to have their versions updated. They will be
> >> pointing at 2.11.0-SNAPSHOT as a parent even though the parent has been
> >> changed to 2.11.1-SNAPSHOT.
> >>
> >
> > The way I see it is that you'd update the POM versions _as if_ you were
> to
> > release and had released the modules that in fact are skipped. For
> example,
> > if log4j-foo is not in the main-modules profile, it's versions would go
> > from 2.11.0-SNAPSHOT to 2.11.1-SNAPSHOT. I suppose that might means an
> > adjustment to a current step or an extra step.
>
> That process would be guaranteed to have things messed up and would make
> users wonder why versions of jars were skipped.
>

How is that different than having some jars produced out of another repo
and not releasing from that repo?

Gary

>
> Ralph
>
>
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.

> On Jan 22, 2018, at 4:02 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> On Mon, Jan 22, 2018 at 3:54 PM, Ralph Goers <ra...@dslextreme.com>
> wrote:
> 
>> 
>> 
>>> On Jan 22, 2018, at 3:44 PM, Gary Gregory <ga...@gmail.com>
>> wrote:
>>> 
>>> On Mon, Jan 22, 2018 at 3:25 PM, Matt Sicker <bo...@gmail.com> wrote:
>>> 
>>>> The profiles idea would help a lot for local development and making it
>>>> easier to run the relevant unit tests when modifying code. This wouldn't
>>>> really help the release process, though, as we'd either need to release
>>>> everything in the repo at once, or we'd end up with unreleased modules
>>>> being tagged on release of anything else in the same repo.
>>>> 
>>> 
>>> Ah... but you COULD make it help in the release just as well but putting
>>> what you are releasing into the main profile! That's the beauty of it.
>>> Sure, you'd tag the whole thing as one. Presumably, you'd think the RM
>>> would make sure at least everything compiles ;-)
>> 
>> I don’t understand. If I perform a release for 2.11.0 and skip certain
>> modules they are not going to have their versions updated. They will be
>> pointing at 2.11.0-SNAPSHOT as a parent even though the parent has been
>> changed to 2.11.1-SNAPSHOT.
>> 
> 
> The way I see it is that you'd update the POM versions _as if_ you were to
> release and had released the modules that in fact are skipped. For example,
> if log4j-foo is not in the main-modules profile, it's versions would go
> from 2.11.0-SNAPSHOT to 2.11.1-SNAPSHOT. I suppose that might means an
> adjustment to a current step or an extra step.

That process would be guaranteed to have things messed up and would make users wonder why versions of jars were skipped.

Ralph



Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 22, 2018 at 3:54 PM, Ralph Goers <ra...@dslextreme.com>
wrote:

>
>
> > On Jan 22, 2018, at 3:44 PM, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > On Mon, Jan 22, 2018 at 3:25 PM, Matt Sicker <bo...@gmail.com> wrote:
> >
> >> The profiles idea would help a lot for local development and making it
> >> easier to run the relevant unit tests when modifying code. This wouldn't
> >> really help the release process, though, as we'd either need to release
> >> everything in the repo at once, or we'd end up with unreleased modules
> >> being tagged on release of anything else in the same repo.
> >>
> >
> > Ah... but you COULD make it help in the release just as well but putting
> > what you are releasing into the main profile! That's the beauty of it.
> > Sure, you'd tag the whole thing as one. Presumably, you'd think the RM
> > would make sure at least everything compiles ;-)
>
> I don’t understand. If I perform a release for 2.11.0 and skip certain
> modules they are not going to have their versions updated. They will be
> pointing at 2.11.0-SNAPSHOT as a parent even though the parent has been
> changed to 2.11.1-SNAPSHOT.
>

The way I see it is that you'd update the POM versions _as if_ you were to
release and had released the modules that in fact are skipped. For example,
if log4j-foo is not in the main-modules profile, it's versions would go
from 2.11.0-SNAPSHOT to 2.11.1-SNAPSHOT. I suppose that might means an
adjustment to a current step or an extra step.

Gary


>
> If I assume all the pom versions are the same at 2.11.0-SNAPSHOT and only
> want to release the mongodb plugin it will update the parent pom and
> mongodb pom to the next version - let’s say 2.11.1-SNAPSHOT. I now cannot
> release log4j as 2.11.0 since the parent pom has already released 2.11.0.
> So you are going to have version numbers skipping all around.
>
> Ralph
>
>
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.

> On Jan 22, 2018, at 3:44 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> On Mon, Jan 22, 2018 at 3:25 PM, Matt Sicker <bo...@gmail.com> wrote:
> 
>> The profiles idea would help a lot for local development and making it
>> easier to run the relevant unit tests when modifying code. This wouldn't
>> really help the release process, though, as we'd either need to release
>> everything in the repo at once, or we'd end up with unreleased modules
>> being tagged on release of anything else in the same repo.
>> 
> 
> Ah... but you COULD make it help in the release just as well but putting
> what you are releasing into the main profile! That's the beauty of it.
> Sure, you'd tag the whole thing as one. Presumably, you'd think the RM
> would make sure at least everything compiles ;-)

I don’t understand. If I perform a release for 2.11.0 and skip certain modules they are not going to have their versions updated. They will be pointing at 2.11.0-SNAPSHOT as a parent even though the parent has been changed to 2.11.1-SNAPSHOT. 

If I assume all the pom versions are the same at 2.11.0-SNAPSHOT and only want to release the mongodb plugin it will update the parent pom and mongodb pom to the next version - let’s say 2.11.1-SNAPSHOT. I now cannot release log4j as 2.11.0 since the parent pom has already released 2.11.0. So you are going to have version numbers skipping all around.

Ralph



Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 22, 2018 at 3:25 PM, Matt Sicker <bo...@gmail.com> wrote:

> The profiles idea would help a lot for local development and making it
> easier to run the relevant unit tests when modifying code. This wouldn't
> really help the release process, though, as we'd either need to release
> everything in the repo at once, or we'd end up with unreleased modules
> being tagged on release of anything else in the same repo.
>

Ah... but you COULD make it help in the release just as well but putting
what you are releasing into the main profile! That's the beauty of it.
Sure, you'd tag the whole thing as one. Presumably, you'd think the RM
would make sure at least everything compiles ;-)

Gary


>
> On 22 January 2018 at 16:23, Gary Gregory <ga...@gmail.com> wrote:
>
> > On Mon, Jan 22, 2018 at 2:58 PM, Matt Sicker <bo...@gmail.com> wrote:
> >
> > > Using Maven profiles could help, though when tagging a release, if we
> > > didn't release every module within, then it'd get out of sync anyways.
> > Then
> > > it would still require building every single component in each release,
> > > even when said components haven't changed at all.
> > >
> >
> > I just tried using profiles to define the main modules and extra ones. It
> > works like a charm. This allows us to move modules around very easily,
> > except that nothing is 'moving'. Further thoughts?
> >
> > Gary
> >
> >
> > >
> > > On 22 January 2018 at 15:50, Gary Gregory <ga...@gmail.com>
> > wrote:
> > >
> > > > Coming back around to the idea of One Repo to Rule Them All*TM* ...
> > what
> > > if
> > > > we used Maven profiles to separate out the "main" build from "other"
> > > > builds?
> > > >
> > > > All that is needed to move modules from one build to another is just
> to
> > > > edit the profile!
> > > >
> > > > Gary
> > > >
> > > > On Mon, Jan 22, 2018 at 12:48 PM, Ralph Goers <
> > > ralph.goers@dslextreme.com>
> > > > wrote:
> > > >
> > > > > I don’t see why this work would require 3.0 as we aren’t planning
> on
> > > > > breaking any contracts to do this.
> > > > >
> > > > > As I’ve said, I am not tied to each plugin having its own repo. I
> am
> > OK
> > > > > with these options; a) each plugin has its own repo and site and is
> > > > > released independently, b) we use the plugins repo and move all the
> > > more
> > > > > lightly used components there. It would have its own site, or c) we
> > > group
> > > > > plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume
> or
> > > > > similar) with each having its own site.
> > > > >
> > > > > Obviously I do not consider continuing to add new plugins to the
> main
> > > > > build as one of the options.
> > > > >
> > > > > Ralph
> > > > >
> > > > > > On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com>
> > wrote:
> > > > > >
> > > > > > This whole conversation just reminds me of all the times in the
> > past
> > > I
> > > > > > suggested full modularization of the core API/SPI. I think such
> an
> > > > effort
> > > > > > would make sense in a 3.0 release.
> > > > > >
> > > > > > In 2.x, perhaps what we can do is define a stable plugin API
> based
> > on
> > > > > what
> > > > > > we already have. Since we're still supporting Java 7, I'd suggest
> > we
> > > > make
> > > > > > abstract classes be the stable part of the API since we don't
> have
> > > > > default
> > > > > > methods on interfaces here.
> > > > > >
> > > > > > And yes, the GitBox integration thing sounds neat, particularly
> for
> > > > code
> > > > > > review, though that's a separate topic entirely.
> > > > > >
> > > > > > On 22 January 2018 at 00:37, Gary Gregory <
> garydgregory@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > >> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <
> > > > > ralph.goers@dslextreme.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> I am very much against the idea of a single repo. Yes, you can
> > have
> > > > > >>> multiple projects in the repo but I am not sure how that can be
> > > > sanely
> > > > > >>> released. I much prefer the model that Maven has taken. They
> are
> > > now
> > > > > >> using
> > > > > >>> gitbox [1] which seems to allow GitHub to be the primary repo.
> > > > > >>
> > > > > >>
> > > > > >> GitHub is a distracting tangent here IMO. I really like GitHub's
> > > code
> > > > > >> commenting feature, but it should not matter if we use Apache's
> > Git
> > > > > hosted
> > > > > >> repos or GitHub's. So I'm not sure why we are mixing GitHub in
> the
> > > > > >> conversation... ;-)
> > > > > >>
> > > > > >> Gary
> > > > > >>
> > > > > >>
> > > > > >>> Every Maven plugin is individually released. Scroll down the
> link
> > > > below
> > > > > >> to
> > > > > >>> the Maven section and you can see all the plugin repos.
> > > > > >>>
> > > > > >>> The upside to this is that it are:
> > > > > >>>        1. It is far easier to perform releases of the
> individual
> > > > > >>> components.
> > > > > >>>        2. It is much easier to accept plugin contributions.
> > > > > >>> The downsides are:
> > > > > >>>        1. A page like https://maven.apache.org/plugins/ <
> > > > > >>> https://maven.apache.org/plugins/> is needed to keep track of
> > the
> > > > > plugin
> > > > > >>> versions.
> > > > > >>>        2. It could make sense to have log4j-parent and
> log4j-bom
> > > > > >>> projects. The first to help keep the builds similar and the
> > second
> > > to
> > > > > >> help
> > > > > >>> customers pick up the latest versions.
> > > > > >>>
> > > > > >>> Ralph
> > > > > >>>
> > > > > >>> [1] https://gitbox.apache.org/repos/asf <
> > > https://gitbox.apache.org/
> > > > > >>> repos/asf>
> > > > > >>>
> > > > > >>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <
> > garydgregory@gmail.com
> > > >
> > > > > >>> wrote:
> > > > > >>>>
> > > > > >>>> Hi All,
> > > > > >>>>
> > > > > >>>> I am writing this to in an effort to understand how we can
> > manage
> > > > and
> > > > > >>> grow
> > > > > >>>> Log4j. I use the term 'grow' not in the 'bigger is better'
> sense
> > > but
> > > > > >> more
> > > > > >>>> in the maturing sense.
> > > > > >>>>
> > > > > >>>> I am prompted to write this by Ralph's comment that log4j-core
> > and
> > > > the
> > > > > >>> main
> > > > > >>>> repo is too big and that releases take too long make,
> partially
> > > > > >> prompted
> > > > > >>> by
> > > > > >>>> my addition of a new module called log4j-jdbc-dbcp2 which
> > > currently
> > > > > >>>> contains one small class with a dependency on Apache Commons
> > DBCP
> > > 2.
> > > > > >>>>
> > > > > >>>> I would like to express my gratitude at Ralph's efforts to
> > > > revitalized
> > > > > >>>> Log4j and performing most release management duties. Thank you
> > > > Ralph!
> > > > > >>>>
> > > > > >>>> I can see two main orthogonal issues:
> > > > > >>>> - The size of the git repo.
> > > > > >>>> - The size of the log4j-core module.
> > > > > >>>>
> > > > > >>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be
> > moved
> > > > to
> > > > > >>>> another 'plugin' repo.
> > > > > >>>>
> > > > > >>>> I really dislike this idea:
> > > > > >>>> - The plugin repo has never been released. Not that one
> > releases a
> > > > > repo
> > > > > >>> but
> > > > > >>>> you get the point.
> > > > > >>>> - How do you keep things in sync between repos and code when
> we
> > > have
> > > > > no
> > > > > >>>> official 'core' SPI.
> > > > > >>>>
> > > > > >>>> For my money, we should keep _everything_ in one repo. Good
> > enough
> > > > for
> > > > > >>>> Google, so good enough for me. What we release out of that
> repo
> > > is a
> > > > > >>>> different story and what I would like to discuss next.
> > > > > >>>>
> > > > > >>>> This is not the same as Maven plugins IMO but the case could
> be
> > > made
> > > > > >> for
> > > > > >>> it
> > > > > >>>> I suppose where a lot/most plugins live in their own repos. It
> > is
> > > > not
> > > > > >> the
> > > > > >>>> same as with Maven IMO because our plugins rely on log4j-core
> > and
> > > > it's
> > > > > >>>> guts, for which we only make loose compatibility guarantees --
> > as
> > > > > >> opposed
> > > > > >>>> to log4j-core where we are strict(er). Maven OTOH, has a API
> for
> > > > > plugin
> > > > > >>>> auteurs.
> > > > > >>>>
> > > > > >>>> For example, log4j-jdbc-dbcp2 replies on the guts of
> log4j-core
> > > and
> > > > we
> > > > > >>> have
> > > > > >>>> no 'official' core SPI, so splitting it off into a separate
> repo
> > > > would
> > > > > >>>> greatly increase the risk of it falling out of sync. It is
> just
> > so
> > > > > much
> > > > > >>>> more easier to maintain when it is all in one repo.
> > > > > >>>>
> > > > > >>>> My proposal is to:
> > > > > >>>>
> > > > > >>>> - Put everything back into one repo (Chainsaw too?)
> > > > > >>>> - Define a core SPI for plugin writers where we make some
> > > statement
> > > > > >> about
> > > > > >>>> BC, more than the casual 'we try not break stuff.'
> > > > > >>>> - Defining what Log4j project 'components' we have and release
> > > based
> > > > > on
> > > > > >>>> that. For example, today, all of the main Log4j repo is one
> > > > component
> > > > > >>> with
> > > > > >>>> many modules. Chainsaw would be another component of the Log4j
> > > > > project.
> > > > > >>>> Maybe we need to redefine components: API, File, JDBC and so
> > on. A
> > > > > >>>> component can have one of more module
> > > > > >>>> - Got for there.
> > > > > >>>>
> > > > > >>>> Thoughts? Help me flush this out?
> > > > > >>>>
> > > > > >>>> Thank you for reading!
> > > > > >>>> Gary
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Matt Sicker <bo...@gmail.com>
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Matt Sicker <bo...@gmail.com>
> > >
> >
>
>
>
> --
> Matt Sicker <bo...@gmail.com>
>

Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
The profiles idea would help a lot for local development and making it
easier to run the relevant unit tests when modifying code. This wouldn't
really help the release process, though, as we'd either need to release
everything in the repo at once, or we'd end up with unreleased modules
being tagged on release of anything else in the same repo.

On 22 January 2018 at 16:23, Gary Gregory <ga...@gmail.com> wrote:

> On Mon, Jan 22, 2018 at 2:58 PM, Matt Sicker <bo...@gmail.com> wrote:
>
> > Using Maven profiles could help, though when tagging a release, if we
> > didn't release every module within, then it'd get out of sync anyways.
> Then
> > it would still require building every single component in each release,
> > even when said components haven't changed at all.
> >
>
> I just tried using profiles to define the main modules and extra ones. It
> works like a charm. This allows us to move modules around very easily,
> except that nothing is 'moving'. Further thoughts?
>
> Gary
>
>
> >
> > On 22 January 2018 at 15:50, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > > Coming back around to the idea of One Repo to Rule Them All*TM* ...
> what
> > if
> > > we used Maven profiles to separate out the "main" build from "other"
> > > builds?
> > >
> > > All that is needed to move modules from one build to another is just to
> > > edit the profile!
> > >
> > > Gary
> > >
> > > On Mon, Jan 22, 2018 at 12:48 PM, Ralph Goers <
> > ralph.goers@dslextreme.com>
> > > wrote:
> > >
> > > > I don’t see why this work would require 3.0 as we aren’t planning on
> > > > breaking any contracts to do this.
> > > >
> > > > As I’ve said, I am not tied to each plugin having its own repo. I am
> OK
> > > > with these options; a) each plugin has its own repo and site and is
> > > > released independently, b) we use the plugins repo and move all the
> > more
> > > > lightly used components there. It would have its own site, or c) we
> > group
> > > > plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
> > > > similar) with each having its own site.
> > > >
> > > > Obviously I do not consider continuing to add new plugins to the main
> > > > build as one of the options.
> > > >
> > > > Ralph
> > > >
> > > > > On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com>
> wrote:
> > > > >
> > > > > This whole conversation just reminds me of all the times in the
> past
> > I
> > > > > suggested full modularization of the core API/SPI. I think such an
> > > effort
> > > > > would make sense in a 3.0 release.
> > > > >
> > > > > In 2.x, perhaps what we can do is define a stable plugin API based
> on
> > > > what
> > > > > we already have. Since we're still supporting Java 7, I'd suggest
> we
> > > make
> > > > > abstract classes be the stable part of the API since we don't have
> > > > default
> > > > > methods on interfaces here.
> > > > >
> > > > > And yes, the GitBox integration thing sounds neat, particularly for
> > > code
> > > > > review, though that's a separate topic entirely.
> > > > >
> > > > > On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com>
> > > > wrote:
> > > > >
> > > > >> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <
> > > > ralph.goers@dslextreme.com>
> > > > >> wrote:
> > > > >>
> > > > >>> I am very much against the idea of a single repo. Yes, you can
> have
> > > > >>> multiple projects in the repo but I am not sure how that can be
> > > sanely
> > > > >>> released. I much prefer the model that Maven has taken. They are
> > now
> > > > >> using
> > > > >>> gitbox [1] which seems to allow GitHub to be the primary repo.
> > > > >>
> > > > >>
> > > > >> GitHub is a distracting tangent here IMO. I really like GitHub's
> > code
> > > > >> commenting feature, but it should not matter if we use Apache's
> Git
> > > > hosted
> > > > >> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
> > > > >> conversation... ;-)
> > > > >>
> > > > >> Gary
> > > > >>
> > > > >>
> > > > >>> Every Maven plugin is individually released. Scroll down the link
> > > below
> > > > >> to
> > > > >>> the Maven section and you can see all the plugin repos.
> > > > >>>
> > > > >>> The upside to this is that it are:
> > > > >>>        1. It is far easier to perform releases of the individual
> > > > >>> components.
> > > > >>>        2. It is much easier to accept plugin contributions.
> > > > >>> The downsides are:
> > > > >>>        1. A page like https://maven.apache.org/plugins/ <
> > > > >>> https://maven.apache.org/plugins/> is needed to keep track of
> the
> > > > plugin
> > > > >>> versions.
> > > > >>>        2. It could make sense to have log4j-parent and log4j-bom
> > > > >>> projects. The first to help keep the builds similar and the
> second
> > to
> > > > >> help
> > > > >>> customers pick up the latest versions.
> > > > >>>
> > > > >>> Ralph
> > > > >>>
> > > > >>> [1] https://gitbox.apache.org/repos/asf <
> > https://gitbox.apache.org/
> > > > >>> repos/asf>
> > > > >>>
> > > > >>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <
> garydgregory@gmail.com
> > >
> > > > >>> wrote:
> > > > >>>>
> > > > >>>> Hi All,
> > > > >>>>
> > > > >>>> I am writing this to in an effort to understand how we can
> manage
> > > and
> > > > >>> grow
> > > > >>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense
> > but
> > > > >> more
> > > > >>>> in the maturing sense.
> > > > >>>>
> > > > >>>> I am prompted to write this by Ralph's comment that log4j-core
> and
> > > the
> > > > >>> main
> > > > >>>> repo is too big and that releases take too long make, partially
> > > > >> prompted
> > > > >>> by
> > > > >>>> my addition of a new module called log4j-jdbc-dbcp2 which
> > currently
> > > > >>>> contains one small class with a dependency on Apache Commons
> DBCP
> > 2.
> > > > >>>>
> > > > >>>> I would like to express my gratitude at Ralph's efforts to
> > > revitalized
> > > > >>>> Log4j and performing most release management duties. Thank you
> > > Ralph!
> > > > >>>>
> > > > >>>> I can see two main orthogonal issues:
> > > > >>>> - The size of the git repo.
> > > > >>>> - The size of the log4j-core module.
> > > > >>>>
> > > > >>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be
> moved
> > > to
> > > > >>>> another 'plugin' repo.
> > > > >>>>
> > > > >>>> I really dislike this idea:
> > > > >>>> - The plugin repo has never been released. Not that one
> releases a
> > > > repo
> > > > >>> but
> > > > >>>> you get the point.
> > > > >>>> - How do you keep things in sync between repos and code when we
> > have
> > > > no
> > > > >>>> official 'core' SPI.
> > > > >>>>
> > > > >>>> For my money, we should keep _everything_ in one repo. Good
> enough
> > > for
> > > > >>>> Google, so good enough for me. What we release out of that repo
> > is a
> > > > >>>> different story and what I would like to discuss next.
> > > > >>>>
> > > > >>>> This is not the same as Maven plugins IMO but the case could be
> > made
> > > > >> for
> > > > >>> it
> > > > >>>> I suppose where a lot/most plugins live in their own repos. It
> is
> > > not
> > > > >> the
> > > > >>>> same as with Maven IMO because our plugins rely on log4j-core
> and
> > > it's
> > > > >>>> guts, for which we only make loose compatibility guarantees --
> as
> > > > >> opposed
> > > > >>>> to log4j-core where we are strict(er). Maven OTOH, has a API for
> > > > plugin
> > > > >>>> auteurs.
> > > > >>>>
> > > > >>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core
> > and
> > > we
> > > > >>> have
> > > > >>>> no 'official' core SPI, so splitting it off into a separate repo
> > > would
> > > > >>>> greatly increase the risk of it falling out of sync. It is just
> so
> > > > much
> > > > >>>> more easier to maintain when it is all in one repo.
> > > > >>>>
> > > > >>>> My proposal is to:
> > > > >>>>
> > > > >>>> - Put everything back into one repo (Chainsaw too?)
> > > > >>>> - Define a core SPI for plugin writers where we make some
> > statement
> > > > >> about
> > > > >>>> BC, more than the casual 'we try not break stuff.'
> > > > >>>> - Defining what Log4j project 'components' we have and release
> > based
> > > > on
> > > > >>>> that. For example, today, all of the main Log4j repo is one
> > > component
> > > > >>> with
> > > > >>>> many modules. Chainsaw would be another component of the Log4j
> > > > project.
> > > > >>>> Maybe we need to redefine components: API, File, JDBC and so
> on. A
> > > > >>>> component can have one of more module
> > > > >>>> - Got for there.
> > > > >>>>
> > > > >>>> Thoughts? Help me flush this out?
> > > > >>>>
> > > > >>>> Thank you for reading!
> > > > >>>> Gary
> > > > >>>
> > > > >>>
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Matt Sicker <bo...@gmail.com>
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Matt Sicker <bo...@gmail.com>
> >
>



-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 22, 2018 at 2:58 PM, Matt Sicker <bo...@gmail.com> wrote:

> Using Maven profiles could help, though when tagging a release, if we
> didn't release every module within, then it'd get out of sync anyways. Then
> it would still require building every single component in each release,
> even when said components haven't changed at all.
>

I just tried using profiles to define the main modules and extra ones. It
works like a charm. This allows us to move modules around very easily,
except that nothing is 'moving'. Further thoughts?

Gary


>
> On 22 January 2018 at 15:50, Gary Gregory <ga...@gmail.com> wrote:
>
> > Coming back around to the idea of One Repo to Rule Them All*TM* ... what
> if
> > we used Maven profiles to separate out the "main" build from "other"
> > builds?
> >
> > All that is needed to move modules from one build to another is just to
> > edit the profile!
> >
> > Gary
> >
> > On Mon, Jan 22, 2018 at 12:48 PM, Ralph Goers <
> ralph.goers@dslextreme.com>
> > wrote:
> >
> > > I don’t see why this work would require 3.0 as we aren’t planning on
> > > breaking any contracts to do this.
> > >
> > > As I’ve said, I am not tied to each plugin having its own repo. I am OK
> > > with these options; a) each plugin has its own repo and site and is
> > > released independently, b) we use the plugins repo and move all the
> more
> > > lightly used components there. It would have its own site, or c) we
> group
> > > plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
> > > similar) with each having its own site.
> > >
> > > Obviously I do not consider continuing to add new plugins to the main
> > > build as one of the options.
> > >
> > > Ralph
> > >
> > > > On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:
> > > >
> > > > This whole conversation just reminds me of all the times in the past
> I
> > > > suggested full modularization of the core API/SPI. I think such an
> > effort
> > > > would make sense in a 3.0 release.
> > > >
> > > > In 2.x, perhaps what we can do is define a stable plugin API based on
> > > what
> > > > we already have. Since we're still supporting Java 7, I'd suggest we
> > make
> > > > abstract classes be the stable part of the API since we don't have
> > > default
> > > > methods on interfaces here.
> > > >
> > > > And yes, the GitBox integration thing sounds neat, particularly for
> > code
> > > > review, though that's a separate topic entirely.
> > > >
> > > > On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com>
> > > wrote:
> > > >
> > > >> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <
> > > ralph.goers@dslextreme.com>
> > > >> wrote:
> > > >>
> > > >>> I am very much against the idea of a single repo. Yes, you can have
> > > >>> multiple projects in the repo but I am not sure how that can be
> > sanely
> > > >>> released. I much prefer the model that Maven has taken. They are
> now
> > > >> using
> > > >>> gitbox [1] which seems to allow GitHub to be the primary repo.
> > > >>
> > > >>
> > > >> GitHub is a distracting tangent here IMO. I really like GitHub's
> code
> > > >> commenting feature, but it should not matter if we use Apache's Git
> > > hosted
> > > >> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
> > > >> conversation... ;-)
> > > >>
> > > >> Gary
> > > >>
> > > >>
> > > >>> Every Maven plugin is individually released. Scroll down the link
> > below
> > > >> to
> > > >>> the Maven section and you can see all the plugin repos.
> > > >>>
> > > >>> The upside to this is that it are:
> > > >>>        1. It is far easier to perform releases of the individual
> > > >>> components.
> > > >>>        2. It is much easier to accept plugin contributions.
> > > >>> The downsides are:
> > > >>>        1. A page like https://maven.apache.org/plugins/ <
> > > >>> https://maven.apache.org/plugins/> is needed to keep track of the
> > > plugin
> > > >>> versions.
> > > >>>        2. It could make sense to have log4j-parent and log4j-bom
> > > >>> projects. The first to help keep the builds similar and the second
> to
> > > >> help
> > > >>> customers pick up the latest versions.
> > > >>>
> > > >>> Ralph
> > > >>>
> > > >>> [1] https://gitbox.apache.org/repos/asf <
> https://gitbox.apache.org/
> > > >>> repos/asf>
> > > >>>
> > > >>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <garydgregory@gmail.com
> >
> > > >>> wrote:
> > > >>>>
> > > >>>> Hi All,
> > > >>>>
> > > >>>> I am writing this to in an effort to understand how we can manage
> > and
> > > >>> grow
> > > >>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense
> but
> > > >> more
> > > >>>> in the maturing sense.
> > > >>>>
> > > >>>> I am prompted to write this by Ralph's comment that log4j-core and
> > the
> > > >>> main
> > > >>>> repo is too big and that releases take too long make, partially
> > > >> prompted
> > > >>> by
> > > >>>> my addition of a new module called log4j-jdbc-dbcp2 which
> currently
> > > >>>> contains one small class with a dependency on Apache Commons DBCP
> 2.
> > > >>>>
> > > >>>> I would like to express my gratitude at Ralph's efforts to
> > revitalized
> > > >>>> Log4j and performing most release management duties. Thank you
> > Ralph!
> > > >>>>
> > > >>>> I can see two main orthogonal issues:
> > > >>>> - The size of the git repo.
> > > >>>> - The size of the log4j-core module.
> > > >>>>
> > > >>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved
> > to
> > > >>>> another 'plugin' repo.
> > > >>>>
> > > >>>> I really dislike this idea:
> > > >>>> - The plugin repo has never been released. Not that one releases a
> > > repo
> > > >>> but
> > > >>>> you get the point.
> > > >>>> - How do you keep things in sync between repos and code when we
> have
> > > no
> > > >>>> official 'core' SPI.
> > > >>>>
> > > >>>> For my money, we should keep _everything_ in one repo. Good enough
> > for
> > > >>>> Google, so good enough for me. What we release out of that repo
> is a
> > > >>>> different story and what I would like to discuss next.
> > > >>>>
> > > >>>> This is not the same as Maven plugins IMO but the case could be
> made
> > > >> for
> > > >>> it
> > > >>>> I suppose where a lot/most plugins live in their own repos. It is
> > not
> > > >> the
> > > >>>> same as with Maven IMO because our plugins rely on log4j-core and
> > it's
> > > >>>> guts, for which we only make loose compatibility guarantees -- as
> > > >> opposed
> > > >>>> to log4j-core where we are strict(er). Maven OTOH, has a API for
> > > plugin
> > > >>>> auteurs.
> > > >>>>
> > > >>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core
> and
> > we
> > > >>> have
> > > >>>> no 'official' core SPI, so splitting it off into a separate repo
> > would
> > > >>>> greatly increase the risk of it falling out of sync. It is just so
> > > much
> > > >>>> more easier to maintain when it is all in one repo.
> > > >>>>
> > > >>>> My proposal is to:
> > > >>>>
> > > >>>> - Put everything back into one repo (Chainsaw too?)
> > > >>>> - Define a core SPI for plugin writers where we make some
> statement
> > > >> about
> > > >>>> BC, more than the casual 'we try not break stuff.'
> > > >>>> - Defining what Log4j project 'components' we have and release
> based
> > > on
> > > >>>> that. For example, today, all of the main Log4j repo is one
> > component
> > > >>> with
> > > >>>> many modules. Chainsaw would be another component of the Log4j
> > > project.
> > > >>>> Maybe we need to redefine components: API, File, JDBC and so on. A
> > > >>>> component can have one of more module
> > > >>>> - Got for there.
> > > >>>>
> > > >>>> Thoughts? Help me flush this out?
> > > >>>>
> > > >>>> Thank you for reading!
> > > >>>> Gary
> > > >>>
> > > >>>
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Matt Sicker <bo...@gmail.com>
> > >
> > >
> > >
> >
>
>
>
> --
> Matt Sicker <bo...@gmail.com>
>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 22, 2018 at 2:58 PM, Matt Sicker <bo...@gmail.com> wrote:

> Using Maven profiles could help, though when tagging a release, if we
> didn't release every module within, then it'd get out of sync anyways. Then
> it would still require building every single component in each release,
> even when said components haven't changed at all.
>

That reminds me that just like when I cast a VOTE I try to say what I did
and how. I would be nice if we had a 'build log' the RM would save some
place when putting up an RC.

Gary


>
> On 22 January 2018 at 15:50, Gary Gregory <ga...@gmail.com> wrote:
>
> > Coming back around to the idea of One Repo to Rule Them All*TM* ... what
> if
> > we used Maven profiles to separate out the "main" build from "other"
> > builds?
> >
> > All that is needed to move modules from one build to another is just to
> > edit the profile!
> >
> > Gary
> >
> > On Mon, Jan 22, 2018 at 12:48 PM, Ralph Goers <
> ralph.goers@dslextreme.com>
> > wrote:
> >
> > > I don’t see why this work would require 3.0 as we aren’t planning on
> > > breaking any contracts to do this.
> > >
> > > As I’ve said, I am not tied to each plugin having its own repo. I am OK
> > > with these options; a) each plugin has its own repo and site and is
> > > released independently, b) we use the plugins repo and move all the
> more
> > > lightly used components there. It would have its own site, or c) we
> group
> > > plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
> > > similar) with each having its own site.
> > >
> > > Obviously I do not consider continuing to add new plugins to the main
> > > build as one of the options.
> > >
> > > Ralph
> > >
> > > > On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:
> > > >
> > > > This whole conversation just reminds me of all the times in the past
> I
> > > > suggested full modularization of the core API/SPI. I think such an
> > effort
> > > > would make sense in a 3.0 release.
> > > >
> > > > In 2.x, perhaps what we can do is define a stable plugin API based on
> > > what
> > > > we already have. Since we're still supporting Java 7, I'd suggest we
> > make
> > > > abstract classes be the stable part of the API since we don't have
> > > default
> > > > methods on interfaces here.
> > > >
> > > > And yes, the GitBox integration thing sounds neat, particularly for
> > code
> > > > review, though that's a separate topic entirely.
> > > >
> > > > On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com>
> > > wrote:
> > > >
> > > >> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <
> > > ralph.goers@dslextreme.com>
> > > >> wrote:
> > > >>
> > > >>> I am very much against the idea of a single repo. Yes, you can have
> > > >>> multiple projects in the repo but I am not sure how that can be
> > sanely
> > > >>> released. I much prefer the model that Maven has taken. They are
> now
> > > >> using
> > > >>> gitbox [1] which seems to allow GitHub to be the primary repo.
> > > >>
> > > >>
> > > >> GitHub is a distracting tangent here IMO. I really like GitHub's
> code
> > > >> commenting feature, but it should not matter if we use Apache's Git
> > > hosted
> > > >> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
> > > >> conversation... ;-)
> > > >>
> > > >> Gary
> > > >>
> > > >>
> > > >>> Every Maven plugin is individually released. Scroll down the link
> > below
> > > >> to
> > > >>> the Maven section and you can see all the plugin repos.
> > > >>>
> > > >>> The upside to this is that it are:
> > > >>>        1. It is far easier to perform releases of the individual
> > > >>> components.
> > > >>>        2. It is much easier to accept plugin contributions.
> > > >>> The downsides are:
> > > >>>        1. A page like https://maven.apache.org/plugins/ <
> > > >>> https://maven.apache.org/plugins/> is needed to keep track of the
> > > plugin
> > > >>> versions.
> > > >>>        2. It could make sense to have log4j-parent and log4j-bom
> > > >>> projects. The first to help keep the builds similar and the second
> to
> > > >> help
> > > >>> customers pick up the latest versions.
> > > >>>
> > > >>> Ralph
> > > >>>
> > > >>> [1] https://gitbox.apache.org/repos/asf <
> https://gitbox.apache.org/
> > > >>> repos/asf>
> > > >>>
> > > >>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <garydgregory@gmail.com
> >
> > > >>> wrote:
> > > >>>>
> > > >>>> Hi All,
> > > >>>>
> > > >>>> I am writing this to in an effort to understand how we can manage
> > and
> > > >>> grow
> > > >>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense
> but
> > > >> more
> > > >>>> in the maturing sense.
> > > >>>>
> > > >>>> I am prompted to write this by Ralph's comment that log4j-core and
> > the
> > > >>> main
> > > >>>> repo is too big and that releases take too long make, partially
> > > >> prompted
> > > >>> by
> > > >>>> my addition of a new module called log4j-jdbc-dbcp2 which
> currently
> > > >>>> contains one small class with a dependency on Apache Commons DBCP
> 2.
> > > >>>>
> > > >>>> I would like to express my gratitude at Ralph's efforts to
> > revitalized
> > > >>>> Log4j and performing most release management duties. Thank you
> > Ralph!
> > > >>>>
> > > >>>> I can see two main orthogonal issues:
> > > >>>> - The size of the git repo.
> > > >>>> - The size of the log4j-core module.
> > > >>>>
> > > >>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved
> > to
> > > >>>> another 'plugin' repo.
> > > >>>>
> > > >>>> I really dislike this idea:
> > > >>>> - The plugin repo has never been released. Not that one releases a
> > > repo
> > > >>> but
> > > >>>> you get the point.
> > > >>>> - How do you keep things in sync between repos and code when we
> have
> > > no
> > > >>>> official 'core' SPI.
> > > >>>>
> > > >>>> For my money, we should keep _everything_ in one repo. Good enough
> > for
> > > >>>> Google, so good enough for me. What we release out of that repo
> is a
> > > >>>> different story and what I would like to discuss next.
> > > >>>>
> > > >>>> This is not the same as Maven plugins IMO but the case could be
> made
> > > >> for
> > > >>> it
> > > >>>> I suppose where a lot/most plugins live in their own repos. It is
> > not
> > > >> the
> > > >>>> same as with Maven IMO because our plugins rely on log4j-core and
> > it's
> > > >>>> guts, for which we only make loose compatibility guarantees -- as
> > > >> opposed
> > > >>>> to log4j-core where we are strict(er). Maven OTOH, has a API for
> > > plugin
> > > >>>> auteurs.
> > > >>>>
> > > >>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core
> and
> > we
> > > >>> have
> > > >>>> no 'official' core SPI, so splitting it off into a separate repo
> > would
> > > >>>> greatly increase the risk of it falling out of sync. It is just so
> > > much
> > > >>>> more easier to maintain when it is all in one repo.
> > > >>>>
> > > >>>> My proposal is to:
> > > >>>>
> > > >>>> - Put everything back into one repo (Chainsaw too?)
> > > >>>> - Define a core SPI for plugin writers where we make some
> statement
> > > >> about
> > > >>>> BC, more than the casual 'we try not break stuff.'
> > > >>>> - Defining what Log4j project 'components' we have and release
> based
> > > on
> > > >>>> that. For example, today, all of the main Log4j repo is one
> > component
> > > >>> with
> > > >>>> many modules. Chainsaw would be another component of the Log4j
> > > project.
> > > >>>> Maybe we need to redefine components: API, File, JDBC and so on. A
> > > >>>> component can have one of more module
> > > >>>> - Got for there.
> > > >>>>
> > > >>>> Thoughts? Help me flush this out?
> > > >>>>
> > > >>>> Thank you for reading!
> > > >>>> Gary
> > > >>>
> > > >>>
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Matt Sicker <bo...@gmail.com>
> > >
> > >
> > >
> >
>
>
>
> --
> Matt Sicker <bo...@gmail.com>
>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 22, 2018 at 3:03 PM, Scott Deboy <sc...@gmail.com> wrote:

> Correcting my prior comment: the nightmare I was referring to was the
> ugliness we did and un-did related to the never-released receivers and
> 'component' companions.  Definitely feels like deja vu.
>
> https://github.com/apache/log4j-component/blob/trunk/pom.xml
>
> apache-log4j-companions-parent etc etc
>

Hi Scott,

I'm not sure I recall or was around that episode. Would you please mind
telling us what the parallel is here and what you see as the lesson for our
current situation?

Thank you!
Gary

>
>
> On 1/22/18, Matt Sicker <bo...@gmail.com> wrote:
> > Using Maven profiles could help, though when tagging a release, if we
> > didn't release every module within, then it'd get out of sync anyways.
> Then
> > it would still require building every single component in each release,
> > even when said components haven't changed at all.
> >
> > On 22 January 2018 at 15:50, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> >> Coming back around to the idea of One Repo to Rule Them All*TM* ... what
> >> if
> >> we used Maven profiles to separate out the "main" build from "other"
> >> builds?
> >>
> >> All that is needed to move modules from one build to another is just to
> >> edit the profile!
> >>
> >> Gary
> >>
> >> On Mon, Jan 22, 2018 at 12:48 PM, Ralph Goers
> >> <ra...@dslextreme.com>
> >> wrote:
> >>
> >> > I don’t see why this work would require 3.0 as we aren’t planning on
> >> > breaking any contracts to do this.
> >> >
> >> > As I’ve said, I am not tied to each plugin having its own repo. I am
> OK
> >> > with these options; a) each plugin has its own repo and site and is
> >> > released independently, b) we use the plugins repo and move all the
> >> > more
> >> > lightly used components there. It would have its own site, or c) we
> >> > group
> >> > plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
> >> > similar) with each having its own site.
> >> >
> >> > Obviously I do not consider continuing to add new plugins to the main
> >> > build as one of the options.
> >> >
> >> > Ralph
> >> >
> >> > > On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:
> >> > >
> >> > > This whole conversation just reminds me of all the times in the past
> >> > > I
> >> > > suggested full modularization of the core API/SPI. I think such an
> >> effort
> >> > > would make sense in a 3.0 release.
> >> > >
> >> > > In 2.x, perhaps what we can do is define a stable plugin API based
> on
> >> > what
> >> > > we already have. Since we're still supporting Java 7, I'd suggest we
> >> make
> >> > > abstract classes be the stable part of the API since we don't have
> >> > default
> >> > > methods on interfaces here.
> >> > >
> >> > > And yes, the GitBox integration thing sounds neat, particularly for
> >> code
> >> > > review, though that's a separate topic entirely.
> >> > >
> >> > > On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com>
> >> > wrote:
> >> > >
> >> > >> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <
> >> > ralph.goers@dslextreme.com>
> >> > >> wrote:
> >> > >>
> >> > >>> I am very much against the idea of a single repo. Yes, you can
> have
> >> > >>> multiple projects in the repo but I am not sure how that can be
> >> sanely
> >> > >>> released. I much prefer the model that Maven has taken. They are
> >> > >>> now
> >> > >> using
> >> > >>> gitbox [1] which seems to allow GitHub to be the primary repo.
> >> > >>
> >> > >>
> >> > >> GitHub is a distracting tangent here IMO. I really like GitHub's
> >> > >> code
> >> > >> commenting feature, but it should not matter if we use Apache's Git
> >> > hosted
> >> > >> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
> >> > >> conversation... ;-)
> >> > >>
> >> > >> Gary
> >> > >>
> >> > >>
> >> > >>> Every Maven plugin is individually released. Scroll down the link
> >> below
> >> > >> to
> >> > >>> the Maven section and you can see all the plugin repos.
> >> > >>>
> >> > >>> The upside to this is that it are:
> >> > >>>        1. It is far easier to perform releases of the individual
> >> > >>> components.
> >> > >>>        2. It is much easier to accept plugin contributions.
> >> > >>> The downsides are:
> >> > >>>        1. A page like https://maven.apache.org/plugins/ <
> >> > >>> https://maven.apache.org/plugins/> is needed to keep track of the
> >> > plugin
> >> > >>> versions.
> >> > >>>        2. It could make sense to have log4j-parent and log4j-bom
> >> > >>> projects. The first to help keep the builds similar and the second
> >> > >>> to
> >> > >> help
> >> > >>> customers pick up the latest versions.
> >> > >>>
> >> > >>> Ralph
> >> > >>>
> >> > >>> [1] https://gitbox.apache.org/repos/asf <
> https://gitbox.apache.org/
> >> > >>> repos/asf>
> >> > >>>
> >> > >>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <
> garydgregory@gmail.com>
> >> > >>> wrote:
> >> > >>>>
> >> > >>>> Hi All,
> >> > >>>>
> >> > >>>> I am writing this to in an effort to understand how we can manage
> >> and
> >> > >>> grow
> >> > >>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense
> >> > >>>> but
> >> > >> more
> >> > >>>> in the maturing sense.
> >> > >>>>
> >> > >>>> I am prompted to write this by Ralph's comment that log4j-core
> and
> >> the
> >> > >>> main
> >> > >>>> repo is too big and that releases take too long make, partially
> >> > >> prompted
> >> > >>> by
> >> > >>>> my addition of a new module called log4j-jdbc-dbcp2 which
> >> > >>>> currently
> >> > >>>> contains one small class with a dependency on Apache Commons DBCP
> >> > >>>> 2.
> >> > >>>>
> >> > >>>> I would like to express my gratitude at Ralph's efforts to
> >> revitalized
> >> > >>>> Log4j and performing most release management duties. Thank you
> >> Ralph!
> >> > >>>>
> >> > >>>> I can see two main orthogonal issues:
> >> > >>>> - The size of the git repo.
> >> > >>>> - The size of the log4j-core module.
> >> > >>>>
> >> > >>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be
> moved
> >> to
> >> > >>>> another 'plugin' repo.
> >> > >>>>
> >> > >>>> I really dislike this idea:
> >> > >>>> - The plugin repo has never been released. Not that one releases
> a
> >> > repo
> >> > >>> but
> >> > >>>> you get the point.
> >> > >>>> - How do you keep things in sync between repos and code when we
> >> > >>>> have
> >> > no
> >> > >>>> official 'core' SPI.
> >> > >>>>
> >> > >>>> For my money, we should keep _everything_ in one repo. Good
> enough
> >> for
> >> > >>>> Google, so good enough for me. What we release out of that repo
> is
> >> > >>>> a
> >> > >>>> different story and what I would like to discuss next.
> >> > >>>>
> >> > >>>> This is not the same as Maven plugins IMO but the case could be
> >> > >>>> made
> >> > >> for
> >> > >>> it
> >> > >>>> I suppose where a lot/most plugins live in their own repos. It is
> >> not
> >> > >> the
> >> > >>>> same as with Maven IMO because our plugins rely on log4j-core and
> >> it's
> >> > >>>> guts, for which we only make loose compatibility guarantees -- as
> >> > >> opposed
> >> > >>>> to log4j-core where we are strict(er). Maven OTOH, has a API for
> >> > plugin
> >> > >>>> auteurs.
> >> > >>>>
> >> > >>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core
> >> > >>>> and
> >> we
> >> > >>> have
> >> > >>>> no 'official' core SPI, so splitting it off into a separate repo
> >> would
> >> > >>>> greatly increase the risk of it falling out of sync. It is just
> so
> >> > much
> >> > >>>> more easier to maintain when it is all in one repo.
> >> > >>>>
> >> > >>>> My proposal is to:
> >> > >>>>
> >> > >>>> - Put everything back into one repo (Chainsaw too?)
> >> > >>>> - Define a core SPI for plugin writers where we make some
> >> > >>>> statement
> >> > >> about
> >> > >>>> BC, more than the casual 'we try not break stuff.'
> >> > >>>> - Defining what Log4j project 'components' we have and release
> >> > >>>> based
> >> > on
> >> > >>>> that. For example, today, all of the main Log4j repo is one
> >> component
> >> > >>> with
> >> > >>>> many modules. Chainsaw would be another component of the Log4j
> >> > project.
> >> > >>>> Maybe we need to redefine components: API, File, JDBC and so on.
> A
> >> > >>>> component can have one of more module
> >> > >>>> - Got for there.
> >> > >>>>
> >> > >>>> Thoughts? Help me flush this out?
> >> > >>>>
> >> > >>>> Thank you for reading!
> >> > >>>> Gary
> >> > >>>
> >> > >>>
> >> > >>
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Matt Sicker <bo...@gmail.com>
> >> >
> >> >
> >> >
> >>
> >
> >
> >
> > --
> > Matt Sicker <bo...@gmail.com>
> >
>

Re: [log4j] The shape of Log4j

Posted by Scott Deboy <sc...@gmail.com>.
Correcting my prior comment: the nightmare I was referring to was the
ugliness we did and un-did related to the never-released receivers and
'component' companions.  Definitely feels like deja vu.

https://github.com/apache/log4j-component/blob/trunk/pom.xml

apache-log4j-companions-parent etc etc


On 1/22/18, Matt Sicker <bo...@gmail.com> wrote:
> Using Maven profiles could help, though when tagging a release, if we
> didn't release every module within, then it'd get out of sync anyways. Then
> it would still require building every single component in each release,
> even when said components haven't changed at all.
>
> On 22 January 2018 at 15:50, Gary Gregory <ga...@gmail.com> wrote:
>
>> Coming back around to the idea of One Repo to Rule Them All*TM* ... what
>> if
>> we used Maven profiles to separate out the "main" build from "other"
>> builds?
>>
>> All that is needed to move modules from one build to another is just to
>> edit the profile!
>>
>> Gary
>>
>> On Mon, Jan 22, 2018 at 12:48 PM, Ralph Goers
>> <ra...@dslextreme.com>
>> wrote:
>>
>> > I don’t see why this work would require 3.0 as we aren’t planning on
>> > breaking any contracts to do this.
>> >
>> > As I’ve said, I am not tied to each plugin having its own repo. I am OK
>> > with these options; a) each plugin has its own repo and site and is
>> > released independently, b) we use the plugins repo and move all the
>> > more
>> > lightly used components there. It would have its own site, or c) we
>> > group
>> > plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
>> > similar) with each having its own site.
>> >
>> > Obviously I do not consider continuing to add new plugins to the main
>> > build as one of the options.
>> >
>> > Ralph
>> >
>> > > On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:
>> > >
>> > > This whole conversation just reminds me of all the times in the past
>> > > I
>> > > suggested full modularization of the core API/SPI. I think such an
>> effort
>> > > would make sense in a 3.0 release.
>> > >
>> > > In 2.x, perhaps what we can do is define a stable plugin API based on
>> > what
>> > > we already have. Since we're still supporting Java 7, I'd suggest we
>> make
>> > > abstract classes be the stable part of the API since we don't have
>> > default
>> > > methods on interfaces here.
>> > >
>> > > And yes, the GitBox integration thing sounds neat, particularly for
>> code
>> > > review, though that's a separate topic entirely.
>> > >
>> > > On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com>
>> > wrote:
>> > >
>> > >> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <
>> > ralph.goers@dslextreme.com>
>> > >> wrote:
>> > >>
>> > >>> I am very much against the idea of a single repo. Yes, you can have
>> > >>> multiple projects in the repo but I am not sure how that can be
>> sanely
>> > >>> released. I much prefer the model that Maven has taken. They are
>> > >>> now
>> > >> using
>> > >>> gitbox [1] which seems to allow GitHub to be the primary repo.
>> > >>
>> > >>
>> > >> GitHub is a distracting tangent here IMO. I really like GitHub's
>> > >> code
>> > >> commenting feature, but it should not matter if we use Apache's Git
>> > hosted
>> > >> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
>> > >> conversation... ;-)
>> > >>
>> > >> Gary
>> > >>
>> > >>
>> > >>> Every Maven plugin is individually released. Scroll down the link
>> below
>> > >> to
>> > >>> the Maven section and you can see all the plugin repos.
>> > >>>
>> > >>> The upside to this is that it are:
>> > >>>        1. It is far easier to perform releases of the individual
>> > >>> components.
>> > >>>        2. It is much easier to accept plugin contributions.
>> > >>> The downsides are:
>> > >>>        1. A page like https://maven.apache.org/plugins/ <
>> > >>> https://maven.apache.org/plugins/> is needed to keep track of the
>> > plugin
>> > >>> versions.
>> > >>>        2. It could make sense to have log4j-parent and log4j-bom
>> > >>> projects. The first to help keep the builds similar and the second
>> > >>> to
>> > >> help
>> > >>> customers pick up the latest versions.
>> > >>>
>> > >>> Ralph
>> > >>>
>> > >>> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
>> > >>> repos/asf>
>> > >>>
>> > >>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
>> > >>> wrote:
>> > >>>>
>> > >>>> Hi All,
>> > >>>>
>> > >>>> I am writing this to in an effort to understand how we can manage
>> and
>> > >>> grow
>> > >>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense
>> > >>>> but
>> > >> more
>> > >>>> in the maturing sense.
>> > >>>>
>> > >>>> I am prompted to write this by Ralph's comment that log4j-core and
>> the
>> > >>> main
>> > >>>> repo is too big and that releases take too long make, partially
>> > >> prompted
>> > >>> by
>> > >>>> my addition of a new module called log4j-jdbc-dbcp2 which
>> > >>>> currently
>> > >>>> contains one small class with a dependency on Apache Commons DBCP
>> > >>>> 2.
>> > >>>>
>> > >>>> I would like to express my gratitude at Ralph's efforts to
>> revitalized
>> > >>>> Log4j and performing most release management duties. Thank you
>> Ralph!
>> > >>>>
>> > >>>> I can see two main orthogonal issues:
>> > >>>> - The size of the git repo.
>> > >>>> - The size of the log4j-core module.
>> > >>>>
>> > >>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved
>> to
>> > >>>> another 'plugin' repo.
>> > >>>>
>> > >>>> I really dislike this idea:
>> > >>>> - The plugin repo has never been released. Not that one releases a
>> > repo
>> > >>> but
>> > >>>> you get the point.
>> > >>>> - How do you keep things in sync between repos and code when we
>> > >>>> have
>> > no
>> > >>>> official 'core' SPI.
>> > >>>>
>> > >>>> For my money, we should keep _everything_ in one repo. Good enough
>> for
>> > >>>> Google, so good enough for me. What we release out of that repo is
>> > >>>> a
>> > >>>> different story and what I would like to discuss next.
>> > >>>>
>> > >>>> This is not the same as Maven plugins IMO but the case could be
>> > >>>> made
>> > >> for
>> > >>> it
>> > >>>> I suppose where a lot/most plugins live in their own repos. It is
>> not
>> > >> the
>> > >>>> same as with Maven IMO because our plugins rely on log4j-core and
>> it's
>> > >>>> guts, for which we only make loose compatibility guarantees -- as
>> > >> opposed
>> > >>>> to log4j-core where we are strict(er). Maven OTOH, has a API for
>> > plugin
>> > >>>> auteurs.
>> > >>>>
>> > >>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core
>> > >>>> and
>> we
>> > >>> have
>> > >>>> no 'official' core SPI, so splitting it off into a separate repo
>> would
>> > >>>> greatly increase the risk of it falling out of sync. It is just so
>> > much
>> > >>>> more easier to maintain when it is all in one repo.
>> > >>>>
>> > >>>> My proposal is to:
>> > >>>>
>> > >>>> - Put everything back into one repo (Chainsaw too?)
>> > >>>> - Define a core SPI for plugin writers where we make some
>> > >>>> statement
>> > >> about
>> > >>>> BC, more than the casual 'we try not break stuff.'
>> > >>>> - Defining what Log4j project 'components' we have and release
>> > >>>> based
>> > on
>> > >>>> that. For example, today, all of the main Log4j repo is one
>> component
>> > >>> with
>> > >>>> many modules. Chainsaw would be another component of the Log4j
>> > project.
>> > >>>> Maybe we need to redefine components: API, File, JDBC and so on. A
>> > >>>> component can have one of more module
>> > >>>> - Got for there.
>> > >>>>
>> > >>>> Thoughts? Help me flush this out?
>> > >>>>
>> > >>>> Thank you for reading!
>> > >>>> Gary
>> > >>>
>> > >>>
>> > >>
>> > >
>> > >
>> > >
>> > > --
>> > > Matt Sicker <bo...@gmail.com>
>> >
>> >
>> >
>>
>
>
>
> --
> Matt Sicker <bo...@gmail.com>
>

Re: [log4j] The shape of Log4j

Posted by Mikael Ståldal <mi...@apache.org>.
Yes, neither Git nor Maven is optimal for mono-repos since you can only 
tag/branch the whole repo, and only have one version for the whole build.

Google have a big mono-repo, but they do not use Git, nor Maven.

I think we should release and version everything in one repo together, 
otherwise it will get very messy.


On 2018-01-22 22:58, Matt Sicker wrote:
> Using Maven profiles could help, though when tagging a release, if we
> didn't release every module within, then it'd get out of sync anyways. Then
> it would still require building every single component in each release,
> even when said components haven't changed at all.
> 
> On 22 January 2018 at 15:50, Gary Gregory <ga...@gmail.com> wrote:
> 
>> Coming back around to the idea of One Repo to Rule Them All*TM* ... what if
>> we used Maven profiles to separate out the "main" build from "other"
>> builds?
>>
>> All that is needed to move modules from one build to another is just to
>> edit the profile!

Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
Using Maven profiles could help, though when tagging a release, if we
didn't release every module within, then it'd get out of sync anyways. Then
it would still require building every single component in each release,
even when said components haven't changed at all.

On 22 January 2018 at 15:50, Gary Gregory <ga...@gmail.com> wrote:

> Coming back around to the idea of One Repo to Rule Them All*TM* ... what if
> we used Maven profiles to separate out the "main" build from "other"
> builds?
>
> All that is needed to move modules from one build to another is just to
> edit the profile!
>
> Gary
>
> On Mon, Jan 22, 2018 at 12:48 PM, Ralph Goers <ra...@dslextreme.com>
> wrote:
>
> > I don’t see why this work would require 3.0 as we aren’t planning on
> > breaking any contracts to do this.
> >
> > As I’ve said, I am not tied to each plugin having its own repo. I am OK
> > with these options; a) each plugin has its own repo and site and is
> > released independently, b) we use the plugins repo and move all the more
> > lightly used components there. It would have its own site, or c) we group
> > plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
> > similar) with each having its own site.
> >
> > Obviously I do not consider continuing to add new plugins to the main
> > build as one of the options.
> >
> > Ralph
> >
> > > On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:
> > >
> > > This whole conversation just reminds me of all the times in the past I
> > > suggested full modularization of the core API/SPI. I think such an
> effort
> > > would make sense in a 3.0 release.
> > >
> > > In 2.x, perhaps what we can do is define a stable plugin API based on
> > what
> > > we already have. Since we're still supporting Java 7, I'd suggest we
> make
> > > abstract classes be the stable part of the API since we don't have
> > default
> > > methods on interfaces here.
> > >
> > > And yes, the GitBox integration thing sounds neat, particularly for
> code
> > > review, though that's a separate topic entirely.
> > >
> > > On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com>
> > wrote:
> > >
> > >> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <
> > ralph.goers@dslextreme.com>
> > >> wrote:
> > >>
> > >>> I am very much against the idea of a single repo. Yes, you can have
> > >>> multiple projects in the repo but I am not sure how that can be
> sanely
> > >>> released. I much prefer the model that Maven has taken. They are now
> > >> using
> > >>> gitbox [1] which seems to allow GitHub to be the primary repo.
> > >>
> > >>
> > >> GitHub is a distracting tangent here IMO. I really like GitHub's code
> > >> commenting feature, but it should not matter if we use Apache's Git
> > hosted
> > >> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
> > >> conversation... ;-)
> > >>
> > >> Gary
> > >>
> > >>
> > >>> Every Maven plugin is individually released. Scroll down the link
> below
> > >> to
> > >>> the Maven section and you can see all the plugin repos.
> > >>>
> > >>> The upside to this is that it are:
> > >>>        1. It is far easier to perform releases of the individual
> > >>> components.
> > >>>        2. It is much easier to accept plugin contributions.
> > >>> The downsides are:
> > >>>        1. A page like https://maven.apache.org/plugins/ <
> > >>> https://maven.apache.org/plugins/> is needed to keep track of the
> > plugin
> > >>> versions.
> > >>>        2. It could make sense to have log4j-parent and log4j-bom
> > >>> projects. The first to help keep the builds similar and the second to
> > >> help
> > >>> customers pick up the latest versions.
> > >>>
> > >>> Ralph
> > >>>
> > >>> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
> > >>> repos/asf>
> > >>>
> > >>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
> > >>> wrote:
> > >>>>
> > >>>> Hi All,
> > >>>>
> > >>>> I am writing this to in an effort to understand how we can manage
> and
> > >>> grow
> > >>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense but
> > >> more
> > >>>> in the maturing sense.
> > >>>>
> > >>>> I am prompted to write this by Ralph's comment that log4j-core and
> the
> > >>> main
> > >>>> repo is too big and that releases take too long make, partially
> > >> prompted
> > >>> by
> > >>>> my addition of a new module called log4j-jdbc-dbcp2 which currently
> > >>>> contains one small class with a dependency on Apache Commons DBCP 2.
> > >>>>
> > >>>> I would like to express my gratitude at Ralph's efforts to
> revitalized
> > >>>> Log4j and performing most release management duties. Thank you
> Ralph!
> > >>>>
> > >>>> I can see two main orthogonal issues:
> > >>>> - The size of the git repo.
> > >>>> - The size of the log4j-core module.
> > >>>>
> > >>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved
> to
> > >>>> another 'plugin' repo.
> > >>>>
> > >>>> I really dislike this idea:
> > >>>> - The plugin repo has never been released. Not that one releases a
> > repo
> > >>> but
> > >>>> you get the point.
> > >>>> - How do you keep things in sync between repos and code when we have
> > no
> > >>>> official 'core' SPI.
> > >>>>
> > >>>> For my money, we should keep _everything_ in one repo. Good enough
> for
> > >>>> Google, so good enough for me. What we release out of that repo is a
> > >>>> different story and what I would like to discuss next.
> > >>>>
> > >>>> This is not the same as Maven plugins IMO but the case could be made
> > >> for
> > >>> it
> > >>>> I suppose where a lot/most plugins live in their own repos. It is
> not
> > >> the
> > >>>> same as with Maven IMO because our plugins rely on log4j-core and
> it's
> > >>>> guts, for which we only make loose compatibility guarantees -- as
> > >> opposed
> > >>>> to log4j-core where we are strict(er). Maven OTOH, has a API for
> > plugin
> > >>>> auteurs.
> > >>>>
> > >>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and
> we
> > >>> have
> > >>>> no 'official' core SPI, so splitting it off into a separate repo
> would
> > >>>> greatly increase the risk of it falling out of sync. It is just so
> > much
> > >>>> more easier to maintain when it is all in one repo.
> > >>>>
> > >>>> My proposal is to:
> > >>>>
> > >>>> - Put everything back into one repo (Chainsaw too?)
> > >>>> - Define a core SPI for plugin writers where we make some statement
> > >> about
> > >>>> BC, more than the casual 'we try not break stuff.'
> > >>>> - Defining what Log4j project 'components' we have and release based
> > on
> > >>>> that. For example, today, all of the main Log4j repo is one
> component
> > >>> with
> > >>>> many modules. Chainsaw would be another component of the Log4j
> > project.
> > >>>> Maybe we need to redefine components: API, File, JDBC and so on. A
> > >>>> component can have one of more module
> > >>>> - Got for there.
> > >>>>
> > >>>> Thoughts? Help me flush this out?
> > >>>>
> > >>>> Thank you for reading!
> > >>>> Gary
> > >>>
> > >>>
> > >>
> > >
> > >
> > >
> > > --
> > > Matt Sicker <bo...@gmail.com>
> >
> >
> >
>



-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
Coming back around to the idea of One Repo to Rule Them All*TM* ... what if
we used Maven profiles to separate out the "main" build from "other" builds?

All that is needed to move modules from one build to another is just to
edit the profile!

Gary

On Mon, Jan 22, 2018 at 12:48 PM, Ralph Goers <ra...@dslextreme.com>
wrote:

> I don’t see why this work would require 3.0 as we aren’t planning on
> breaking any contracts to do this.
>
> As I’ve said, I am not tied to each plugin having its own repo. I am OK
> with these options; a) each plugin has its own repo and site and is
> released independently, b) we use the plugins repo and move all the more
> lightly used components there. It would have its own site, or c) we group
> plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
> similar) with each having its own site.
>
> Obviously I do not consider continuing to add new plugins to the main
> build as one of the options.
>
> Ralph
>
> > On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:
> >
> > This whole conversation just reminds me of all the times in the past I
> > suggested full modularization of the core API/SPI. I think such an effort
> > would make sense in a 3.0 release.
> >
> > In 2.x, perhaps what we can do is define a stable plugin API based on
> what
> > we already have. Since we're still supporting Java 7, I'd suggest we make
> > abstract classes be the stable part of the API since we don't have
> default
> > methods on interfaces here.
> >
> > And yes, the GitBox integration thing sounds neat, particularly for code
> > review, though that's a separate topic entirely.
> >
> > On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> >> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <
> ralph.goers@dslextreme.com>
> >> wrote:
> >>
> >>> I am very much against the idea of a single repo. Yes, you can have
> >>> multiple projects in the repo but I am not sure how that can be sanely
> >>> released. I much prefer the model that Maven has taken. They are now
> >> using
> >>> gitbox [1] which seems to allow GitHub to be the primary repo.
> >>
> >>
> >> GitHub is a distracting tangent here IMO. I really like GitHub's code
> >> commenting feature, but it should not matter if we use Apache's Git
> hosted
> >> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
> >> conversation... ;-)
> >>
> >> Gary
> >>
> >>
> >>> Every Maven plugin is individually released. Scroll down the link below
> >> to
> >>> the Maven section and you can see all the plugin repos.
> >>>
> >>> The upside to this is that it are:
> >>>        1. It is far easier to perform releases of the individual
> >>> components.
> >>>        2. It is much easier to accept plugin contributions.
> >>> The downsides are:
> >>>        1. A page like https://maven.apache.org/plugins/ <
> >>> https://maven.apache.org/plugins/> is needed to keep track of the
> plugin
> >>> versions.
> >>>        2. It could make sense to have log4j-parent and log4j-bom
> >>> projects. The first to help keep the builds similar and the second to
> >> help
> >>> customers pick up the latest versions.
> >>>
> >>> Ralph
> >>>
> >>> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
> >>> repos/asf>
> >>>
> >>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
> >>> wrote:
> >>>>
> >>>> Hi All,
> >>>>
> >>>> I am writing this to in an effort to understand how we can manage and
> >>> grow
> >>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense but
> >> more
> >>>> in the maturing sense.
> >>>>
> >>>> I am prompted to write this by Ralph's comment that log4j-core and the
> >>> main
> >>>> repo is too big and that releases take too long make, partially
> >> prompted
> >>> by
> >>>> my addition of a new module called log4j-jdbc-dbcp2 which currently
> >>>> contains one small class with a dependency on Apache Commons DBCP 2.
> >>>>
> >>>> I would like to express my gratitude at Ralph's efforts to revitalized
> >>>> Log4j and performing most release management duties. Thank you Ralph!
> >>>>
> >>>> I can see two main orthogonal issues:
> >>>> - The size of the git repo.
> >>>> - The size of the log4j-core module.
> >>>>
> >>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
> >>>> another 'plugin' repo.
> >>>>
> >>>> I really dislike this idea:
> >>>> - The plugin repo has never been released. Not that one releases a
> repo
> >>> but
> >>>> you get the point.
> >>>> - How do you keep things in sync between repos and code when we have
> no
> >>>> official 'core' SPI.
> >>>>
> >>>> For my money, we should keep _everything_ in one repo. Good enough for
> >>>> Google, so good enough for me. What we release out of that repo is a
> >>>> different story and what I would like to discuss next.
> >>>>
> >>>> This is not the same as Maven plugins IMO but the case could be made
> >> for
> >>> it
> >>>> I suppose where a lot/most plugins live in their own repos. It is not
> >> the
> >>>> same as with Maven IMO because our plugins rely on log4j-core and it's
> >>>> guts, for which we only make loose compatibility guarantees -- as
> >> opposed
> >>>> to log4j-core where we are strict(er). Maven OTOH, has a API for
> plugin
> >>>> auteurs.
> >>>>
> >>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we
> >>> have
> >>>> no 'official' core SPI, so splitting it off into a separate repo would
> >>>> greatly increase the risk of it falling out of sync. It is just so
> much
> >>>> more easier to maintain when it is all in one repo.
> >>>>
> >>>> My proposal is to:
> >>>>
> >>>> - Put everything back into one repo (Chainsaw too?)
> >>>> - Define a core SPI for plugin writers where we make some statement
> >> about
> >>>> BC, more than the casual 'we try not break stuff.'
> >>>> - Defining what Log4j project 'components' we have and release based
> on
> >>>> that. For example, today, all of the main Log4j repo is one component
> >>> with
> >>>> many modules. Chainsaw would be another component of the Log4j
> project.
> >>>> Maybe we need to redefine components: API, File, JDBC and so on. A
> >>>> component can have one of more module
> >>>> - Got for there.
> >>>>
> >>>> Thoughts? Help me flush this out?
> >>>>
> >>>> Thank you for reading!
> >>>> Gary
> >>>
> >>>
> >>
> >
> >
> >
> > --
> > Matt Sicker <bo...@gmail.com>
>
>
>

Re: [log4j] The shape of Log4j

Posted by Robert Middleton <os...@gmail.com>.
If you're interested in reading about how Google uses their monorepo,
they have a paper you can read through:
https://research.google.com/pubs/pub45424.html

It's not clear to me reading the paper if they do this for all of
their projects(including public repos on github) or just in-house
code; it sounds like the latter. It's not something that would be
practical for most projects.

-Robert Middleton

On Wed, Jan 24, 2018 at 7:45 PM, Matt Sicker <bo...@gmail.com> wrote:
> I'm going to sum up my view here: if we split up repos to include plugins
> that depend on log4j-core, then we first need to define and release a
> stable plugin API. If we can't guarantee BC for plugins, then there's no
> realistic way to both split out additional plugins and have them be
> officially endorsed plugins. If said plugins were completely outside Apache
> (e.g., personal GitHub pages), then it wouldn't be as strict.
>
> Also, I find it hard to believe that Google really uses a monorepo anymore.
> Such an idea wouldn't work too well with software that isn't continuously
> released.
>
> I suppose that's an alternative: I'd be open to splitting up repos however
> desired if releases were continually published to avoid version number
> synchronisation efforts, but I'm not so sure how well that meshes with the
> Apache Way for releasing software.

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Wed, Jan 24, 2018 at 10:05 PM, Ralph Goers <ra...@dslextreme.com>
wrote:

> Thanks for that Remko.
>
> We are about due for another Log4j 2 release. I won’t be able to get a
> release of log4j-server and then move the plugins to the plugins project in
> a short enough time to warrant holding off a Log4j 2 release. So I am going
> to say Gary should go ahead and add the mongo3 module for now. I will
> imagine it will take a couple of months before I can do the work but I will
> let everyone know when I start to work on that. I wouldn’t move anything
> out of the main build until the plugins repo is ready for a release.
>
> FWIW, since it takes me at least 4 hours to perform the release I can’t
> guarantee when I will get to it, but I would hope it would be in the next
> couple of weeks.
>
> Does that sound reasonable?
>


Hello,

I'll commit log4j-mongodb3 today. Let's see how the rest plays out...

Gary


>
> Ralph
>
>
>
>
> > On Jan 24, 2018, at 7:20 PM, Remko Popma <re...@gmail.com> wrote:
> >
> > I’m still not convinced that moving the items that Ralph enumerated out
> of core will make a significant difference in the time it takes to do a
> release (I believe other measures will be more effective), but fine.
> >
> > If Ralph wants this as a precondition to continue to perform the Log4j2
> releases then I can accept that. I’m not planning to start doing the Log4j2
> releases and I’m grateful that others are willing to do so.
> >
> > However, an important question that needs to be answered before we do
> this is: how do we detect when changes in the main Log4j2 project broke one
> of the plugins when they live in another repo?
> >
> > Creating a plugin API is one option but this will take time to design,
> develop and test.
> >
> > One possible alternative: Can we set up a continuous integration service
> that builds the main project together with the plugins subproject? (We
> probably need this regardless of whether there is a plugin API or not.) I’m
> open to other solutions but we do need some process in place to detect
> breaking changes.
> >
> > FYI I strongly feel the perf module should stay in the main project/repo
> or it won’t be useable.
> >
> > Finally, I think breaking up log4j-plugins into many separate repos is
> not a good idea. It may sound nice on the surface but the overhead is
> large. Are the people pushing for the repo-per-module idea willing to do
> the release work for these plugins? And is everyone else willing to review
> and vote for 10 plugin releases instead of just one? And integrate 10
> websites instead of one?
> >
> > (Shameless plug) Every java main() method deserves http://picocli.info
> >
> >> On Jan 25, 2018, at 9:45, Matt Sicker <bo...@gmail.com> wrote:
> >>
> >> I'm going to sum up my view here: if we split up repos to include
> plugins
> >> that depend on log4j-core, then we first need to define and release a
> >> stable plugin API. If we can't guarantee BC for plugins, then there's no
> >> realistic way to both split out additional plugins and have them be
> >> officially endorsed plugins. If said plugins were completely outside
> Apache
> >> (e.g., personal GitHub pages), then it wouldn't be as strict.
> >>
> >> Also, I find it hard to believe that Google really uses a monorepo
> anymore.
> >> Such an idea wouldn't work too well with software that isn't
> continuously
> >> released.
> >>
> >> I suppose that's an alternative: I'd be open to splitting up repos
> however
> >> desired if releases were continually published to avoid version number
> >> synchronisation efforts, but I'm not so sure how well that meshes with
> the
> >> Apache Way for releasing software.
> >
>
>
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
Thanks for that Remko.

We are about due for another Log4j 2 release. I won’t be able to get a release of log4j-server and then move the plugins to the plugins project in a short enough time to warrant holding off a Log4j 2 release. So I am going to say Gary should go ahead and add the mongo3 module for now. I will imagine it will take a couple of months before I can do the work but I will let everyone know when I start to work on that. I wouldn’t move anything out of the main build until the plugins repo is ready for a release.

FWIW, since it takes me at least 4 hours to perform the release I can’t guarantee when I will get to it, but I would hope it would be in the next couple of weeks.

Does that sound reasonable?

Ralph




> On Jan 24, 2018, at 7:20 PM, Remko Popma <re...@gmail.com> wrote:
> 
> I’m still not convinced that moving the items that Ralph enumerated out of core will make a significant difference in the time it takes to do a release (I believe other measures will be more effective), but fine. 
> 
> If Ralph wants this as a precondition to continue to perform the Log4j2 releases then I can accept that. I’m not planning to start doing the Log4j2 releases and I’m grateful that others are willing to do so. 
> 
> However, an important question that needs to be answered before we do this is: how do we detect when changes in the main Log4j2 project broke one of the plugins when they live in another repo?
> 
> Creating a plugin API is one option but this will take time to design, develop and test. 
> 
> One possible alternative: Can we set up a continuous integration service that builds the main project together with the plugins subproject? (We probably need this regardless of whether there is a plugin API or not.) I’m open to other solutions but we do need some process in place to detect breaking changes. 
> 
> FYI I strongly feel the perf module should stay in the main project/repo or it won’t be useable. 
> 
> Finally, I think breaking up log4j-plugins into many separate repos is not a good idea. It may sound nice on the surface but the overhead is large. Are the people pushing for the repo-per-module idea willing to do the release work for these plugins? And is everyone else willing to review and vote for 10 plugin releases instead of just one? And integrate 10 websites instead of one?
> 
> (Shameless plug) Every java main() method deserves http://picocli.info
> 
>> On Jan 25, 2018, at 9:45, Matt Sicker <bo...@gmail.com> wrote:
>> 
>> I'm going to sum up my view here: if we split up repos to include plugins
>> that depend on log4j-core, then we first need to define and release a
>> stable plugin API. If we can't guarantee BC for plugins, then there's no
>> realistic way to both split out additional plugins and have them be
>> officially endorsed plugins. If said plugins were completely outside Apache
>> (e.g., personal GitHub pages), then it wouldn't be as strict.
>> 
>> Also, I find it hard to believe that Google really uses a monorepo anymore.
>> Such an idea wouldn't work too well with software that isn't continuously
>> released.
>> 
>> I suppose that's an alternative: I'd be open to splitting up repos however
>> desired if releases were continually published to avoid version number
>> synchronisation efforts, but I'm not so sure how well that meshes with the
>> Apache Way for releasing software.
> 



Re: [log4j] The shape of Log4j

Posted by Scott Deboy <sc...@gmail.com>.
+1

The added complexity of managing the various repo builds and compatibility
outweighs the build-time related cost of adding this code to core, easily.

Just consider the time required to RM these other repos. There is no
savings here to be found.

Scott

On Jan 24, 2018 6:20 PM, "Remko Popma" <re...@gmail.com> wrote:

> I’m still not convinced that moving the items that Ralph enumerated out of
> core will make a significant difference in the time it takes to do a
> release (I believe other measures will be more effective), but fine.
>
> If Ralph wants this as a precondition to continue to perform the Log4j2
> releases then I can accept that. I’m not planning to start doing the Log4j2
> releases and I’m grateful that others are willing to do so.
>
> However, an important question that needs to be answered before we do this
> is: how do we detect when changes in the main Log4j2 project broke one of
> the plugins when they live in another repo?
>
> Creating a plugin API is one option but this will take time to design,
> develop and test.
>
> One possible alternative: Can we set up a continuous integration service
> that builds the main project together with the plugins subproject? (We
> probably need this regardless of whether there is a plugin API or not.) I’m
> open to other solutions but we do need some process in place to detect
> breaking changes.
>
> FYI I strongly feel the perf module should stay in the main project/repo
> or it won’t be useable.
>
> Finally, I think breaking up log4j-plugins into many separate repos is not
> a good idea. It may sound nice on the surface but the overhead is large.
> Are the people pushing for the repo-per-module idea willing to do the
> release work for these plugins? And is everyone else willing to review and
> vote for 10 plugin releases instead of just one? And integrate 10 websites
> instead of one?
>
> (Shameless plug) Every java main() method deserves http://picocli.info
>
> > On Jan 25, 2018, at 9:45, Matt Sicker <bo...@gmail.com> wrote:
> >
> > I'm going to sum up my view here: if we split up repos to include plugins
> > that depend on log4j-core, then we first need to define and release a
> > stable plugin API. If we can't guarantee BC for plugins, then there's no
> > realistic way to both split out additional plugins and have them be
> > officially endorsed plugins. If said plugins were completely outside
> Apache
> > (e.g., personal GitHub pages), then it wouldn't be as strict.
> >
> > Also, I find it hard to believe that Google really uses a monorepo
> anymore.
> > Such an idea wouldn't work too well with software that isn't continuously
> > released.
> >
> > I suppose that's an alternative: I'd be open to splitting up repos
> however
> > desired if releases were continually published to avoid version number
> > synchronisation efforts, but I'm not so sure how well that meshes with
> the
> > Apache Way for releasing software.
>

Re: [log4j] The shape of Log4j

Posted by Remko Popma <re...@gmail.com>.
I’m still not convinced that moving the items that Ralph enumerated out of core will make a significant difference in the time it takes to do a release (I believe other measures will be more effective), but fine. 

If Ralph wants this as a precondition to continue to perform the Log4j2 releases then I can accept that. I’m not planning to start doing the Log4j2 releases and I’m grateful that others are willing to do so. 

However, an important question that needs to be answered before we do this is: how do we detect when changes in the main Log4j2 project broke one of the plugins when they live in another repo?

Creating a plugin API is one option but this will take time to design, develop and test. 

One possible alternative: Can we set up a continuous integration service that builds the main project together with the plugins subproject? (We probably need this regardless of whether there is a plugin API or not.) I’m open to other solutions but we do need some process in place to detect breaking changes. 

FYI I strongly feel the perf module should stay in the main project/repo or it won’t be useable. 

Finally, I think breaking up log4j-plugins into many separate repos is not a good idea. It may sound nice on the surface but the overhead is large. Are the people pushing for the repo-per-module idea willing to do the release work for these plugins? And is everyone else willing to review and vote for 10 plugin releases instead of just one? And integrate 10 websites instead of one?

(Shameless plug) Every java main() method deserves http://picocli.info

> On Jan 25, 2018, at 9:45, Matt Sicker <bo...@gmail.com> wrote:
> 
> I'm going to sum up my view here: if we split up repos to include plugins
> that depend on log4j-core, then we first need to define and release a
> stable plugin API. If we can't guarantee BC for plugins, then there's no
> realistic way to both split out additional plugins and have them be
> officially endorsed plugins. If said plugins were completely outside Apache
> (e.g., personal GitHub pages), then it wouldn't be as strict.
> 
> Also, I find it hard to believe that Google really uses a monorepo anymore.
> Such an idea wouldn't work too well with software that isn't continuously
> released.
> 
> I suppose that's an alternative: I'd be open to splitting up repos however
> desired if releases were continually published to avoid version number
> synchronisation efforts, but I'm not so sure how well that meshes with the
> Apache Way for releasing software.

Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
On 29 January 2018 at 15:04, Gary Gregory <ga...@gmail.com> wrote:
>
> Those two are done in git master.
>

Thanks!


> How would we deal with dependencies on Apache Commons Compress for example?
> Just slice and dice the code to minimize the code depending on CC and put
> that in a new module log4j-compress?
>

I don't see the harm in slicing and dicing as long as we stay consistent in
the new package naming schemes after splitting.

-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Sat, Jan 27, 2018 at 3:07 PM, Matt Sicker <bo...@gmail.com> wrote:

> Moving the Kafka and ZeroMQ appenders to their own modules sounds like a
> good start! Appenders that require third party plugins seem to be a great
> candidate for splitting into their own modules.
>

Those two are done in git master.

How would we deal with dependencies on Apache Commons Compress for example?
Just slice and dice the code to minimize the code depending on CC and put
that in a new module log4j-compress?

Gary


> On 27 January 2018 at 16:06, Gary Gregory <ga...@gmail.com> wrote:
>
> > As a first cut I was thinking of moving out Kafka and ZeroMQ. The other
> > stuff is interesting...
> >
> > Gary
> >
> > On Jan 27, 2018 3:01 PM, "Matt Sicker" <bo...@gmail.com> wrote:
> >
> > > On 27 January 2018 at 10:39, Gary Gregory <ga...@gmail.com>
> > wrote:
> > >
> > > > Should a goal be that log4j-core have no external dependencies?
> > > >
> > >
> > > This would be difficult without shading in LMAX Disruptor. Either that,
> > or
> > > we may need some log4j-core API tweaks to allow the AsyncLogger code to
> > > live in its own module (which would also make it easier to enable async
> > > logging as a user anyways).
> > >
> > > --
> > > Matt Sicker <bo...@gmail.com>
> > >
> >
>
>
>
> --
> Matt Sicker <bo...@gmail.com>
>

Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
Moving the Kafka and ZeroMQ appenders to their own modules sounds like a
good start! Appenders that require third party plugins seem to be a great
candidate for splitting into their own modules.

On 27 January 2018 at 16:06, Gary Gregory <ga...@gmail.com> wrote:

> As a first cut I was thinking of moving out Kafka and ZeroMQ. The other
> stuff is interesting...
>
> Gary
>
> On Jan 27, 2018 3:01 PM, "Matt Sicker" <bo...@gmail.com> wrote:
>
> > On 27 January 2018 at 10:39, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > > Should a goal be that log4j-core have no external dependencies?
> > >
> >
> > This would be difficult without shading in LMAX Disruptor. Either that,
> or
> > we may need some log4j-core API tweaks to allow the AsyncLogger code to
> > live in its own module (which would also make it easier to enable async
> > logging as a user anyways).
> >
> > --
> > Matt Sicker <bo...@gmail.com>
> >
>



-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
As a first cut I was thinking of moving out Kafka and ZeroMQ. The other
stuff is interesting...

Gary

On Jan 27, 2018 3:01 PM, "Matt Sicker" <bo...@gmail.com> wrote:

> On 27 January 2018 at 10:39, Gary Gregory <ga...@gmail.com> wrote:
>
> > Should a goal be that log4j-core have no external dependencies?
> >
>
> This would be difficult without shading in LMAX Disruptor. Either that, or
> we may need some log4j-core API tweaks to allow the AsyncLogger code to
> live in its own module (which would also make it easier to enable async
> logging as a user anyways).
>
> --
> Matt Sicker <bo...@gmail.com>
>

Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
On 27 January 2018 at 10:39, Gary Gregory <ga...@gmail.com> wrote:

> Should a goal be that log4j-core have no external dependencies?
>

This would be difficult without shading in LMAX Disruptor. Either that, or
we may need some log4j-core API tweaks to allow the AsyncLogger code to
live in its own module (which would also make it easier to enable async
logging as a user anyways).

-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Sun, Jan 28, 2018 at 9:54 PM, Ralph Goers <ra...@dslextreme.com>
wrote:

> I didn’t realize that java.base was that limited. Are these all required
> at run time or only build? For example, the annotation processor is not
> used when log4j is running so I would suspect it is required to be present
> in the build and I would think it would have to be present when the
> compiler is running.
>
> Where are we using JNDI?
>
> While scripting support certainly could be optional, it would probably
> uglify the code to do that.
>
> I have mixed feelings about whether it makes sense for JMX to be optional
> or not.
>
> I really can’t see how the xml could be optional.
>

Based on
https://logging.apache.org/log4j/2.x/manual/configuration.html#Properties,
isn't XML optional?

I can I configure everything with Properties?

Gary



>
> Whatever we decide, it will need to be documented.
>
> Ralph
>
>
>
> > On Jan 28, 2018, at 9:35 PM, Matt Sicker <bo...@gmail.com> wrote:
> >
> > That's rather limiting. Here's what we're already using:
> >
> > * java.compiler: annotation processing (could potentially be split, but
> > this situation is already confusing enough for users)
> > * java.management: JMX
> > * java.naming: JNDI
> > * java.scripting: javascript/groovy/etc plugins
> > * java.xml: XML configuration parsing
> >
> > I may have missed some others, but it may be difficult to trim it down to
> > just java.base. Even with some of the simpler ones, we'll end up with
> > several additional modules to detangle that.
> >
> > If we could include multiple logical modules in a single physical module,
> > then this wouldn't be as tedious.
> >
> > On 27 January 2018 at 20:01, Ralph Goers <ra...@dslextreme.com>
> wrote:
> >
> >> Also, in Java 9 it must only require the java.base module.
> >>
> >> Ralph
> >>
> >>> On Jan 27, 2018, at 6:49 PM, Matt Sicker <bo...@gmail.com> wrote:
> >>>
> >>> On 27 January 2018 at 16:18, Ralph Goers <ra...@dslextreme.com>
> >> wrote:
> >>>
> >>>> The requirement is that log4j-core have no required dependencies. I
> >> should
> >>>> have as few optional dependencies as possible.
> >>>>
> >>>
> >>> That sounds perfectly reasonable. LMAX and Jackson are good examples.
> >>>
> >>> --
> >>> Matt Sicker <bo...@gmail.com>
> >>
> >>
> >>
> >
> >
> > --
> > Matt Sicker <bo...@gmail.com>
>
>
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
I didn’t realize that java.base was that limited. Are these all required at run time or only build? For example, the annotation processor is not used when log4j is running so I would suspect it is required to be present in the build and I would think it would have to be present when the compiler is running.

Where are we using JNDI? 

While scripting support certainly could be optional, it would probably uglify the code to do that.

I have mixed feelings about whether it makes sense for JMX to be optional or not.  

I really can’t see how the xml could be optional.

Whatever we decide, it will need to be documented.

Ralph



> On Jan 28, 2018, at 9:35 PM, Matt Sicker <bo...@gmail.com> wrote:
> 
> That's rather limiting. Here's what we're already using:
> 
> * java.compiler: annotation processing (could potentially be split, but
> this situation is already confusing enough for users)
> * java.management: JMX
> * java.naming: JNDI
> * java.scripting: javascript/groovy/etc plugins
> * java.xml: XML configuration parsing
> 
> I may have missed some others, but it may be difficult to trim it down to
> just java.base. Even with some of the simpler ones, we'll end up with
> several additional modules to detangle that.
> 
> If we could include multiple logical modules in a single physical module,
> then this wouldn't be as tedious.
> 
> On 27 January 2018 at 20:01, Ralph Goers <ra...@dslextreme.com> wrote:
> 
>> Also, in Java 9 it must only require the java.base module.
>> 
>> Ralph
>> 
>>> On Jan 27, 2018, at 6:49 PM, Matt Sicker <bo...@gmail.com> wrote:
>>> 
>>> On 27 January 2018 at 16:18, Ralph Goers <ra...@dslextreme.com>
>> wrote:
>>> 
>>>> The requirement is that log4j-core have no required dependencies. I
>> should
>>>> have as few optional dependencies as possible.
>>>> 
>>> 
>>> That sounds perfectly reasonable. LMAX and Jackson are good examples.
>>> 
>>> --
>>> Matt Sicker <bo...@gmail.com>
>> 
>> 
>> 
> 
> 
> -- 
> Matt Sicker <bo...@gmail.com>



Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
JNDI is used in log4j-web for being able to locate a LoggingContext or
configuration (forget which), and there's a JNDI lookup in log4j-core.
Might be other places, but those are the main ones I can remember.

Essentially, anything that relies on javax.* other than for some select
crypto and security stuff are not in java.base. See the listing: <
https://docs.oracle.com/javase/9/docs/api/java.base-summary.html>.

On 29 January 2018 at 08:20, Gary Gregory <ga...@gmail.com> wrote:

> On Sun, Jan 28, 2018 at 9:35 PM, Matt Sicker <bo...@gmail.com> wrote:
>
> > That's rather limiting. Here's what we're already using:
> >
> > * java.compiler: annotation processing (could potentially be split, but
> > this situation is already confusing enough for users)
> > * java.management: JMX
> > * java.naming: JNDI
> > * java.scripting: javascript/groovy/etc plugins
> > * java.xml: XML configuration parsing
> >
>
> Also, some low hanging fruits:
>
> * java.sql: for JDBC
> * JMS depends on a Java EE API jar.
>
> Gary
>
>
> > I may have missed some others, but it may be difficult to trim it down to
> > just java.base. Even with some of the simpler ones, we'll end up with
> > several additional modules to detangle that.
> >
> > If we could include multiple logical modules in a single physical module,
> > then this wouldn't be as tedious.
> >
> > On 27 January 2018 at 20:01, Ralph Goers <ra...@dslextreme.com>
> > wrote:
> >
> > > Also, in Java 9 it must only require the java.base module.
> > >
> > > Ralph
> > >
> > > > On Jan 27, 2018, at 6:49 PM, Matt Sicker <bo...@gmail.com> wrote:
> > > >
> > > > On 27 January 2018 at 16:18, Ralph Goers <ralph.goers@dslextreme.com
> >
> > > wrote:
> > > >
> > > >> The requirement is that log4j-core have no required dependencies. I
> > > should
> > > >> have as few optional dependencies as possible.
> > > >>
> > > >
> > > > That sounds perfectly reasonable. LMAX and Jackson are good examples.
> > > >
> > > > --
> > > > Matt Sicker <bo...@gmail.com>
> > >
> > >
> > >
> >
> >
> > --
> > Matt Sicker <bo...@gmail.com>
> >
>



-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Tue, Jan 30, 2018 at 1:29 PM, Gary Gregory <ga...@gmail.com>
wrote:

>
> On Tue, Jan 30, 2018 at 1:05 PM, Matt Sicker <bo...@gmail.com> wrote:
>
>> It looks like you can use jdeps for this.
>>
>
> The code to support the packages option is commented out :-( even in
> current snapshots:
>
> https://gitbox.apache.org/repos/asf?p=maven-jdeps-
> plugin.git;a=blob;f=src/main/java/org/apache/maven/plugin/
> jdeps/AbstractJDepsMojo.java;h=35b322a7d8683e3dd2c6cfa97370c3
> b25330763a;hb=HEAD
>

Asked: https://issues.apache.org/jira/browse/MJDEPS-12

Gary


>
> Gary
>
>
>>
>> On 29 January 2018 at 16:11, Gary Gregory <ga...@gmail.com> wrote:
>>
>> > On Mon, Jan 29, 2018 at 2:39 PM, Matt Sicker <bo...@gmail.com> wrote:
>> >
>> > > On 29 January 2018 at 13:10, Gary Gregory <ga...@gmail.com>
>> > wrote:
>> > > >
>> > > > Speaking of java.sql (and javax.sql): How can we get the Maven
>> build to
>> > > > FAIL if log4-core Java codes uses java[x].sql code?
>> > > >
>> > >
>> > > I'm not sure if there are any Maven plugins for enforcing this yet,
>> but
>> > I'd
>> > > imagine it'd be something similar to animal sniffer. That, or a custom
>> > > plugin for verifying post-compile (or would it be post-resources).
>> > >
>> >
>> > I wonder if I need to use the animal sniffer to generate a data file
>> just
>> > for java[x].sql or for the JRE except java[x].sql...
>> >
>> > Gary
>> >
>> > >
>> > >
>> > > --
>> > > Matt Sicker <bo...@gmail.com>
>> > >
>> >
>>
>>
>>
>> --
>> Matt Sicker <bo...@gmail.com>
>>
>
>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Tue, Jan 30, 2018 at 1:05 PM, Matt Sicker <bo...@gmail.com> wrote:

> It looks like you can use jdeps for this.
>

The code to support the packages option is commented out :-( even in
current snapshots:

https://gitbox.apache.org/repos/asf?p=maven-jdeps-plugin.git;a=blob;f=src/main/java/org/apache/maven/plugin/jdeps/AbstractJDepsMojo.java;h=35b322a7d8683e3dd2c6cfa97370c3b25330763a;hb=HEAD

Gary


>
> On 29 January 2018 at 16:11, Gary Gregory <ga...@gmail.com> wrote:
>
> > On Mon, Jan 29, 2018 at 2:39 PM, Matt Sicker <bo...@gmail.com> wrote:
> >
> > > On 29 January 2018 at 13:10, Gary Gregory <ga...@gmail.com>
> > wrote:
> > > >
> > > > Speaking of java.sql (and javax.sql): How can we get the Maven build
> to
> > > > FAIL if log4-core Java codes uses java[x].sql code?
> > > >
> > >
> > > I'm not sure if there are any Maven plugins for enforcing this yet, but
> > I'd
> > > imagine it'd be something similar to animal sniffer. That, or a custom
> > > plugin for verifying post-compile (or would it be post-resources).
> > >
> >
> > I wonder if I need to use the animal sniffer to generate a data file just
> > for java[x].sql or for the JRE except java[x].sql...
> >
> > Gary
> >
> > >
> > >
> > > --
> > > Matt Sicker <bo...@gmail.com>
> > >
> >
>
>
>
> --
> Matt Sicker <bo...@gmail.com>
>

Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
It looks like you can use jdeps for this.

On 29 January 2018 at 16:11, Gary Gregory <ga...@gmail.com> wrote:

> On Mon, Jan 29, 2018 at 2:39 PM, Matt Sicker <bo...@gmail.com> wrote:
>
> > On 29 January 2018 at 13:10, Gary Gregory <ga...@gmail.com>
> wrote:
> > >
> > > Speaking of java.sql (and javax.sql): How can we get the Maven build to
> > > FAIL if log4-core Java codes uses java[x].sql code?
> > >
> >
> > I'm not sure if there are any Maven plugins for enforcing this yet, but
> I'd
> > imagine it'd be something similar to animal sniffer. That, or a custom
> > plugin for verifying post-compile (or would it be post-resources).
> >
>
> I wonder if I need to use the animal sniffer to generate a data file just
> for java[x].sql or for the JRE except java[x].sql...
>
> Gary
>
> >
> >
> > --
> > Matt Sicker <bo...@gmail.com>
> >
>



-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 29, 2018 at 2:39 PM, Matt Sicker <bo...@gmail.com> wrote:

> On 29 January 2018 at 13:10, Gary Gregory <ga...@gmail.com> wrote:
> >
> > Speaking of java.sql (and javax.sql): How can we get the Maven build to
> > FAIL if log4-core Java codes uses java[x].sql code?
> >
>
> I'm not sure if there are any Maven plugins for enforcing this yet, but I'd
> imagine it'd be something similar to animal sniffer. That, or a custom
> plugin for verifying post-compile (or would it be post-resources).
>

I wonder if I need to use the animal sniffer to generate a data file just
for java[x].sql or for the JRE except java[x].sql...

Gary

>
>
> --
> Matt Sicker <bo...@gmail.com>
>

Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
On 29 January 2018 at 13:10, Gary Gregory <ga...@gmail.com> wrote:
>
> Speaking of java.sql (and javax.sql): How can we get the Maven build to
> FAIL if log4-core Java codes uses java[x].sql code?
>

I'm not sure if there are any Maven plugins for enforcing this yet, but I'd
imagine it'd be something similar to animal sniffer. That, or a custom
plugin for verifying post-compile (or would it be post-resources).


-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 29, 2018 at 7:20 AM, Gary Gregory <ga...@gmail.com>
wrote:

>
>
> On Sun, Jan 28, 2018 at 9:35 PM, Matt Sicker <bo...@gmail.com> wrote:
>
>> That's rather limiting. Here's what we're already using:
>>
>> * java.compiler: annotation processing (could potentially be split, but
>> this situation is already confusing enough for users)
>> * java.management: JMX
>> * java.naming: JNDI
>> * java.scripting: javascript/groovy/etc plugins
>> * java.xml: XML configuration parsing
>>
>
> Also, some low hanging fruits:
>
> * java.sql: for JDBC
>

Speaking of java.sql (and javax.sql): How can we get the Maven build to
FAIL if log4-core Java codes uses java[x].sql code?

Gary



> * JMS depends on a Java EE API jar.
>
> Gary
>
>
>> I may have missed some others, but it may be difficult to trim it down to
>> just java.base. Even with some of the simpler ones, we'll end up with
>> several additional modules to detangle that.
>>
>> If we could include multiple logical modules in a single physical module,
>> then this wouldn't be as tedious.
>>
>> On 27 January 2018 at 20:01, Ralph Goers <ra...@dslextreme.com>
>> wrote:
>>
>> > Also, in Java 9 it must only require the java.base module.
>> >
>> > Ralph
>> >
>> > > On Jan 27, 2018, at 6:49 PM, Matt Sicker <bo...@gmail.com> wrote:
>> > >
>> > > On 27 January 2018 at 16:18, Ralph Goers <ra...@dslextreme.com>
>> > wrote:
>> > >
>> > >> The requirement is that log4j-core have no required dependencies. I
>> > should
>> > >> have as few optional dependencies as possible.
>> > >>
>> > >
>> > > That sounds perfectly reasonable. LMAX and Jackson are good examples.
>> > >
>> > > --
>> > > Matt Sicker <bo...@gmail.com>
>> >
>> >
>> >
>>
>>
>> --
>> Matt Sicker <bo...@gmail.com>
>>
>
>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Sun, Jan 28, 2018 at 9:35 PM, Matt Sicker <bo...@gmail.com> wrote:

> That's rather limiting. Here's what we're already using:
>
> * java.compiler: annotation processing (could potentially be split, but
> this situation is already confusing enough for users)
> * java.management: JMX
> * java.naming: JNDI
> * java.scripting: javascript/groovy/etc plugins
> * java.xml: XML configuration parsing
>

Also, some low hanging fruits:

* java.sql: for JDBC
* JMS depends on a Java EE API jar.

Gary


> I may have missed some others, but it may be difficult to trim it down to
> just java.base. Even with some of the simpler ones, we'll end up with
> several additional modules to detangle that.
>
> If we could include multiple logical modules in a single physical module,
> then this wouldn't be as tedious.
>
> On 27 January 2018 at 20:01, Ralph Goers <ra...@dslextreme.com>
> wrote:
>
> > Also, in Java 9 it must only require the java.base module.
> >
> > Ralph
> >
> > > On Jan 27, 2018, at 6:49 PM, Matt Sicker <bo...@gmail.com> wrote:
> > >
> > > On 27 January 2018 at 16:18, Ralph Goers <ra...@dslextreme.com>
> > wrote:
> > >
> > >> The requirement is that log4j-core have no required dependencies. I
> > should
> > >> have as few optional dependencies as possible.
> > >>
> > >
> > > That sounds perfectly reasonable. LMAX and Jackson are good examples.
> > >
> > > --
> > > Matt Sicker <bo...@gmail.com>
> >
> >
> >
>
>
> --
> Matt Sicker <bo...@gmail.com>
>

Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
That's rather limiting. Here's what we're already using:

* java.compiler: annotation processing (could potentially be split, but
this situation is already confusing enough for users)
* java.management: JMX
* java.naming: JNDI
* java.scripting: javascript/groovy/etc plugins
* java.xml: XML configuration parsing

I may have missed some others, but it may be difficult to trim it down to
just java.base. Even with some of the simpler ones, we'll end up with
several additional modules to detangle that.

If we could include multiple logical modules in a single physical module,
then this wouldn't be as tedious.

On 27 January 2018 at 20:01, Ralph Goers <ra...@dslextreme.com> wrote:

> Also, in Java 9 it must only require the java.base module.
>
> Ralph
>
> > On Jan 27, 2018, at 6:49 PM, Matt Sicker <bo...@gmail.com> wrote:
> >
> > On 27 January 2018 at 16:18, Ralph Goers <ra...@dslextreme.com>
> wrote:
> >
> >> The requirement is that log4j-core have no required dependencies. I
> should
> >> have as few optional dependencies as possible.
> >>
> >
> > That sounds perfectly reasonable. LMAX and Jackson are good examples.
> >
> > --
> > Matt Sicker <bo...@gmail.com>
>
>
>


-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
Also, in Java 9 it must only require the java.base module.

Ralph

> On Jan 27, 2018, at 6:49 PM, Matt Sicker <bo...@gmail.com> wrote:
> 
> On 27 January 2018 at 16:18, Ralph Goers <ra...@dslextreme.com> wrote:
> 
>> The requirement is that log4j-core have no required dependencies. I should
>> have as few optional dependencies as possible.
>> 
> 
> That sounds perfectly reasonable. LMAX and Jackson are good examples.
> 
> -- 
> Matt Sicker <bo...@gmail.com>



Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
On 27 January 2018 at 16:18, Ralph Goers <ra...@dslextreme.com> wrote:

> The requirement is that log4j-core have no required dependencies. I should
> have as few optional dependencies as possible.
>

That sounds perfectly reasonable. LMAX and Jackson are good examples.

-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.

> On Jan 27, 2018, at 9:39 AM, Gary Gregory <ga...@gmail.com> wrote:
> 
> Should a goal be that log4j-core have no external dependencies?

The requirement is that log4j-core have no required dependencies. I should have as few optional dependencies as possible.

Ralph

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
Should a goal be that log4j-core have no external dependencies?

Gary

On Sat, Jan 27, 2018 at 12:57 AM, Remko Popma <re...@gmail.com> wrote:

> Perhaps https://issues.apache.org/jira/browse/LOG4J2-2226 (Log4j2
> modularization) can also function as the epic for moving towards the plugin
> API that Matt mentioned.
>
> (Shameless plug) Every java main() method deserves http://picocli.info
>
> > On Jan 25, 2018, at 9:45, Matt Sicker <bo...@gmail.com> wrote:
> >
> > I'm going to sum up my view here: if we split up repos to include plugins
> > that depend on log4j-core, then we first need to define and release a
> > stable plugin API. If we can't guarantee BC for plugins, then there's no
> > realistic way to both split out additional plugins and have them be
> > officially endorsed plugins. If said plugins were completely outside
> Apache
> > (e.g., personal GitHub pages), then it wouldn't be as strict.
> >
> > Also, I find it hard to believe that Google really uses a monorepo
> anymore.
> > Such an idea wouldn't work too well with software that isn't continuously
> > released.
> >
> > I suppose that's an alternative: I'd be open to splitting up repos
> however
> > desired if releases were continually published to avoid version number
> > synchronisation efforts, but I'm not so sure how well that meshes with
> the
> > Apache Way for releasing software.
>

Re: [log4j] The shape of Log4j

Posted by Remko Popma <re...@gmail.com>.
Perhaps https://issues.apache.org/jira/browse/LOG4J2-2226 (Log4j2 modularization) can also function as the epic for moving towards the plugin API that Matt mentioned. 

(Shameless plug) Every java main() method deserves http://picocli.info

> On Jan 25, 2018, at 9:45, Matt Sicker <bo...@gmail.com> wrote:
> 
> I'm going to sum up my view here: if we split up repos to include plugins
> that depend on log4j-core, then we first need to define and release a
> stable plugin API. If we can't guarantee BC for plugins, then there's no
> realistic way to both split out additional plugins and have them be
> officially endorsed plugins. If said plugins were completely outside Apache
> (e.g., personal GitHub pages), then it wouldn't be as strict.
> 
> Also, I find it hard to believe that Google really uses a monorepo anymore.
> Such an idea wouldn't work too well with software that isn't continuously
> released.
> 
> I suppose that's an alternative: I'd be open to splitting up repos however
> desired if releases were continually published to avoid version number
> synchronisation efforts, but I'm not so sure how well that meshes with the
> Apache Way for releasing software.

Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
I'm going to sum up my view here: if we split up repos to include plugins
that depend on log4j-core, then we first need to define and release a
stable plugin API. If we can't guarantee BC for plugins, then there's no
realistic way to both split out additional plugins and have them be
officially endorsed plugins. If said plugins were completely outside Apache
(e.g., personal GitHub pages), then it wouldn't be as strict.

Also, I find it hard to believe that Google really uses a monorepo anymore.
Such an idea wouldn't work too well with software that isn't continuously
released.

I suppose that's an alternative: I'd be open to splitting up repos however
desired if releases were continually published to avoid version number
synchronisation efforts, but I'm not so sure how well that meshes with the
Apache Way for releasing software.

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.

> On Jan 24, 2018, at 3:02 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> On Wed, Jan 24, 2018 at 2:33 PM, Ralph Goers <ralph.goers@dslextreme.com <ma...@dslextreme.com>>
> wrote:
> 
>> 
>> 
>>> On Jan 24, 2018, at 1:56 PM, Gary Gregory <ga...@gmail.com>
>> wrote:
>>> 
>>> On Wed, Jan 24, 2018 at 1:18 PM, Ralph Goers <ralph.goers@dslextreme.com
>>> 
>>> wrote:
>>> 
>>>> A vote can’t override a veto of a code commit.
>>>> 
>>> 
>>> I'm trying to get things to move along toward some kind of workable
>>> solution we can all live with. Why are you talking about procedure? It
>> just
>>> feels obstructionist :-(
>>> 
>> 
>> You can view it either way.  I look at your position of insisting the
>> stuff stay in the main build as being obstructionist on the one hand and
>> creating a bloated mess on the other.
>> 
>> I’ve been complaining about the build for at least a year and finally feel
>> the issue needs to be forced.
> 
> 
> Hi All,
> 
> But complaining is not achieving anything. Vetoing commits is not achieving
> anything. It's not a solution :-(
> 
> I am hoping for a concrete solution we all like.
> 
> Right now we have a monorepo. I say that since log4j-tools has never been
> released, forgotten, orphaned. log4j-plugins is EMPTY.

So your problem is simply that you want someone else to do the work of creating the shells for those projects and don’t want to do the work yourself. If that is all it takes I have no problem with that. As soon as I get Log4j Audit released I plain to release log4j-server. Creating a shell for log4j-plugins won’t take much work.

> 
> I'm fine with a monorepo, the advantages are huge.

I see only disadvantages. The advantages you stress are not really there. For example, you continually update versions of dependencies. While that validates we work with the latest release it does nothing to insure we continue to be compatible with the release originally developed for. I could easily argue that dependency versions should only be upgraded when required.

> 
> Someone's going to do a release once in a while and it takes time. I do
> releases in Apache Commons all the time, less often in Apache
> HttpComponents, so I get it, it's a time suck no matter what. Our builds at
> work take anywhere from 15 minutes to... hours and hours. In the grand
> scheme of things Log4j is not a long build IMO at 25-30 minutes for 'mvn
> clean install'.
> 
> With a monorepo and more and more modules, we can slice and dice it with
> profiles to release modules in batches if you like. That's what I'd do, but
> I want us to get to a consensus. I'm not sure how to get there. Imagine I
> start an email thread that says [VOTE] or [POOL] Add log4j-mongodb3 to the
> repo and there are more +1s than -1s, then what?

I would argue that this decision requires consensus, not more +1s than -1s. Unfortunately, the guidelines on what consensus means is less than clear. In some cases it can mean no -1s, in others something over 2/3 majority or more. In general, it is that the “community” is agreeable the outcome.

Ralph

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Wed, Jan 24, 2018 at 2:33 PM, Ralph Goers <ra...@dslextreme.com>
wrote:

>
>
> > On Jan 24, 2018, at 1:56 PM, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > On Wed, Jan 24, 2018 at 1:18 PM, Ralph Goers <ralph.goers@dslextreme.com
> >
> > wrote:
> >
> >> A vote can’t override a veto of a code commit.
> >>
> >
> > I'm trying to get things to move along toward some kind of workable
> > solution we can all live with. Why are you talking about procedure? It
> just
> > feels obstructionist :-(
> >
>
> You can view it either way.  I look at your position of insisting the
> stuff stay in the main build as being obstructionist on the one hand and
> creating a bloated mess on the other.
>
> I’ve been complaining about the build for at least a year and finally feel
> the issue needs to be forced.


Hi All,

But complaining is not achieving anything. Vetoing commits is not achieving
anything. It's not a solution :-(

I am hoping for a concrete solution we all like.

Right now we have a monorepo. I say that since log4j-tools has never been
released, forgotten, orphaned. log4j-plugins is EMPTY.

I'm fine with a monorepo, the advantages are huge.

Someone's going to do a release once in a while and it takes time. I do
releases in Apache Commons all the time, less often in Apache
HttpComponents, so I get it, it's a time suck no matter what. Our builds at
work take anywhere from 15 minutes to... hours and hours. In the grand
scheme of things Log4j is not a long build IMO at 25-30 minutes for 'mvn
clean install'.

With a monorepo and more and more modules, we can slice and dice it with
profiles to release modules in batches if you like. That's what I'd do, but
I want us to get to a consensus. I'm not sure how to get there. Imagine I
start an email thread that says [VOTE] or [POOL] Add log4j-mongodb3 to the
repo and there are more +1s than -1s, then what?

Thank you,
Gary



> Plus, I see us keep adding modules for things that are not related to
> what, in my opinion, is the main purpose of the main repo - to build a Java
> logging framework that meets the vast majority of users needs. Other
> things, like Mongo appenders, are esoteric add-ons that don’t belong there.
> Over time I can see this growing to outlandish proportions as we keep
> adding things that allow you to integrate with whatever the latest thing
> is. To be honest, I am not sure why we don’t have Appenders that write to
> Solr, ElasticSearch, or any/every one of the other things listed at
> http://nosql-database.org/ <http://nosql-database.org/>. I have no
> problem with those integrations but they have no business being in the main
> build.  I just don’t see how you could possibly believe that having
> integrations with all those components in the main build would be a good
> thing.
>
> This discussion does make me wonder how often you perform a full build. I
> found it telling that your guidance in the pull request left off running a
> full build to detect whether the fix broke anything. As I have said before,
> I almost always run a full build before I commit. Based on issues I’ve
> found in the past with some of your commits I am pretty sure you don’t do
> this, which is why you probably don’t consider this to be as big a problem
> as I do.
>
>
> Ralph
>
>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Wed, Jan 24, 2018 at 2:33 PM, Ralph Goers <ra...@dslextreme.com>
wrote:

>
>
> > On Jan 24, 2018, at 1:56 PM, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > On Wed, Jan 24, 2018 at 1:18 PM, Ralph Goers <ralph.goers@dslextreme.com
> >
> > wrote:
> >
> >> A vote can’t override a veto of a code commit.
> >>
> >
> > I'm trying to get things to move along toward some kind of workable
> > solution we can all live with. Why are you talking about procedure? It
> just
> > feels obstructionist :-(
> >
>
> You can view it either way.  I look at your position of insisting the
> stuff stay in the main build as being obstructionist on the one hand and
> creating a bloated mess on the other.
>
> I’ve been complaining about the build for at least a year and finally feel
> the issue needs to be forced. Plus, I see us keep adding modules for things
> that are not related to what, in my opinion, is the main purpose of the
> main repo - to build a Java logging framework that meets the vast majority
> of users needs. Other things, like Mongo appenders, are esoteric add-ons
> that don’t belong there. Over time I can see this growing to outlandish
> proportions as we keep adding things that allow you to integrate with
> whatever the latest thing is. To be honest, I am not sure why we don’t have
> Appenders that write to Solr, ElasticSearch, or any/every one of the other
> things listed at http://nosql-database.org/ <http://nosql-database.org/>.
> I have no problem with those integrations but they have no business being
> in the main build.  I just don’t see how you could possibly believe that
> having integrations with all those components in the main build would be a
> good thing.
>
> This discussion does make me wonder how often you perform a full build. I
> found it telling that your guidance in the pull request left off running a
> full build to detect whether the fix broke anything. As I have said before,
> I almost always run a full build before I commit. Based on issues I’ve
> found in the past with some of your commits I am pretty sure you don’t do
> this, which is why you probably don’t consider this to be as big a problem
> as I do.
>

I usually do not run a full build if I change an appender like MongoDB.
Nothing I can change in there should break anything else. Usually.
I usually do run a full build if I change something in log4j-core.
I do not like to waste my time so I do not even run a build if I change a
Javadoc. But sometimes I do, to make sure the Eclipse and Oracle compiler
see things the same way. Or, if I make a change in generics since the
Eclipse compiler is more lenient than Oracle.

Gary


>
> Ralph
>
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.

> On Jan 24, 2018, at 1:56 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> On Wed, Jan 24, 2018 at 1:18 PM, Ralph Goers <ra...@dslextreme.com>
> wrote:
> 
>> A vote can’t override a veto of a code commit.
>> 
> 
> I'm trying to get things to move along toward some kind of workable
> solution we can all live with. Why are you talking about procedure? It just
> feels obstructionist :-(
> 

You can view it either way.  I look at your position of insisting the stuff stay in the main build as being obstructionist on the one hand and creating a bloated mess on the other.

I’ve been complaining about the build for at least a year and finally feel the issue needs to be forced. Plus, I see us keep adding modules for things that are not related to what, in my opinion, is the main purpose of the main repo - to build a Java logging framework that meets the vast majority of users needs. Other things, like Mongo appenders, are esoteric add-ons that don’t belong there. Over time I can see this growing to outlandish proportions as we keep adding things that allow you to integrate with whatever the latest thing is. To be honest, I am not sure why we don’t have Appenders that write to Solr, ElasticSearch, or any/every one of the other things listed at http://nosql-database.org/ <http://nosql-database.org/>. I have no problem with those integrations but they have no business being in the main build.  I just don’t see how you could possibly believe that having integrations with all those components in the main build would be a good thing.

This discussion does make me wonder how often you perform a full build. I found it telling that your guidance in the pull request left off running a full build to detect whether the fix broke anything. As I have said before, I almost always run a full build before I commit. Based on issues I’ve found in the past with some of your commits I am pretty sure you don’t do this, which is why you probably don’t consider this to be as big a problem as I do.


Ralph


Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Wed, Jan 24, 2018 at 1:18 PM, Ralph Goers <ra...@dslextreme.com>
wrote:

> A vote can’t override a veto of a code commit.
>

I'm trying to get things to move along toward some kind of workable
solution we can all live with. Why are you talking about procedure? It just
feels obstructionist :-(

Gary


> Ralph
>
> > On Jan 24, 2018, at 1:00 PM, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > I really feel like we're not getting anywhere here :-(
> >
> > I have a log4j-mongodb3 appender ready to commit and nowhere to put it.
> The
> > empty repo/kitchen-sink/dumping-grounds log4j-plugins is not acceptable
> to
> > me.
> >
> > Ralph does not want it in the main repo. I say it's fine.
> >
> > Now what?
> >
> > Should I call a VOTE?
> >
> > Gary
> >
> >
> > On Tue, Jan 23, 2018 at 7:40 AM, Ralph Goers <ralph.goers@dslextreme.com
> >
> > wrote:
> >
> >>
> >>
> >>> On Jan 23, 2018, at 6:51 AM, Gary Gregory <ga...@gmail.com>
> >> wrote:
> >>>
> >>> On Jan 23, 2018 5:09 AM, "Mikael Ståldal" <mi...@apache.org> wrote:
> >>>
> >>> There is no NoSQL appender. There are Cassandra, CouchDB and MongoDB
> >>> appenders. I don't think we should bundle them together.
> >>>
> >>>
> >>> Note that these are already separate modules/jar produced out of the
> main
> >>> repo.
> >>
> >> Actually, they are produced “in” the main repo. If they were “out of”
> the
> >> main repo I wouldn’t be complaining ;-)
> >>
> >> Ralph
> >>
>
>
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
A vote can’t override a veto of a code commit.

Ralph

> On Jan 24, 2018, at 1:00 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> I really feel like we're not getting anywhere here :-(
> 
> I have a log4j-mongodb3 appender ready to commit and nowhere to put it. The
> empty repo/kitchen-sink/dumping-grounds log4j-plugins is not acceptable to
> me.
> 
> Ralph does not want it in the main repo. I say it's fine.
> 
> Now what?
> 
> Should I call a VOTE?
> 
> Gary
> 
> 
> On Tue, Jan 23, 2018 at 7:40 AM, Ralph Goers <ra...@dslextreme.com>
> wrote:
> 
>> 
>> 
>>> On Jan 23, 2018, at 6:51 AM, Gary Gregory <ga...@gmail.com>
>> wrote:
>>> 
>>> On Jan 23, 2018 5:09 AM, "Mikael Ståldal" <mi...@apache.org> wrote:
>>> 
>>> There is no NoSQL appender. There are Cassandra, CouchDB and MongoDB
>>> appenders. I don't think we should bundle them together.
>>> 
>>> 
>>> Note that these are already separate modules/jar produced out of the main
>>> repo.
>> 
>> Actually, they are produced “in” the main repo. If they were “out of” the
>> main repo I wouldn’t be complaining ;-)
>> 
>> Ralph
>> 



Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Wed, Jan 24, 2018 at 1:27 PM, Ralph Goers <ra...@dslextreme.com>
wrote:

> You left off two of the other options:
> 1. A repo for related plugins.
> 2. A repo for each plugin.
>
> I am open to any option where these plugins are not included in the main
> repo. It sounds like the only option you are open to is including them in
> the main repo.
>

I just see too many downsides to both. That's a lot of repos that will
suffer from breaking and orphaning (is that even a word?) We are not Maven
that has a better defined SPI. If the build time is a concern, I would
rather see the build adapt to (using modules for example) releasing
different batches of jars.

Gary


>
> Ralph
>
> > On Jan 24, 2018, at 1:00 PM, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > I really feel like we're not getting anywhere here :-(
> >
> > I have a log4j-mongodb3 appender ready to commit and nowhere to put it.
> The
> > empty repo/kitchen-sink/dumping-grounds log4j-plugins is not acceptable
> to
> > me.
> >
> > Ralph does not want it in the main repo. I say it's fine.
> >
> > Now what?
> >
> > Should I call a VOTE?
> >
> > Gary
> >
> >
> > On Tue, Jan 23, 2018 at 7:40 AM, Ralph Goers <ralph.goers@dslextreme.com
> >
> > wrote:
> >
> >>
> >>
> >>> On Jan 23, 2018, at 6:51 AM, Gary Gregory <ga...@gmail.com>
> >> wrote:
> >>>
> >>> On Jan 23, 2018 5:09 AM, "Mikael Ståldal" <mi...@apache.org> wrote:
> >>>
> >>> There is no NoSQL appender. There are Cassandra, CouchDB and MongoDB
> >>> appenders. I don't think we should bundle them together.
> >>>
> >>>
> >>> Note that these are already separate modules/jar produced out of the
> main
> >>> repo.
> >>
> >> Actually, they are produced “in” the main repo. If they were “out of”
> the
> >> main repo I wouldn’t be complaining ;-)
> >>
> >> Ralph
> >>
>
>
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
You left off two of the other options:
1. A repo for related plugins.
2. A repo for each plugin.

I am open to any option where these plugins are not included in the main repo. It sounds like the only option you are open to is including them in the main repo.

Ralph

> On Jan 24, 2018, at 1:00 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> I really feel like we're not getting anywhere here :-(
> 
> I have a log4j-mongodb3 appender ready to commit and nowhere to put it. The
> empty repo/kitchen-sink/dumping-grounds log4j-plugins is not acceptable to
> me.
> 
> Ralph does not want it in the main repo. I say it's fine.
> 
> Now what?
> 
> Should I call a VOTE?
> 
> Gary
> 
> 
> On Tue, Jan 23, 2018 at 7:40 AM, Ralph Goers <ra...@dslextreme.com>
> wrote:
> 
>> 
>> 
>>> On Jan 23, 2018, at 6:51 AM, Gary Gregory <ga...@gmail.com>
>> wrote:
>>> 
>>> On Jan 23, 2018 5:09 AM, "Mikael Ståldal" <mi...@apache.org> wrote:
>>> 
>>> There is no NoSQL appender. There are Cassandra, CouchDB and MongoDB
>>> appenders. I don't think we should bundle them together.
>>> 
>>> 
>>> Note that these are already separate modules/jar produced out of the main
>>> repo.
>> 
>> Actually, they are produced “in” the main repo. If they were “out of” the
>> main repo I wouldn’t be complaining ;-)
>> 
>> Ralph
>> 



Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
I really feel like we're not getting anywhere here :-(

I have a log4j-mongodb3 appender ready to commit and nowhere to put it. The
empty repo/kitchen-sink/dumping-grounds log4j-plugins is not acceptable to
me.

Ralph does not want it in the main repo. I say it's fine.

Now what?

Should I call a VOTE?

Gary


On Tue, Jan 23, 2018 at 7:40 AM, Ralph Goers <ra...@dslextreme.com>
wrote:

>
>
> > On Jan 23, 2018, at 6:51 AM, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > On Jan 23, 2018 5:09 AM, "Mikael Ståldal" <mi...@apache.org> wrote:
> >
> > There is no NoSQL appender. There are Cassandra, CouchDB and MongoDB
> > appenders. I don't think we should bundle them together.
> >
> >
> > Note that these are already separate modules/jar produced out of the main
> > repo.
>
> Actually, they are produced “in” the main repo. If they were “out of” the
> main repo I wouldn’t be complaining ;-)
>
> Ralph
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.

> On Jan 23, 2018, at 6:51 AM, Gary Gregory <ga...@gmail.com> wrote:
> 
> On Jan 23, 2018 5:09 AM, "Mikael Ståldal" <mi...@apache.org> wrote:
> 
> There is no NoSQL appender. There are Cassandra, CouchDB and MongoDB
> appenders. I don't think we should bundle them together.
> 
> 
> Note that these are already separate modules/jar produced out of the main
> repo.

Actually, they are produced “in” the main repo. If they were “out of” the main repo I wouldn’t be complaining ;-)

Ralph

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Jan 23, 2018 5:09 AM, "Mikael Ståldal" <mi...@apache.org> wrote:

There is no NoSQL appender. There are Cassandra, CouchDB and MongoDB
appenders. I don't think we should bundle them together.


Note that these are already separate modules/jar produced out of the main
repo.

Gary




On 2018-01-22 23:29, Ralph Goers wrote:

> If it was up to me I would move the following Appenders: Cassandra, Flume,
> JDBC, JMS, JPA, HTTP, Kafka, NoSQL, SMTP, ZeroMQ/JeroMQ, CouchDB, MongoDB.
> In addition to those I would move the taglib, imx-gui and samples modules
> and possibly the perf module. Although it is probably lightly used I would
> consider keeping the iostreams module, although to be honest I am not sure
> why it isn’t part of the api module.
>

Re: [log4j] The shape of Log4j

Posted by Mikael Ståldal <mi...@apache.org>.
There is no NoSQL appender. There are Cassandra, CouchDB and MongoDB 
appenders. I don't think we should bundle them together.


On 2018-01-22 23:29, Ralph Goers wrote:
> If it was up to me I would move the following Appenders: Cassandra, Flume, JDBC, JMS, JPA, HTTP, Kafka, NoSQL, SMTP, ZeroMQ/JeroMQ, CouchDB, MongoDB. In addition to those I would move the taglib, imx-gui and samples modules and possibly the perf module. Although it is probably lightly used I would consider keeping the iostreams module, although to be honest I am not sure why it isn’t part of the api module.

Re: [log4j] The shape of Log4j

Posted by Remko Popma <re...@gmail.com>.
> On Jan 23, 2018, at 7:29, Ralph Goers <ra...@dslextreme.com> wrote:
> 
> IMO the main repo should contain the stuff 80% (or more) of our user’s use and the stuff that is common across all plugins.  So the file appenders and console appenders all belong, probably most of the existing lookups and filters. Since the Syslog Appender probably fits in I would also keep the socket appenders as well.  
> 
> If it was up to me I would move the following Appenders: Cassandra, Flume, JDBC, JMS, JPA, HTTP, Kafka, NoSQL, SMTP, ZeroMQ/JeroMQ, CouchDB, MongoDB.
> In addition to those I would move the taglib, imx-gui and samples modules and possibly the perf module.

Two questions:
1. How much faster do you estimate this will make the build?
2. How do we verify that changes in log4j-core didn’t break any of the plugins?


About the perf module:
If you take perf out of the main repo we will only be able to create performance tests against pre-built binaries. This is impractical and the added overhead likely means performance testing becomes too cumbersome for anyone to do it any more. I think that’s a bad idea. 


> Although it is probably lightly used I would consider keeping the iostreams module, although to be honest I am not sure why it isn’t part of the api module.
> 
> Ralph
> 
>> On Jan 22, 2018, at 2:19 PM, Gary Gregory <ga...@gmail.com> wrote:
>> 
>> I'm curious: What is threshold where it would be OK to add stuff to the
>> main repo?
>> 
>> Gary
>> 
>> On Mon, Jan 22, 2018 at 12:48 PM, Ralph Goers <ra...@dslextreme.com>
>> wrote:
>> 
>>> I don’t see why this work would require 3.0 as we aren’t planning on
>>> breaking any contracts to do this.
>>> 
>>> As I’ve said, I am not tied to each plugin having its own repo. I am OK
>>> with these options; a) each plugin has its own repo and site and is
>>> released independently, b) we use the plugins repo and move all the more
>>> lightly used components there. It would have its own site, or c) we group
>>> plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
>>> similar) with each having its own site.
>>> 
>>> Obviously I do not consider continuing to add new plugins to the main
>>> build as one of the options.
>>> 
>>> Ralph
>>> 
>>>> On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:
>>>> 
>>>> This whole conversation just reminds me of all the times in the past I
>>>> suggested full modularization of the core API/SPI. I think such an effort
>>>> would make sense in a 3.0 release.
>>>> 
>>>> In 2.x, perhaps what we can do is define a stable plugin API based on
>>> what
>>>> we already have. Since we're still supporting Java 7, I'd suggest we make
>>>> abstract classes be the stable part of the API since we don't have
>>> default
>>>> methods on interfaces here.
>>>> 
>>>> And yes, the GitBox integration thing sounds neat, particularly for code
>>>> review, though that's a separate topic entirely.
>>>> 
>>>> On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com>
>>> wrote:
>>>> 
>>>>> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <
>>> ralph.goers@dslextreme.com>
>>>>> wrote:
>>>>> 
>>>>>> I am very much against the idea of a single repo. Yes, you can have
>>>>>> multiple projects in the repo but I am not sure how that can be sanely
>>>>>> released. I much prefer the model that Maven has taken. They are now
>>>>> using
>>>>>> gitbox [1] which seems to allow GitHub to be the primary repo.
>>>>> 
>>>>> 
>>>>> GitHub is a distracting tangent here IMO. I really like GitHub's code
>>>>> commenting feature, but it should not matter if we use Apache's Git
>>> hosted
>>>>> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
>>>>> conversation... ;-)
>>>>> 
>>>>> Gary
>>>>> 
>>>>> 
>>>>>> Every Maven plugin is individually released. Scroll down the link below
>>>>> to
>>>>>> the Maven section and you can see all the plugin repos.
>>>>>> 
>>>>>> The upside to this is that it are:
>>>>>>      1. It is far easier to perform releases of the individual
>>>>>> components.
>>>>>>      2. It is much easier to accept plugin contributions.
>>>>>> The downsides are:
>>>>>>      1. A page like https://maven.apache.org/plugins/ <
>>>>>> https://maven.apache.org/plugins/> is needed to keep track of the
>>> plugin
>>>>>> versions.
>>>>>>      2. It could make sense to have log4j-parent and log4j-bom
>>>>>> projects. The first to help keep the builds similar and the second to
>>>>> help
>>>>>> customers pick up the latest versions.
>>>>>> 
>>>>>> Ralph
>>>>>> 
>>>>>> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
>>>>>> repos/asf>
>>>>>> 
>>>>>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Hi All,
>>>>>>> 
>>>>>>> I am writing this to in an effort to understand how we can manage and
>>>>>> grow
>>>>>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense but
>>>>> more
>>>>>>> in the maturing sense.
>>>>>>> 
>>>>>>> I am prompted to write this by Ralph's comment that log4j-core and the
>>>>>> main
>>>>>>> repo is too big and that releases take too long make, partially
>>>>> prompted
>>>>>> by
>>>>>>> my addition of a new module called log4j-jdbc-dbcp2 which currently
>>>>>>> contains one small class with a dependency on Apache Commons DBCP 2.
>>>>>>> 
>>>>>>> I would like to express my gratitude at Ralph's efforts to revitalized
>>>>>>> Log4j and performing most release management duties. Thank you Ralph!
>>>>>>> 
>>>>>>> I can see two main orthogonal issues:
>>>>>>> - The size of the git repo.
>>>>>>> - The size of the log4j-core module.
>>>>>>> 
>>>>>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
>>>>>>> another 'plugin' repo.
>>>>>>> 
>>>>>>> I really dislike this idea:
>>>>>>> - The plugin repo has never been released. Not that one releases a
>>> repo
>>>>>> but
>>>>>>> you get the point.
>>>>>>> - How do you keep things in sync between repos and code when we have
>>> no
>>>>>>> official 'core' SPI.
>>>>>>> 
>>>>>>> For my money, we should keep _everything_ in one repo. Good enough for
>>>>>>> Google, so good enough for me. What we release out of that repo is a
>>>>>>> different story and what I would like to discuss next.
>>>>>>> 
>>>>>>> This is not the same as Maven plugins IMO but the case could be made
>>>>> for
>>>>>> it
>>>>>>> I suppose where a lot/most plugins live in their own repos. It is not
>>>>> the
>>>>>>> same as with Maven IMO because our plugins rely on log4j-core and it's
>>>>>>> guts, for which we only make loose compatibility guarantees -- as
>>>>> opposed
>>>>>>> to log4j-core where we are strict(er). Maven OTOH, has a API for
>>> plugin
>>>>>>> auteurs.
>>>>>>> 
>>>>>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we
>>>>>> have
>>>>>>> no 'official' core SPI, so splitting it off into a separate repo would
>>>>>>> greatly increase the risk of it falling out of sync. It is just so
>>> much
>>>>>>> more easier to maintain when it is all in one repo.
>>>>>>> 
>>>>>>> My proposal is to:
>>>>>>> 
>>>>>>> - Put everything back into one repo (Chainsaw too?)
>>>>>>> - Define a core SPI for plugin writers where we make some statement
>>>>> about
>>>>>>> BC, more than the casual 'we try not break stuff.'
>>>>>>> - Defining what Log4j project 'components' we have and release based
>>> on
>>>>>>> that. For example, today, all of the main Log4j repo is one component
>>>>>> with
>>>>>>> many modules. Chainsaw would be another component of the Log4j
>>> project.
>>>>>>> Maybe we need to redefine components: API, File, JDBC and so on. A
>>>>>>> component can have one of more module
>>>>>>> - Got for there.
>>>>>>> 
>>>>>>> Thoughts? Help me flush this out?
>>>>>>> 
>>>>>>> Thank you for reading!
>>>>>>> Gary
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Matt Sicker <bo...@gmail.com>
>>> 
>>> 
>>> 
> 
> 

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Wed, Jan 24, 2018 at 2:15 PM, Mikael Ståldal <mi...@apache.org> wrote:

> On 2018-01-22 23:37, Matt Sicker wrote:
>
>> On 22 January 2018 at 16:29, Ralph Goers <ra...@dslextreme.com>
>> wrote:
>>
>>>
>>> If it was up to me I would move the following Appenders: Cassandra,
>>> Flume,
>>> JDBC, JMS, JPA, HTTP, Kafka, NoSQL, SMTP, ZeroMQ/JeroMQ, CouchDB,
>>> MongoDB.
>>> In addition to those I would move the taglib, imx-gui and samples modules
>>> and possibly the perf module. Although it is probably lightly used I
>>> would
>>> consider keeping the iostreams module, although to be honest I am not
>>> sure
>>> why it isn’t part of the api module.
>>>
>>>
>> I like this idea generally since those plugins are rarely updated after
>> being developed.
>>
>
> At least Kafka appender has been updated a number of times since it was
> incepted.
>
> > The iostreams API could be added to log4j-api.
>
> I think we should be very careful about adding new stuff to log4j-api.
>

Agreed, it seems fines where it is.

Gary

Re: [log4j] The shape of Log4j

Posted by Mikael Ståldal <mi...@apache.org>.
On 2018-01-22 23:37, Matt Sicker wrote:
> On 22 January 2018 at 16:29, Ralph Goers <ra...@dslextreme.com> wrote:
>>
>> If it was up to me I would move the following Appenders: Cassandra, Flume,
>> JDBC, JMS, JPA, HTTP, Kafka, NoSQL, SMTP, ZeroMQ/JeroMQ, CouchDB, MongoDB.
>> In addition to those I would move the taglib, imx-gui and samples modules
>> and possibly the perf module. Although it is probably lightly used I would
>> consider keeping the iostreams module, although to be honest I am not sure
>> why it isn’t part of the api module.
>>
> 
> I like this idea generally since those plugins are rarely updated after
> being developed. 

At least Kafka appender has been updated a number of times since it was 
incepted.

 > The iostreams API could be added to log4j-api.

I think we should be very careful about adding new stuff to log4j-api.

Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
On 22 January 2018 at 16:29, Ralph Goers <ra...@dslextreme.com> wrote:

> IMO the main repo should contain the stuff 80% (or more) of our user’s use
> and the stuff that is common across all plugins.  So the file appenders and
> console appenders all belong, probably most of the existing lookups and
> filters. Since the Syslog Appender probably fits in I would also keep the
> socket appenders as well.
>
> If it was up to me I would move the following Appenders: Cassandra, Flume,
> JDBC, JMS, JPA, HTTP, Kafka, NoSQL, SMTP, ZeroMQ/JeroMQ, CouchDB, MongoDB.
> In addition to those I would move the taglib, imx-gui and samples modules
> and possibly the perf module. Although it is probably lightly used I would
> consider keeping the iostreams module, although to be honest I am not sure
> why it isn’t part of the api module.
>

I like this idea generally since those plugins are rarely updated after
being developed. The iostreams API could be added to log4j-api. The actual
implementation details of how to go about separating repos, though, seems
to be the complicated part.


>
> Ralph
>
> > On Jan 22, 2018, at 2:19 PM, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > I'm curious: What is threshold where it would be OK to add stuff to the
> > main repo?
> >
> > Gary
> >
> > On Mon, Jan 22, 2018 at 12:48 PM, Ralph Goers <
> ralph.goers@dslextreme.com>
> > wrote:
> >
> >> I don’t see why this work would require 3.0 as we aren’t planning on
> >> breaking any contracts to do this.
> >>
> >> As I’ve said, I am not tied to each plugin having its own repo. I am OK
> >> with these options; a) each plugin has its own repo and site and is
> >> released independently, b) we use the plugins repo and move all the more
> >> lightly used components there. It would have its own site, or c) we
> group
> >> plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
> >> similar) with each having its own site.
> >>
> >> Obviously I do not consider continuing to add new plugins to the main
> >> build as one of the options.
> >>
> >> Ralph
> >>
> >>> On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:
> >>>
> >>> This whole conversation just reminds me of all the times in the past I
> >>> suggested full modularization of the core API/SPI. I think such an
> effort
> >>> would make sense in a 3.0 release.
> >>>
> >>> In 2.x, perhaps what we can do is define a stable plugin API based on
> >> what
> >>> we already have. Since we're still supporting Java 7, I'd suggest we
> make
> >>> abstract classes be the stable part of the API since we don't have
> >> default
> >>> methods on interfaces here.
> >>>
> >>> And yes, the GitBox integration thing sounds neat, particularly for
> code
> >>> review, though that's a separate topic entirely.
> >>>
> >>> On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com>
> >> wrote:
> >>>
> >>>> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <
> >> ralph.goers@dslextreme.com>
> >>>> wrote:
> >>>>
> >>>>> I am very much against the idea of a single repo. Yes, you can have
> >>>>> multiple projects in the repo but I am not sure how that can be
> sanely
> >>>>> released. I much prefer the model that Maven has taken. They are now
> >>>> using
> >>>>> gitbox [1] which seems to allow GitHub to be the primary repo.
> >>>>
> >>>>
> >>>> GitHub is a distracting tangent here IMO. I really like GitHub's code
> >>>> commenting feature, but it should not matter if we use Apache's Git
> >> hosted
> >>>> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
> >>>> conversation... ;-)
> >>>>
> >>>> Gary
> >>>>
> >>>>
> >>>>> Every Maven plugin is individually released. Scroll down the link
> below
> >>>> to
> >>>>> the Maven section and you can see all the plugin repos.
> >>>>>
> >>>>> The upside to this is that it are:
> >>>>>       1. It is far easier to perform releases of the individual
> >>>>> components.
> >>>>>       2. It is much easier to accept plugin contributions.
> >>>>> The downsides are:
> >>>>>       1. A page like https://maven.apache.org/plugins/ <
> >>>>> https://maven.apache.org/plugins/> is needed to keep track of the
> >> plugin
> >>>>> versions.
> >>>>>       2. It could make sense to have log4j-parent and log4j-bom
> >>>>> projects. The first to help keep the builds similar and the second to
> >>>> help
> >>>>> customers pick up the latest versions.
> >>>>>
> >>>>> Ralph
> >>>>>
> >>>>> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
> >>>>> repos/asf>
> >>>>>
> >>>>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi All,
> >>>>>>
> >>>>>> I am writing this to in an effort to understand how we can manage
> and
> >>>>> grow
> >>>>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense but
> >>>> more
> >>>>>> in the maturing sense.
> >>>>>>
> >>>>>> I am prompted to write this by Ralph's comment that log4j-core and
> the
> >>>>> main
> >>>>>> repo is too big and that releases take too long make, partially
> >>>> prompted
> >>>>> by
> >>>>>> my addition of a new module called log4j-jdbc-dbcp2 which currently
> >>>>>> contains one small class with a dependency on Apache Commons DBCP 2.
> >>>>>>
> >>>>>> I would like to express my gratitude at Ralph's efforts to
> revitalized
> >>>>>> Log4j and performing most release management duties. Thank you
> Ralph!
> >>>>>>
> >>>>>> I can see two main orthogonal issues:
> >>>>>> - The size of the git repo.
> >>>>>> - The size of the log4j-core module.
> >>>>>>
> >>>>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved
> to
> >>>>>> another 'plugin' repo.
> >>>>>>
> >>>>>> I really dislike this idea:
> >>>>>> - The plugin repo has never been released. Not that one releases a
> >> repo
> >>>>> but
> >>>>>> you get the point.
> >>>>>> - How do you keep things in sync between repos and code when we have
> >> no
> >>>>>> official 'core' SPI.
> >>>>>>
> >>>>>> For my money, we should keep _everything_ in one repo. Good enough
> for
> >>>>>> Google, so good enough for me. What we release out of that repo is a
> >>>>>> different story and what I would like to discuss next.
> >>>>>>
> >>>>>> This is not the same as Maven plugins IMO but the case could be made
> >>>> for
> >>>>> it
> >>>>>> I suppose where a lot/most plugins live in their own repos. It is
> not
> >>>> the
> >>>>>> same as with Maven IMO because our plugins rely on log4j-core and
> it's
> >>>>>> guts, for which we only make loose compatibility guarantees -- as
> >>>> opposed
> >>>>>> to log4j-core where we are strict(er). Maven OTOH, has a API for
> >> plugin
> >>>>>> auteurs.
> >>>>>>
> >>>>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and
> we
> >>>>> have
> >>>>>> no 'official' core SPI, so splitting it off into a separate repo
> would
> >>>>>> greatly increase the risk of it falling out of sync. It is just so
> >> much
> >>>>>> more easier to maintain when it is all in one repo.
> >>>>>>
> >>>>>> My proposal is to:
> >>>>>>
> >>>>>> - Put everything back into one repo (Chainsaw too?)
> >>>>>> - Define a core SPI for plugin writers where we make some statement
> >>>> about
> >>>>>> BC, more than the casual 'we try not break stuff.'
> >>>>>> - Defining what Log4j project 'components' we have and release based
> >> on
> >>>>>> that. For example, today, all of the main Log4j repo is one
> component
> >>>>> with
> >>>>>> many modules. Chainsaw would be another component of the Log4j
> >> project.
> >>>>>> Maybe we need to redefine components: API, File, JDBC and so on. A
> >>>>>> component can have one of more module
> >>>>>> - Got for there.
> >>>>>>
> >>>>>> Thoughts? Help me flush this out?
> >>>>>>
> >>>>>> Thank you for reading!
> >>>>>> Gary
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Matt Sicker <bo...@gmail.com>
> >>
> >>
> >>
>
>
>


-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
IMO the main repo should contain the stuff 80% (or more) of our user’s use and the stuff that is common across all plugins.  So the file appenders and console appenders all belong, probably most of the existing lookups and filters. Since the Syslog Appender probably fits in I would also keep the socket appenders as well.  

If it was up to me I would move the following Appenders: Cassandra, Flume, JDBC, JMS, JPA, HTTP, Kafka, NoSQL, SMTP, ZeroMQ/JeroMQ, CouchDB, MongoDB. In addition to those I would move the taglib, imx-gui and samples modules and possibly the perf module. Although it is probably lightly used I would consider keeping the iostreams module, although to be honest I am not sure why it isn’t part of the api module.

Ralph

> On Jan 22, 2018, at 2:19 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> I'm curious: What is threshold where it would be OK to add stuff to the
> main repo?
> 
> Gary
> 
> On Mon, Jan 22, 2018 at 12:48 PM, Ralph Goers <ra...@dslextreme.com>
> wrote:
> 
>> I don’t see why this work would require 3.0 as we aren’t planning on
>> breaking any contracts to do this.
>> 
>> As I’ve said, I am not tied to each plugin having its own repo. I am OK
>> with these options; a) each plugin has its own repo and site and is
>> released independently, b) we use the plugins repo and move all the more
>> lightly used components there. It would have its own site, or c) we group
>> plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
>> similar) with each having its own site.
>> 
>> Obviously I do not consider continuing to add new plugins to the main
>> build as one of the options.
>> 
>> Ralph
>> 
>>> On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:
>>> 
>>> This whole conversation just reminds me of all the times in the past I
>>> suggested full modularization of the core API/SPI. I think such an effort
>>> would make sense in a 3.0 release.
>>> 
>>> In 2.x, perhaps what we can do is define a stable plugin API based on
>> what
>>> we already have. Since we're still supporting Java 7, I'd suggest we make
>>> abstract classes be the stable part of the API since we don't have
>> default
>>> methods on interfaces here.
>>> 
>>> And yes, the GitBox integration thing sounds neat, particularly for code
>>> review, though that's a separate topic entirely.
>>> 
>>> On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com>
>> wrote:
>>> 
>>>> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <
>> ralph.goers@dslextreme.com>
>>>> wrote:
>>>> 
>>>>> I am very much against the idea of a single repo. Yes, you can have
>>>>> multiple projects in the repo but I am not sure how that can be sanely
>>>>> released. I much prefer the model that Maven has taken. They are now
>>>> using
>>>>> gitbox [1] which seems to allow GitHub to be the primary repo.
>>>> 
>>>> 
>>>> GitHub is a distracting tangent here IMO. I really like GitHub's code
>>>> commenting feature, but it should not matter if we use Apache's Git
>> hosted
>>>> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
>>>> conversation... ;-)
>>>> 
>>>> Gary
>>>> 
>>>> 
>>>>> Every Maven plugin is individually released. Scroll down the link below
>>>> to
>>>>> the Maven section and you can see all the plugin repos.
>>>>> 
>>>>> The upside to this is that it are:
>>>>>       1. It is far easier to perform releases of the individual
>>>>> components.
>>>>>       2. It is much easier to accept plugin contributions.
>>>>> The downsides are:
>>>>>       1. A page like https://maven.apache.org/plugins/ <
>>>>> https://maven.apache.org/plugins/> is needed to keep track of the
>> plugin
>>>>> versions.
>>>>>       2. It could make sense to have log4j-parent and log4j-bom
>>>>> projects. The first to help keep the builds similar and the second to
>>>> help
>>>>> customers pick up the latest versions.
>>>>> 
>>>>> Ralph
>>>>> 
>>>>> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
>>>>> repos/asf>
>>>>> 
>>>>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
>>>>> wrote:
>>>>>> 
>>>>>> Hi All,
>>>>>> 
>>>>>> I am writing this to in an effort to understand how we can manage and
>>>>> grow
>>>>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense but
>>>> more
>>>>>> in the maturing sense.
>>>>>> 
>>>>>> I am prompted to write this by Ralph's comment that log4j-core and the
>>>>> main
>>>>>> repo is too big and that releases take too long make, partially
>>>> prompted
>>>>> by
>>>>>> my addition of a new module called log4j-jdbc-dbcp2 which currently
>>>>>> contains one small class with a dependency on Apache Commons DBCP 2.
>>>>>> 
>>>>>> I would like to express my gratitude at Ralph's efforts to revitalized
>>>>>> Log4j and performing most release management duties. Thank you Ralph!
>>>>>> 
>>>>>> I can see two main orthogonal issues:
>>>>>> - The size of the git repo.
>>>>>> - The size of the log4j-core module.
>>>>>> 
>>>>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
>>>>>> another 'plugin' repo.
>>>>>> 
>>>>>> I really dislike this idea:
>>>>>> - The plugin repo has never been released. Not that one releases a
>> repo
>>>>> but
>>>>>> you get the point.
>>>>>> - How do you keep things in sync between repos and code when we have
>> no
>>>>>> official 'core' SPI.
>>>>>> 
>>>>>> For my money, we should keep _everything_ in one repo. Good enough for
>>>>>> Google, so good enough for me. What we release out of that repo is a
>>>>>> different story and what I would like to discuss next.
>>>>>> 
>>>>>> This is not the same as Maven plugins IMO but the case could be made
>>>> for
>>>>> it
>>>>>> I suppose where a lot/most plugins live in their own repos. It is not
>>>> the
>>>>>> same as with Maven IMO because our plugins rely on log4j-core and it's
>>>>>> guts, for which we only make loose compatibility guarantees -- as
>>>> opposed
>>>>>> to log4j-core where we are strict(er). Maven OTOH, has a API for
>> plugin
>>>>>> auteurs.
>>>>>> 
>>>>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we
>>>>> have
>>>>>> no 'official' core SPI, so splitting it off into a separate repo would
>>>>>> greatly increase the risk of it falling out of sync. It is just so
>> much
>>>>>> more easier to maintain when it is all in one repo.
>>>>>> 
>>>>>> My proposal is to:
>>>>>> 
>>>>>> - Put everything back into one repo (Chainsaw too?)
>>>>>> - Define a core SPI for plugin writers where we make some statement
>>>> about
>>>>>> BC, more than the casual 'we try not break stuff.'
>>>>>> - Defining what Log4j project 'components' we have and release based
>> on
>>>>>> that. For example, today, all of the main Log4j repo is one component
>>>>> with
>>>>>> many modules. Chainsaw would be another component of the Log4j
>> project.
>>>>>> Maybe we need to redefine components: API, File, JDBC and so on. A
>>>>>> component can have one of more module
>>>>>> - Got for there.
>>>>>> 
>>>>>> Thoughts? Help me flush this out?
>>>>>> 
>>>>>> Thank you for reading!
>>>>>> Gary
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Matt Sicker <bo...@gmail.com>
>> 
>> 
>> 



Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
I'm curious: What is threshold where it would be OK to add stuff to the
main repo?

Gary

On Mon, Jan 22, 2018 at 12:48 PM, Ralph Goers <ra...@dslextreme.com>
wrote:

> I don’t see why this work would require 3.0 as we aren’t planning on
> breaking any contracts to do this.
>
> As I’ve said, I am not tied to each plugin having its own repo. I am OK
> with these options; a) each plugin has its own repo and site and is
> released independently, b) we use the plugins repo and move all the more
> lightly used components there. It would have its own site, or c) we group
> plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
> similar) with each having its own site.
>
> Obviously I do not consider continuing to add new plugins to the main
> build as one of the options.
>
> Ralph
>
> > On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:
> >
> > This whole conversation just reminds me of all the times in the past I
> > suggested full modularization of the core API/SPI. I think such an effort
> > would make sense in a 3.0 release.
> >
> > In 2.x, perhaps what we can do is define a stable plugin API based on
> what
> > we already have. Since we're still supporting Java 7, I'd suggest we make
> > abstract classes be the stable part of the API since we don't have
> default
> > methods on interfaces here.
> >
> > And yes, the GitBox integration thing sounds neat, particularly for code
> > review, though that's a separate topic entirely.
> >
> > On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> >> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <
> ralph.goers@dslextreme.com>
> >> wrote:
> >>
> >>> I am very much against the idea of a single repo. Yes, you can have
> >>> multiple projects in the repo but I am not sure how that can be sanely
> >>> released. I much prefer the model that Maven has taken. They are now
> >> using
> >>> gitbox [1] which seems to allow GitHub to be the primary repo.
> >>
> >>
> >> GitHub is a distracting tangent here IMO. I really like GitHub's code
> >> commenting feature, but it should not matter if we use Apache's Git
> hosted
> >> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
> >> conversation... ;-)
> >>
> >> Gary
> >>
> >>
> >>> Every Maven plugin is individually released. Scroll down the link below
> >> to
> >>> the Maven section and you can see all the plugin repos.
> >>>
> >>> The upside to this is that it are:
> >>>        1. It is far easier to perform releases of the individual
> >>> components.
> >>>        2. It is much easier to accept plugin contributions.
> >>> The downsides are:
> >>>        1. A page like https://maven.apache.org/plugins/ <
> >>> https://maven.apache.org/plugins/> is needed to keep track of the
> plugin
> >>> versions.
> >>>        2. It could make sense to have log4j-parent and log4j-bom
> >>> projects. The first to help keep the builds similar and the second to
> >> help
> >>> customers pick up the latest versions.
> >>>
> >>> Ralph
> >>>
> >>> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
> >>> repos/asf>
> >>>
> >>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
> >>> wrote:
> >>>>
> >>>> Hi All,
> >>>>
> >>>> I am writing this to in an effort to understand how we can manage and
> >>> grow
> >>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense but
> >> more
> >>>> in the maturing sense.
> >>>>
> >>>> I am prompted to write this by Ralph's comment that log4j-core and the
> >>> main
> >>>> repo is too big and that releases take too long make, partially
> >> prompted
> >>> by
> >>>> my addition of a new module called log4j-jdbc-dbcp2 which currently
> >>>> contains one small class with a dependency on Apache Commons DBCP 2.
> >>>>
> >>>> I would like to express my gratitude at Ralph's efforts to revitalized
> >>>> Log4j and performing most release management duties. Thank you Ralph!
> >>>>
> >>>> I can see two main orthogonal issues:
> >>>> - The size of the git repo.
> >>>> - The size of the log4j-core module.
> >>>>
> >>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
> >>>> another 'plugin' repo.
> >>>>
> >>>> I really dislike this idea:
> >>>> - The plugin repo has never been released. Not that one releases a
> repo
> >>> but
> >>>> you get the point.
> >>>> - How do you keep things in sync between repos and code when we have
> no
> >>>> official 'core' SPI.
> >>>>
> >>>> For my money, we should keep _everything_ in one repo. Good enough for
> >>>> Google, so good enough for me. What we release out of that repo is a
> >>>> different story and what I would like to discuss next.
> >>>>
> >>>> This is not the same as Maven plugins IMO but the case could be made
> >> for
> >>> it
> >>>> I suppose where a lot/most plugins live in their own repos. It is not
> >> the
> >>>> same as with Maven IMO because our plugins rely on log4j-core and it's
> >>>> guts, for which we only make loose compatibility guarantees -- as
> >> opposed
> >>>> to log4j-core where we are strict(er). Maven OTOH, has a API for
> plugin
> >>>> auteurs.
> >>>>
> >>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we
> >>> have
> >>>> no 'official' core SPI, so splitting it off into a separate repo would
> >>>> greatly increase the risk of it falling out of sync. It is just so
> much
> >>>> more easier to maintain when it is all in one repo.
> >>>>
> >>>> My proposal is to:
> >>>>
> >>>> - Put everything back into one repo (Chainsaw too?)
> >>>> - Define a core SPI for plugin writers where we make some statement
> >> about
> >>>> BC, more than the casual 'we try not break stuff.'
> >>>> - Defining what Log4j project 'components' we have and release based
> on
> >>>> that. For example, today, all of the main Log4j repo is one component
> >>> with
> >>>> many modules. Chainsaw would be another component of the Log4j
> project.
> >>>> Maybe we need to redefine components: API, File, JDBC and so on. A
> >>>> component can have one of more module
> >>>> - Got for there.
> >>>>
> >>>> Thoughts? Help me flush this out?
> >>>>
> >>>> Thank you for reading!
> >>>> Gary
> >>>
> >>>
> >>
> >
> >
> >
> > --
> > Matt Sicker <bo...@gmail.com>
>
>
>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 22, 2018 at 2:25 PM, Gary Gregory <ga...@gmail.com>
wrote:

>
>
> On Mon, Jan 22, 2018 at 2:21 PM, Gary Gregory <ga...@gmail.com>
> wrote:
>
>>
>>
>> On Mon, Jan 22, 2018 at 12:58 PM, Matt Sicker <bo...@gmail.com> wrote:
>>
>>> On 22 January 2018 at 13:48, Ralph Goers <ra...@dslextreme.com>
>>> wrote:
>>>
>>> > I don’t see why this work would require 3.0 as we aren’t planning on
>>> > breaking any contracts to do this.
>>> >
>>>
>>> I'm referring to something more in line with past proposals of having a
>>> sort of log4j-core-api or similar which is essentially just the different
>>> plugin APIs along with the plugin loader API, and log4j-core would
>>> contain
>>> the most commonly used plugins (file, console, meta appenders, filters,
>>> layouts). This is an orthogonal idea to what we're trying to solve,
>>> though.
>>>
>>
>> (bit send by accident)
>
> I'm not only worried about that but also ending up with some brittle
> dependency chain like:
>
> log4j-jdbc-dbcp2
>   depends on log4j-jdbc (does not exist yet)
>      depends on log4j-core
>          depends on log4j-api.
>
> Today this works out of the box. Would it become, for example:
>
> log4j-2.11.0-jdbc-dbcp2-1.0
>   depends on log4j-2.11.0-jdbc-1.0 (does not exist yet)
>      depends on log4j-core-2.11.0
>          depends on log4j-api-2.11.0
>
> or would I not bother coding the required Log4j version to get:
>
> log4j-jdbc-dbcp2-1.0
>   depends on log4j-jdbc-1.0 (does not exist yet)
>      depends on log4j-core-2.11.0
>          depends on log4j-api-2.11.0
>

Or maybe I use a matching version:

log4j-jdbc-dbcp2-2.11.0
>   depends on log4j-jdbc-2.11.0 (does not exist yet)
>      depends on log4j-core-2.11.0
>          depends on log4j-api-2.11.0
>
>
?

Gary


>
> ?
>

> Gary
>
>
>
>>
>>>
>>> > As I’ve said, I am not tied to each plugin having its own repo. I am OK
>>> > with these options; a) each plugin has its own repo and site and is
>>> > released independently, b) we use the plugins repo and move all the
>>> more
>>> > lightly used components there. It would have its own site, or c) we
>>> group
>>> > plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
>>> > similar) with each having its own site.
>>> >
>>>
>>> If we have separate repos, we'll want a better semantic versioning scheme
>>> here. I'd imagine we'd need something more enforceable at compile time
>>> and
>>> run time to verify that the version of log4j-core is compatible with the
>>> plugins in question.
>>>
>>> Also, a question from earlier asked how we can verify the plugins
>>> continue
>>> to work against the latest log4j-core. We can set this up in Jenkins to
>>> autobuild when upstream dependencies are built. So for example, we can
>>> tie
>>> the sister repos to the main one via snapshots and make Jenkins perform
>>> the
>>> CI on it.
>>>
>>> --
>>> Matt Sicker <bo...@gmail.com>
>>>
>>
>>
>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 22, 2018 at 2:21 PM, Gary Gregory <ga...@gmail.com>
wrote:

>
>
> On Mon, Jan 22, 2018 at 12:58 PM, Matt Sicker <bo...@gmail.com> wrote:
>
>> On 22 January 2018 at 13:48, Ralph Goers <ra...@dslextreme.com>
>> wrote:
>>
>> > I don’t see why this work would require 3.0 as we aren’t planning on
>> > breaking any contracts to do this.
>> >
>>
>> I'm referring to something more in line with past proposals of having a
>> sort of log4j-core-api or similar which is essentially just the different
>> plugin APIs along with the plugin loader API, and log4j-core would contain
>> the most commonly used plugins (file, console, meta appenders, filters,
>> layouts). This is an orthogonal idea to what we're trying to solve,
>> though.
>>
>
> (bit send by accident)

I'm not only worried about that but also ending up with some brittle
dependency chain like:

log4j-jdbc-dbcp2
  depends on log4j-jdbc (does not exist yet)
     depends on log4j-core
         depends on log4j-api.

Today this works out of the box. Would it become, for example:

log4j-2.11.0-jdbc-dbcp2-1.0
  depends on log4j-2.11.0-jdbc-1.0 (does not exist yet)
     depends on log4j-core-2.11.0
         depends on log4j-api-2.11.0

or would I not bother coding the required Log4j version to get:

log4j-jdbc-dbcp2-1.0
  depends on log4j-jdbc-1.0 (does not exist yet)
     depends on log4j-core-2.11.0
         depends on log4j-api-2.11.0

?

Gary



>
>>
>> > As I’ve said, I am not tied to each plugin having its own repo. I am OK
>> > with these options; a) each plugin has its own repo and site and is
>> > released independently, b) we use the plugins repo and move all the more
>> > lightly used components there. It would have its own site, or c) we
>> group
>> > plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
>> > similar) with each having its own site.
>> >
>>
>> If we have separate repos, we'll want a better semantic versioning scheme
>> here. I'd imagine we'd need something more enforceable at compile time and
>> run time to verify that the version of log4j-core is compatible with the
>> plugins in question.
>>
>> Also, a question from earlier asked how we can verify the plugins continue
>> to work against the latest log4j-core. We can set this up in Jenkins to
>> autobuild when upstream dependencies are built. So for example, we can tie
>> the sister repos to the main one via snapshots and make Jenkins perform
>> the
>> CI on it.
>>
>> --
>> Matt Sicker <bo...@gmail.com>
>>
>
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.

> On Jan 22, 2018, at 2:21 PM, Gary Gregory <ga...@gmail.com> wrote:
> 
> On Mon, Jan 22, 2018 at 12:58 PM, Matt Sicker <bo...@gmail.com> wrote:
> 
>> On 22 January 2018 at 13:48, Ralph Goers <ra...@dslextreme.com>
>> wrote:
>> 
>>> I don’t see why this work would require 3.0 as we aren’t planning on
>>> breaking any contracts to do this.
>>> 
>> 
>> I'm referring to something more in line with past proposals of having a
>> sort of log4j-core-api or similar which is essentially just the different
>> plugin APIs along with the plugin loader API, and log4j-core would contain
>> the most commonly used plugins (file, console, meta appenders, filters,
>> layouts). This is an orthogonal idea to what we're trying to solve, though.
>> 
> 
> I'm not only worried about that but also ending up with some brittle
> dependency chain like:
> 
> log4j-jdbc-dbcp2
> depends on log4j-jdbc (does not exist yet) depends on log4j-core depends on
> log4j-api. Today this works out of the box.

That dependency chain exists regardless of whether the module is part of the main build or part of some other build. 

Ralph

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 22, 2018 at 12:58 PM, Matt Sicker <bo...@gmail.com> wrote:

> On 22 January 2018 at 13:48, Ralph Goers <ra...@dslextreme.com>
> wrote:
>
> > I don’t see why this work would require 3.0 as we aren’t planning on
> > breaking any contracts to do this.
> >
>
> I'm referring to something more in line with past proposals of having a
> sort of log4j-core-api or similar which is essentially just the different
> plugin APIs along with the plugin loader API, and log4j-core would contain
> the most commonly used plugins (file, console, meta appenders, filters,
> layouts). This is an orthogonal idea to what we're trying to solve, though.
>

I'm not only worried about that but also ending up with some brittle
dependency chain like:

log4j-jdbc-dbcp2
depends on log4j-jdbc (does not exist yet) depends on log4j-core depends on
log4j-api. Today this works out of the box.


>
> > As I’ve said, I am not tied to each plugin having its own repo. I am OK
> > with these options; a) each plugin has its own repo and site and is
> > released independently, b) we use the plugins repo and move all the more
> > lightly used components there. It would have its own site, or c) we group
> > plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
> > similar) with each having its own site.
> >
>
> If we have separate repos, we'll want a better semantic versioning scheme
> here. I'd imagine we'd need something more enforceable at compile time and
> run time to verify that the version of log4j-core is compatible with the
> plugins in question.
>
> Also, a question from earlier asked how we can verify the plugins continue
> to work against the latest log4j-core. We can set this up in Jenkins to
> autobuild when upstream dependencies are built. So for example, we can tie
> the sister repos to the main one via snapshots and make Jenkins perform the
> CI on it.
>
> --
> Matt Sicker <bo...@gmail.com>
>

Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
On 22 January 2018 at 13:48, Ralph Goers <ra...@dslextreme.com> wrote:

> I don’t see why this work would require 3.0 as we aren’t planning on
> breaking any contracts to do this.
>

I'm referring to something more in line with past proposals of having a
sort of log4j-core-api or similar which is essentially just the different
plugin APIs along with the plugin loader API, and log4j-core would contain
the most commonly used plugins (file, console, meta appenders, filters,
layouts). This is an orthogonal idea to what we're trying to solve, though.


> As I’ve said, I am not tied to each plugin having its own repo. I am OK
> with these options; a) each plugin has its own repo and site and is
> released independently, b) we use the plugins repo and move all the more
> lightly used components there. It would have its own site, or c) we group
> plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or
> similar) with each having its own site.
>

If we have separate repos, we'll want a better semantic versioning scheme
here. I'd imagine we'd need something more enforceable at compile time and
run time to verify that the version of log4j-core is compatible with the
plugins in question.

Also, a question from earlier asked how we can verify the plugins continue
to work against the latest log4j-core. We can set this up in Jenkins to
autobuild when upstream dependencies are built. So for example, we can tie
the sister repos to the main one via snapshots and make Jenkins perform the
CI on it.

-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
I don’t see why this work would require 3.0 as we aren’t planning on breaking any contracts to do this.

As I’ve said, I am not tied to each plugin having its own repo. I am OK with these options; a) each plugin has its own repo and site and is released independently, b) we use the plugins repo and move all the more lightly used components there. It would have its own site, or c) we group plugins by how they are related (RDBMS, NoSQL, “Transport” (Flume or similar) with each having its own site.

Obviously I do not consider continuing to add new plugins to the main build as one of the options.

Ralph

> On Jan 22, 2018, at 12:36 PM, Matt Sicker <bo...@gmail.com> wrote:
> 
> This whole conversation just reminds me of all the times in the past I
> suggested full modularization of the core API/SPI. I think such an effort
> would make sense in a 3.0 release.
> 
> In 2.x, perhaps what we can do is define a stable plugin API based on what
> we already have. Since we're still supporting Java 7, I'd suggest we make
> abstract classes be the stable part of the API since we don't have default
> methods on interfaces here.
> 
> And yes, the GitBox integration thing sounds neat, particularly for code
> review, though that's a separate topic entirely.
> 
> On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com> wrote:
> 
>> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <ra...@dslextreme.com>
>> wrote:
>> 
>>> I am very much against the idea of a single repo. Yes, you can have
>>> multiple projects in the repo but I am not sure how that can be sanely
>>> released. I much prefer the model that Maven has taken. They are now
>> using
>>> gitbox [1] which seems to allow GitHub to be the primary repo.
>> 
>> 
>> GitHub is a distracting tangent here IMO. I really like GitHub's code
>> commenting feature, but it should not matter if we use Apache's Git hosted
>> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
>> conversation... ;-)
>> 
>> Gary
>> 
>> 
>>> Every Maven plugin is individually released. Scroll down the link below
>> to
>>> the Maven section and you can see all the plugin repos.
>>> 
>>> The upside to this is that it are:
>>>        1. It is far easier to perform releases of the individual
>>> components.
>>>        2. It is much easier to accept plugin contributions.
>>> The downsides are:
>>>        1. A page like https://maven.apache.org/plugins/ <
>>> https://maven.apache.org/plugins/> is needed to keep track of the plugin
>>> versions.
>>>        2. It could make sense to have log4j-parent and log4j-bom
>>> projects. The first to help keep the builds similar and the second to
>> help
>>> customers pick up the latest versions.
>>> 
>>> Ralph
>>> 
>>> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
>>> repos/asf>
>>> 
>>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
>>> wrote:
>>>> 
>>>> Hi All,
>>>> 
>>>> I am writing this to in an effort to understand how we can manage and
>>> grow
>>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense but
>> more
>>>> in the maturing sense.
>>>> 
>>>> I am prompted to write this by Ralph's comment that log4j-core and the
>>> main
>>>> repo is too big and that releases take too long make, partially
>> prompted
>>> by
>>>> my addition of a new module called log4j-jdbc-dbcp2 which currently
>>>> contains one small class with a dependency on Apache Commons DBCP 2.
>>>> 
>>>> I would like to express my gratitude at Ralph's efforts to revitalized
>>>> Log4j and performing most release management duties. Thank you Ralph!
>>>> 
>>>> I can see two main orthogonal issues:
>>>> - The size of the git repo.
>>>> - The size of the log4j-core module.
>>>> 
>>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
>>>> another 'plugin' repo.
>>>> 
>>>> I really dislike this idea:
>>>> - The plugin repo has never been released. Not that one releases a repo
>>> but
>>>> you get the point.
>>>> - How do you keep things in sync between repos and code when we have no
>>>> official 'core' SPI.
>>>> 
>>>> For my money, we should keep _everything_ in one repo. Good enough for
>>>> Google, so good enough for me. What we release out of that repo is a
>>>> different story and what I would like to discuss next.
>>>> 
>>>> This is not the same as Maven plugins IMO but the case could be made
>> for
>>> it
>>>> I suppose where a lot/most plugins live in their own repos. It is not
>> the
>>>> same as with Maven IMO because our plugins rely on log4j-core and it's
>>>> guts, for which we only make loose compatibility guarantees -- as
>> opposed
>>>> to log4j-core where we are strict(er). Maven OTOH, has a API for plugin
>>>> auteurs.
>>>> 
>>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we
>>> have
>>>> no 'official' core SPI, so splitting it off into a separate repo would
>>>> greatly increase the risk of it falling out of sync. It is just so much
>>>> more easier to maintain when it is all in one repo.
>>>> 
>>>> My proposal is to:
>>>> 
>>>> - Put everything back into one repo (Chainsaw too?)
>>>> - Define a core SPI for plugin writers where we make some statement
>> about
>>>> BC, more than the casual 'we try not break stuff.'
>>>> - Defining what Log4j project 'components' we have and release based on
>>>> that. For example, today, all of the main Log4j repo is one component
>>> with
>>>> many modules. Chainsaw would be another component of the Log4j project.
>>>> Maybe we need to redefine components: API, File, JDBC and so on. A
>>>> component can have one of more module
>>>> - Got for there.
>>>> 
>>>> Thoughts? Help me flush this out?
>>>> 
>>>> Thank you for reading!
>>>> Gary
>>> 
>>> 
>> 
> 
> 
> 
> -- 
> Matt Sicker <bo...@gmail.com>



Re: [log4j] The shape of Log4j

Posted by Matt Sicker <bo...@gmail.com>.
This whole conversation just reminds me of all the times in the past I
suggested full modularization of the core API/SPI. I think such an effort
would make sense in a 3.0 release.

In 2.x, perhaps what we can do is define a stable plugin API based on what
we already have. Since we're still supporting Java 7, I'd suggest we make
abstract classes be the stable part of the API since we don't have default
methods on interfaces here.

And yes, the GitBox integration thing sounds neat, particularly for code
review, though that's a separate topic entirely.

On 22 January 2018 at 00:37, Gary Gregory <ga...@gmail.com> wrote:

> On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <ra...@dslextreme.com>
> wrote:
>
> > I am very much against the idea of a single repo. Yes, you can have
> > multiple projects in the repo but I am not sure how that can be sanely
> > released. I much prefer the model that Maven has taken. They are now
> using
> > gitbox [1] which seems to allow GitHub to be the primary repo.
>
>
> GitHub is a distracting tangent here IMO. I really like GitHub's code
> commenting feature, but it should not matter if we use Apache's Git hosted
> repos or GitHub's. So I'm not sure why we are mixing GitHub in the
> conversation... ;-)
>
> Gary
>
>
> > Every Maven plugin is individually released. Scroll down the link below
> to
> > the Maven section and you can see all the plugin repos.
> >
> > The upside to this is that it are:
> >         1. It is far easier to perform releases of the individual
> > components.
> >         2. It is much easier to accept plugin contributions.
> > The downsides are:
> >         1. A page like https://maven.apache.org/plugins/ <
> > https://maven.apache.org/plugins/> is needed to keep track of the plugin
> > versions.
> >         2. It could make sense to have log4j-parent and log4j-bom
> > projects. The first to help keep the builds similar and the second to
> help
> > customers pick up the latest versions.
> >
> > Ralph
> >
> > [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
> > repos/asf>
> >
> > > On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
> > wrote:
> > >
> > > Hi All,
> > >
> > > I am writing this to in an effort to understand how we can manage and
> > grow
> > > Log4j. I use the term 'grow' not in the 'bigger is better' sense but
> more
> > > in the maturing sense.
> > >
> > > I am prompted to write this by Ralph's comment that log4j-core and the
> > main
> > > repo is too big and that releases take too long make, partially
> prompted
> > by
> > > my addition of a new module called log4j-jdbc-dbcp2 which currently
> > > contains one small class with a dependency on Apache Commons DBCP 2.
> > >
> > > I would like to express my gratitude at Ralph's efforts to revitalized
> > > Log4j and performing most release management duties. Thank you Ralph!
> > >
> > > I can see two main orthogonal issues:
> > > - The size of the git repo.
> > > - The size of the log4j-core module.
> > >
> > > Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
> > > another 'plugin' repo.
> > >
> > > I really dislike this idea:
> > > - The plugin repo has never been released. Not that one releases a repo
> > but
> > > you get the point.
> > > - How do you keep things in sync between repos and code when we have no
> > > official 'core' SPI.
> > >
> > > For my money, we should keep _everything_ in one repo. Good enough for
> > > Google, so good enough for me. What we release out of that repo is a
> > > different story and what I would like to discuss next.
> > >
> > > This is not the same as Maven plugins IMO but the case could be made
> for
> > it
> > > I suppose where a lot/most plugins live in their own repos. It is not
> the
> > > same as with Maven IMO because our plugins rely on log4j-core and it's
> > > guts, for which we only make loose compatibility guarantees -- as
> opposed
> > > to log4j-core where we are strict(er). Maven OTOH, has a API for plugin
> > > auteurs.
> > >
> > > For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we
> > have
> > > no 'official' core SPI, so splitting it off into a separate repo would
> > > greatly increase the risk of it falling out of sync. It is just so much
> > > more easier to maintain when it is all in one repo.
> > >
> > > My proposal is to:
> > >
> > > - Put everything back into one repo (Chainsaw too?)
> > > - Define a core SPI for plugin writers where we make some statement
> about
> > > BC, more than the casual 'we try not break stuff.'
> > > - Defining what Log4j project 'components' we have and release based on
> > > that. For example, today, all of the main Log4j repo is one component
> > with
> > > many modules. Chainsaw would be another component of the Log4j project.
> > > Maybe we need to redefine components: API, File, JDBC and so on. A
> > > component can have one of more module
> > > - Got for there.
> > >
> > > Thoughts? Help me flush this out?
> > >
> > > Thank you for reading!
> > > Gary
> >
> >
>



-- 
Matt Sicker <bo...@gmail.com>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Sun, Jan 21, 2018 at 4:14 PM, Ralph Goers <ra...@dslextreme.com>
wrote:

> I am very much against the idea of a single repo. Yes, you can have
> multiple projects in the repo but I am not sure how that can be sanely
> released. I much prefer the model that Maven has taken. They are now using
> gitbox [1] which seems to allow GitHub to be the primary repo.


GitHub is a distracting tangent here IMO. I really like GitHub's code
commenting feature, but it should not matter if we use Apache's Git hosted
repos or GitHub's. So I'm not sure why we are mixing GitHub in the
conversation... ;-)

Gary


> Every Maven plugin is individually released. Scroll down the link below to
> the Maven section and you can see all the plugin repos.
>
> The upside to this is that it are:
>         1. It is far easier to perform releases of the individual
> components.
>         2. It is much easier to accept plugin contributions.
> The downsides are:
>         1. A page like https://maven.apache.org/plugins/ <
> https://maven.apache.org/plugins/> is needed to keep track of the plugin
> versions.
>         2. It could make sense to have log4j-parent and log4j-bom
> projects. The first to help keep the builds similar and the second to help
> customers pick up the latest versions.
>
> Ralph
>
> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
> repos/asf>
>
> > On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > Hi All,
> >
> > I am writing this to in an effort to understand how we can manage and
> grow
> > Log4j. I use the term 'grow' not in the 'bigger is better' sense but more
> > in the maturing sense.
> >
> > I am prompted to write this by Ralph's comment that log4j-core and the
> main
> > repo is too big and that releases take too long make, partially prompted
> by
> > my addition of a new module called log4j-jdbc-dbcp2 which currently
> > contains one small class with a dependency on Apache Commons DBCP 2.
> >
> > I would like to express my gratitude at Ralph's efforts to revitalized
> > Log4j and performing most release management duties. Thank you Ralph!
> >
> > I can see two main orthogonal issues:
> > - The size of the git repo.
> > - The size of the log4j-core module.
> >
> > Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
> > another 'plugin' repo.
> >
> > I really dislike this idea:
> > - The plugin repo has never been released. Not that one releases a repo
> but
> > you get the point.
> > - How do you keep things in sync between repos and code when we have no
> > official 'core' SPI.
> >
> > For my money, we should keep _everything_ in one repo. Good enough for
> > Google, so good enough for me. What we release out of that repo is a
> > different story and what I would like to discuss next.
> >
> > This is not the same as Maven plugins IMO but the case could be made for
> it
> > I suppose where a lot/most plugins live in their own repos. It is not the
> > same as with Maven IMO because our plugins rely on log4j-core and it's
> > guts, for which we only make loose compatibility guarantees -- as opposed
> > to log4j-core where we are strict(er). Maven OTOH, has a API for plugin
> > auteurs.
> >
> > For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we
> have
> > no 'official' core SPI, so splitting it off into a separate repo would
> > greatly increase the risk of it falling out of sync. It is just so much
> > more easier to maintain when it is all in one repo.
> >
> > My proposal is to:
> >
> > - Put everything back into one repo (Chainsaw too?)
> > - Define a core SPI for plugin writers where we make some statement about
> > BC, more than the casual 'we try not break stuff.'
> > - Defining what Log4j project 'components' we have and release based on
> > that. For example, today, all of the main Log4j repo is one component
> with
> > many modules. Chainsaw would be another component of the Log4j project.
> > Maybe we need to redefine components: API, File, JDBC and so on. A
> > component can have one of more module
> > - Got for there.
> >
> > Thoughts? Help me flush this out?
> >
> > Thank you for reading!
> > Gary
>
>

Re: [log4j] The shape of Log4j

Posted by Scott Deboy <sc...@gmail.com>.
I recall the log4j logging-parent nightmare of a number of years back.
Please let's not re-visit that.

https://github.com/apache/logging-parent/blob/master/pom.xml

Scott

On Jan 21, 2018 4:51 PM, "Remko Popma" <re...@gmail.com> wrote:

> Log4j2 is not like Maven. Maven (I assume) has a plugin API. Log4j2
> modules depend on the internals of log4j-core.
>
> I agree with Gary that not being able to verify that a core change doesn’t
> break any modules when they live outside the main project is a valid
> concern.
>
> Why are we talking about modules and repos when the issue is that it takes
> too long to do a release? Maven reports most modules as taking a handful of
> seconds to build.
>
> I keep thinking:
>
> 1. We’re running the tests twice during a release. Let’s change this.
> Logically, once should be enough. What prevents us from doing this?
>
> 2. Core tests take unnecessarily long. By running some tests in parallel
> and some other tests without forking I have hope we can run all core tests
> in ~15 minutes.
>
> Won’t those two changes be much more impactful in reducing the release
> time than any amount of module or repo reshuffling?
>
> Remko
>
> (Shameless plug) Every java main() method deserves http://picocli.info
>
> > On Jan 22, 2018, at 8:14, Ralph Goers <ra...@dslextreme.com>
> wrote:
> >
> > I am very much against the idea of a single repo. Yes, you can have
> multiple projects in the repo but I am not sure how that can be sanely
> released. I much prefer the model that Maven has taken. They are now using
> gitbox [1] which seems to allow GitHub to be the primary repo. Every Maven
> plugin is individually released. Scroll down the link below to the Maven
> section and you can see all the plugin repos.
> >
> > The upside to this is that it are:
> >    1. It is far easier to perform releases of the individual components.
> >    2. It is much easier to accept plugin contributions.
> > The downsides are:
> >    1. A page like https://maven.apache.org/plugins/ <
> https://maven.apache.org/plugins/> is needed to keep track of the plugin
> versions.
> >    2. It could make sense to have log4j-parent and log4j-bom projects.
> The first to help keep the builds similar and the second to help customers
> pick up the latest versions.
> >
> > Ralph
> >
> > [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
> repos/asf>
> >
> >> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
> wrote:
> >>
> >> Hi All,
> >>
> >> I am writing this to in an effort to understand how we can manage and
> grow
> >> Log4j. I use the term 'grow' not in the 'bigger is better' sense but
> more
> >> in the maturing sense.
> >>
> >> I am prompted to write this by Ralph's comment that log4j-core and the
> main
> >> repo is too big and that releases take too long make, partially
> prompted by
> >> my addition of a new module called log4j-jdbc-dbcp2 which currently
> >> contains one small class with a dependency on Apache Commons DBCP 2.
> >>
> >> I would like to express my gratitude at Ralph's efforts to revitalized
> >> Log4j and performing most release management duties. Thank you Ralph!
> >>
> >> I can see two main orthogonal issues:
> >> - The size of the git repo.
> >> - The size of the log4j-core module.
> >>
> >> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
> >> another 'plugin' repo.
> >>
> >> I really dislike this idea:
> >> - The plugin repo has never been released. Not that one releases a repo
> but
> >> you get the point.
> >> - How do you keep things in sync between repos and code when we have no
> >> official 'core' SPI.
> >>
> >> For my money, we should keep _everything_ in one repo. Good enough for
> >> Google, so good enough for me. What we release out of that repo is a
> >> different story and what I would like to discuss next.
> >>
> >> This is not the same as Maven plugins IMO but the case could be made
> for it
> >> I suppose where a lot/most plugins live in their own repos. It is not
> the
> >> same as with Maven IMO because our plugins rely on log4j-core and it's
> >> guts, for which we only make loose compatibility guarantees -- as
> opposed
> >> to log4j-core where we are strict(er). Maven OTOH, has a API for plugin
> >> auteurs.
> >>
> >> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we
> have
> >> no 'official' core SPI, so splitting it off into a separate repo would
> >> greatly increase the risk of it falling out of sync. It is just so much
> >> more easier to maintain when it is all in one repo.
> >>
> >> My proposal is to:
> >>
> >> - Put everything back into one repo (Chainsaw too?)
> >> - Define a core SPI for plugin writers where we make some statement
> about
> >> BC, more than the casual 'we try not break stuff.'
> >> - Defining what Log4j project 'components' we have and release based on
> >> that. For example, today, all of the main Log4j repo is one component
> with
> >> many modules. Chainsaw would be another component of the Log4j project.
> >> Maybe we need to redefine components: API, File, JDBC and so on. A
> >> component can have one of more module
> >> - Got for there.
> >>
> >> Thoughts? Help me flush this out?
> >>
> >> Thank you for reading!
> >> Gary
> >
>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Sun, Jan 21, 2018 at 5:51 PM, Remko Popma <re...@gmail.com> wrote:

> Log4j2 is not like Maven. Maven (I assume) has a plugin API. Log4j2
> modules depend on the internals of log4j-core.
>
> I agree with Gary that not being able to verify that a core change doesn’t
> break any modules when they live outside the main project is a valid
> concern.
>

This is a large and important concern for me. Whenever I take a serious
look at an appender for enterprise level use with my customers, I end up
having to add some features. This has happened for JMS, JDBC, Socket,
Console with JAnsi, and Core stuff (WatchManager for example.) Eventually
this will stop but then we will want to keep up with new feature as well.
So maybe it won't stop.

With the API, Core, and all these Appenders releasing in sync, everything
matches and all is well. If I need to add a feature to the JDBC Appender
that requires some other changes in Core? No problem. Even in different
modules, everything matches in a release.

I can't imagine the nightmare having to deal with updating a JMS plugin
3.0.x for Log4j 2.8.2 and a JDBC plugin 1.2.x for Log4j 2.11.0, and so on.
It does not seem we have the level of maturity Maven has, but Maven is on
their THIRD major release, so I would expect their SPI and object model to
be mature and plugin authoring to be easier.

Would we end up with module names like we have for Scala? Would I have a
log4j-2.11.0-jdbc-dbcp2-1.0.jar (the 1.0 release of a Log4j 2.11.0 JDBC
with Apache Commons DBCP 2 plugin)? Nasty :-(

I'm still looking for a plan or roadmap here.

Gary


> Why are we talking about modules and repos when the issue is that it takes
> too long to do a release? Maven reports most modules as taking a handful of
> seconds to build.
>
> I keep thinking:
>
> 1. We’re running the tests twice during a release. Let’s change this.
> Logically, once should be enough. What prevents us from doing this?
>
> 2. Core tests take unnecessarily long. By running some tests in parallel
> and some other tests without forking I have hope we can run all core tests
> in ~15 minutes.
>
> Won’t those two changes be much more impactful in reducing the release
> time than any amount of module or repo reshuffling?
>
> Remko
>
> (Shameless plug) Every java main() method deserves http://picocli.info
>
> > On Jan 22, 2018, at 8:14, Ralph Goers <ra...@dslextreme.com>
> wrote:
> >
> > I am very much against the idea of a single repo. Yes, you can have
> multiple projects in the repo but I am not sure how that can be sanely
> released. I much prefer the model that Maven has taken. They are now using
> gitbox [1] which seems to allow GitHub to be the primary repo. Every Maven
> plugin is individually released. Scroll down the link below to the Maven
> section and you can see all the plugin repos.
> >
> > The upside to this is that it are:
> >    1. It is far easier to perform releases of the individual components.
> >    2. It is much easier to accept plugin contributions.
> > The downsides are:
> >    1. A page like https://maven.apache.org/plugins/ <
> https://maven.apache.org/plugins/> is needed to keep track of the plugin
> versions.
> >    2. It could make sense to have log4j-parent and log4j-bom projects.
> The first to help keep the builds similar and the second to help customers
> pick up the latest versions.
> >
> > Ralph
> >
> > [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
> repos/asf>
> >
> >> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
> wrote:
> >>
> >> Hi All,
> >>
> >> I am writing this to in an effort to understand how we can manage and
> grow
> >> Log4j. I use the term 'grow' not in the 'bigger is better' sense but
> more
> >> in the maturing sense.
> >>
> >> I am prompted to write this by Ralph's comment that log4j-core and the
> main
> >> repo is too big and that releases take too long make, partially
> prompted by
> >> my addition of a new module called log4j-jdbc-dbcp2 which currently
> >> contains one small class with a dependency on Apache Commons DBCP 2.
> >>
> >> I would like to express my gratitude at Ralph's efforts to revitalized
> >> Log4j and performing most release management duties. Thank you Ralph!
> >>
> >> I can see two main orthogonal issues:
> >> - The size of the git repo.
> >> - The size of the log4j-core module.
> >>
> >> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
> >> another 'plugin' repo.
> >>
> >> I really dislike this idea:
> >> - The plugin repo has never been released. Not that one releases a repo
> but
> >> you get the point.
> >> - How do you keep things in sync between repos and code when we have no
> >> official 'core' SPI.
> >>
> >> For my money, we should keep _everything_ in one repo. Good enough for
> >> Google, so good enough for me. What we release out of that repo is a
> >> different story and what I would like to discuss next.
> >>
> >> This is not the same as Maven plugins IMO but the case could be made
> for it
> >> I suppose where a lot/most plugins live in their own repos. It is not
> the
> >> same as with Maven IMO because our plugins rely on log4j-core and it's
> >> guts, for which we only make loose compatibility guarantees -- as
> opposed
> >> to log4j-core where we are strict(er). Maven OTOH, has a API for plugin
> >> auteurs.
> >>
> >> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we
> have
> >> no 'official' core SPI, so splitting it off into a separate repo would
> >> greatly increase the risk of it falling out of sync. It is just so much
> >> more easier to maintain when it is all in one repo.
> >>
> >> My proposal is to:
> >>
> >> - Put everything back into one repo (Chainsaw too?)
> >> - Define a core SPI for plugin writers where we make some statement
> about
> >> BC, more than the casual 'we try not break stuff.'
> >> - Defining what Log4j project 'components' we have and release based on
> >> that. For example, today, all of the main Log4j repo is one component
> with
> >> many modules. Chainsaw would be another component of the Log4j project.
> >> Maybe we need to redefine components: API, File, JDBC and so on. A
> >> component can have one of more module
> >> - Got for there.
> >>
> >> Thoughts? Help me flush this out?
> >>
> >> Thank you for reading!
> >> Gary
> >
>

Re: [log4j] The shape of Log4j

Posted by Gary Gregory <ga...@gmail.com>.
On Sun, Jan 21, 2018 at 5:51 PM, Remko Popma <re...@gmail.com> wrote:

> Log4j2 is not like Maven. Maven (I assume) has a plugin API. Log4j2
> modules depend on the internals of log4j-core.
>
> I agree with Gary that not being able to verify that a core change doesn’t
> break any modules when they live outside the main project is a valid
> concern.
>
> Why are we talking about modules and repos when the issue is that it takes
> too long to do a release? Maven reports most modules as taking a handful of
> seconds to build.
>
> I keep thinking:
>
> 1. We’re running the tests twice during a release. Let’s change this.
> Logically, once should be enough. What prevents us from doing this?
>
> 2. Core tests take unnecessarily long. By running some tests in parallel
> and some other tests without forking I have hope we can run all core tests
> in ~15 minutes.
>

Speaking of long running tests, we have 117 calls to Thread.sleep() in our
tests. That's gotta slow things down. At work, I am very careful and advise
to use Thread.sleep() in tests only as a LAST RESORT. Instead we use all
sorts of clever synchronization using java.util.concurrent classes like
latches, some listeners, some JUnit test rules and whatnots. Some of this
trickery comes at the cost of changing some guts, but that also takes a lot
of timing randomness out of our builds.

Gary


> Won’t those two changes be much more impactful in reducing the release
> time than any amount of module or repo reshuffling?
>
> Remko
>
> (Shameless plug) Every java main() method deserves http://picocli.info
>
> > On Jan 22, 2018, at 8:14, Ralph Goers <ra...@dslextreme.com>
> wrote:
> >
> > I am very much against the idea of a single repo. Yes, you can have
> multiple projects in the repo but I am not sure how that can be sanely
> released. I much prefer the model that Maven has taken. They are now using
> gitbox [1] which seems to allow GitHub to be the primary repo. Every Maven
> plugin is individually released. Scroll down the link below to the Maven
> section and you can see all the plugin repos.
> >
> > The upside to this is that it are:
> >    1. It is far easier to perform releases of the individual components.
> >    2. It is much easier to accept plugin contributions.
> > The downsides are:
> >    1. A page like https://maven.apache.org/plugins/ <
> https://maven.apache.org/plugins/> is needed to keep track of the plugin
> versions.
> >    2. It could make sense to have log4j-parent and log4j-bom projects.
> The first to help keep the builds similar and the second to help customers
> pick up the latest versions.
> >
> > Ralph
> >
> > [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/
> repos/asf>
> >
> >> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com>
> wrote:
> >>
> >> Hi All,
> >>
> >> I am writing this to in an effort to understand how we can manage and
> grow
> >> Log4j. I use the term 'grow' not in the 'bigger is better' sense but
> more
> >> in the maturing sense.
> >>
> >> I am prompted to write this by Ralph's comment that log4j-core and the
> main
> >> repo is too big and that releases take too long make, partially
> prompted by
> >> my addition of a new module called log4j-jdbc-dbcp2 which currently
> >> contains one small class with a dependency on Apache Commons DBCP 2.
> >>
> >> I would like to express my gratitude at Ralph's efforts to revitalized
> >> Log4j and performing most release management duties. Thank you Ralph!
> >>
> >> I can see two main orthogonal issues:
> >> - The size of the git repo.
> >> - The size of the log4j-core module.
> >>
> >> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
> >> another 'plugin' repo.
> >>
> >> I really dislike this idea:
> >> - The plugin repo has never been released. Not that one releases a repo
> but
> >> you get the point.
> >> - How do you keep things in sync between repos and code when we have no
> >> official 'core' SPI.
> >>
> >> For my money, we should keep _everything_ in one repo. Good enough for
> >> Google, so good enough for me. What we release out of that repo is a
> >> different story and what I would like to discuss next.
> >>
> >> This is not the same as Maven plugins IMO but the case could be made
> for it
> >> I suppose where a lot/most plugins live in their own repos. It is not
> the
> >> same as with Maven IMO because our plugins rely on log4j-core and it's
> >> guts, for which we only make loose compatibility guarantees -- as
> opposed
> >> to log4j-core where we are strict(er). Maven OTOH, has a API for plugin
> >> auteurs.
> >>
> >> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we
> have
> >> no 'official' core SPI, so splitting it off into a separate repo would
> >> greatly increase the risk of it falling out of sync. It is just so much
> >> more easier to maintain when it is all in one repo.
> >>
> >> My proposal is to:
> >>
> >> - Put everything back into one repo (Chainsaw too?)
> >> - Define a core SPI for plugin writers where we make some statement
> about
> >> BC, more than the casual 'we try not break stuff.'
> >> - Defining what Log4j project 'components' we have and release based on
> >> that. For example, today, all of the main Log4j repo is one component
> with
> >> many modules. Chainsaw would be another component of the Log4j project.
> >> Maybe we need to redefine components: API, File, JDBC and so on. A
> >> component can have one of more module
> >> - Got for there.
> >>
> >> Thoughts? Help me flush this out?
> >>
> >> Thank you for reading!
> >> Gary
> >
>

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
FWIW, Some of this is borne out by looking at the stats at https://repository.apache.org/#central-stat <https://repository.apache.org/#central-stat>. The good news is that the downloads are 3 times what they were a year ago.  But you can see that the usage of these extra components is 1000 times smaller than log4j-core.

Ralph

> On Jan 21, 2018, at 9:35 PM, Ralph Goers <ra...@dslextreme.com> wrote:
> 
> Remko, 
> 
> I am all for speeding up the build. I have been complaining about this for well over a year and it keeps getting worse with every release. The compile of log4j-core takes less than 20 seconds on the machine provided by my employer. On the same machine the tests take 11 minutes. On this machine the whole build takes about 16 minutes. This is a bit frustrating as I perform a complete build at least once before I commit 95% of my changes.
> 
> My personal machine, which is where I run the release builds, is over 6 years old now and is considerably slower. On that machine a “normal” build currently takes . In addition, the release build does perform more work and runs through the build twice (although I am not sure it runs the tests twice anymore) so it is considerably longer. Also with every release we add new unit tests. It wasn’t too long ago that I was marveling that we had over 900 unit tests. We now have over 2,400. In addition, each release also requires building the web site, which can take a long time as well. 
> 
> While I suspect about 1/3 of the time of a “normal” build could be cut by executing some stuff in parallel that “fix” will only last for a short while as we keep adding components related to third party projects (Mongo, Cassandra, DBCP2, etc) that most likely don’t change much once they are done. I can’t remember the tag library module required changes. Many of them only change when the third party project changes (such as Mongo 2 vs Mongo 3). On that note I really don’t want to see Mongo 2 and Mongo 3 projects as part of the Log4j main build.
> 
> In addition to speeding up the build I would prefer to have the main log4j releases follow the 80/20 rule - they should contain only the stuff 80% of our users will use. I would venture a guess that at least 80% only use the File and Console appenders.
> 
> While we may not have as firm of a plugin API as Maven does, almost everything a user should need to create an Appender, Layout, Lookup or Filter should be as backward compatible as the Log4j API. If we can’t guarantee backward compatibility for those components we have a big problem as users are creating these kinds of components all the time. We cannot be breaking them. So from that standpoint we should have no problem moving those kinds of plugins out of the main build.
> 
> Ralph 
> 
> 
> 
>> On Jan 21, 2018, at 5:51 PM, Remko Popma <re...@gmail.com> wrote:
>> 
>> Log4j2 is not like Maven. Maven (I assume) has a plugin API. Log4j2 modules depend on the internals of log4j-core. 
>> 
>> I agree with Gary that not being able to verify that a core change doesn’t break any modules when they live outside the main project is a valid concern. 
>> 
>> Why are we talking about modules and repos when the issue is that it takes too long to do a release? Maven reports most modules as taking a handful of seconds to build. 
>> 
>> I keep thinking:
>> 
>> 1. We’re running the tests twice during a release. Let’s change this. Logically, once should be enough. What prevents us from doing this?
>> 
>> 2. Core tests take unnecessarily long. By running some tests in parallel and some other tests without forking I have hope we can run all core tests in ~15 minutes. 
>> 
>> Won’t those two changes be much more impactful in reducing the release time than any amount of module or repo reshuffling?
>> 
>> Remko 
>> 
>> (Shameless plug) Every java main() method deserves http://picocli.info
>> 
>>> On Jan 22, 2018, at 8:14, Ralph Goers <ra...@dslextreme.com> wrote:
>>> 
>>> I am very much against the idea of a single repo. Yes, you can have multiple projects in the repo but I am not sure how that can be sanely released. I much prefer the model that Maven has taken. They are now using gitbox [1] which seems to allow GitHub to be the primary repo. Every Maven plugin is individually released. Scroll down the link below to the Maven section and you can see all the plugin repos.
>>> 
>>> The upside to this is that it are: 
>>>  1. It is far easier to perform releases of the individual components. 
>>>  2. It is much easier to accept plugin contributions.
>>> The downsides are:
>>>  1. A page like https://maven.apache.org/plugins/ <https://maven.apache.org/plugins/> is needed to keep track of the plugin versions.
>>>  2. It could make sense to have log4j-parent and log4j-bom projects. The first to help keep the builds similar and the second to help customers pick up the latest versions. 
>>> 
>>> Ralph
>>> 
>>> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/repos/asf>
>>> 
>>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com> wrote:
>>>> 
>>>> Hi All,
>>>> 
>>>> I am writing this to in an effort to understand how we can manage and grow
>>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense but more
>>>> in the maturing sense.
>>>> 
>>>> I am prompted to write this by Ralph's comment that log4j-core and the main
>>>> repo is too big and that releases take too long make, partially prompted by
>>>> my addition of a new module called log4j-jdbc-dbcp2 which currently
>>>> contains one small class with a dependency on Apache Commons DBCP 2.
>>>> 
>>>> I would like to express my gratitude at Ralph's efforts to revitalized
>>>> Log4j and performing most release management duties. Thank you Ralph!
>>>> 
>>>> I can see two main orthogonal issues:
>>>> - The size of the git repo.
>>>> - The size of the log4j-core module.
>>>> 
>>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
>>>> another 'plugin' repo.
>>>> 
>>>> I really dislike this idea:
>>>> - The plugin repo has never been released. Not that one releases a repo but
>>>> you get the point.
>>>> - How do you keep things in sync between repos and code when we have no
>>>> official 'core' SPI.
>>>> 
>>>> For my money, we should keep _everything_ in one repo. Good enough for
>>>> Google, so good enough for me. What we release out of that repo is a
>>>> different story and what I would like to discuss next.
>>>> 
>>>> This is not the same as Maven plugins IMO but the case could be made for it
>>>> I suppose where a lot/most plugins live in their own repos. It is not the
>>>> same as with Maven IMO because our plugins rely on log4j-core and it's
>>>> guts, for which we only make loose compatibility guarantees -- as opposed
>>>> to log4j-core where we are strict(er). Maven OTOH, has a API for plugin
>>>> auteurs.
>>>> 
>>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we have
>>>> no 'official' core SPI, so splitting it off into a separate repo would
>>>> greatly increase the risk of it falling out of sync. It is just so much
>>>> more easier to maintain when it is all in one repo.
>>>> 
>>>> My proposal is to:
>>>> 
>>>> - Put everything back into one repo (Chainsaw too?)
>>>> - Define a core SPI for plugin writers where we make some statement about
>>>> BC, more than the casual 'we try not break stuff.'
>>>> - Defining what Log4j project 'components' we have and release based on
>>>> that. For example, today, all of the main Log4j repo is one component with
>>>> many modules. Chainsaw would be another component of the Log4j project.
>>>> Maybe we need to redefine components: API, File, JDBC and so on. A
>>>> component can have one of more module
>>>> - Got for there.
>>>> 
>>>> Thoughts? Help me flush this out?
>>>> 
>>>> Thank you for reading!
>>>> Gary
>>> 
>> 
> 
> 
> 


Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
Remko, 

I am all for speeding up the build. I have been complaining about this for well over a year and it keeps getting worse with every release. The compile of log4j-core takes less than 20 seconds on the machine provided by my employer. On the same machine the tests take 11 minutes. On this machine the whole build takes about 16 minutes. This is a bit frustrating as I perform a complete build at least once before I commit 95% of my changes.

My personal machine, which is where I run the release builds, is over 6 years old now and is considerably slower. On that machine a “normal” build currently takes . In addition, the release build does perform more work and runs through the build twice (although I am not sure it runs the tests twice anymore) so it is considerably longer. Also with every release we add new unit tests. It wasn’t too long ago that I was marveling that we had over 900 unit tests. We now have over 2,400. In addition, each release also requires building the web site, which can take a long time as well. 

While I suspect about 1/3 of the time of a “normal” build could be cut by executing some stuff in parallel that “fix” will only last for a short while as we keep adding components related to third party projects (Mongo, Cassandra, DBCP2, etc) that most likely don’t change much once they are done. I can’t remember the tag library module required changes. Many of them only change when the third party project changes (such as Mongo 2 vs Mongo 3). On that note I really don’t want to see Mongo 2 and Mongo 3 projects as part of the Log4j main build.

In addition to speeding up the build I would prefer to have the main log4j releases follow the 80/20 rule - they should contain only the stuff 80% of our users will use. I would venture a guess that at least 80% only use the File and Console appenders.

While we may not have as firm of a plugin API as Maven does, almost everything a user should need to create an Appender, Layout, Lookup or Filter should be as backward compatible as the Log4j API. If we can’t guarantee backward compatibility for those components we have a big problem as users are creating these kinds of components all the time. We cannot be breaking them. So from that standpoint we should have no problem moving those kinds of plugins out of the main build.

Ralph 



> On Jan 21, 2018, at 5:51 PM, Remko Popma <re...@gmail.com> wrote:
> 
> Log4j2 is not like Maven. Maven (I assume) has a plugin API. Log4j2 modules depend on the internals of log4j-core. 
> 
> I agree with Gary that not being able to verify that a core change doesn’t break any modules when they live outside the main project is a valid concern. 
> 
> Why are we talking about modules and repos when the issue is that it takes too long to do a release? Maven reports most modules as taking a handful of seconds to build. 
> 
> I keep thinking:
> 
> 1. We’re running the tests twice during a release. Let’s change this. Logically, once should be enough. What prevents us from doing this?
> 
> 2. Core tests take unnecessarily long. By running some tests in parallel and some other tests without forking I have hope we can run all core tests in ~15 minutes. 
> 
> Won’t those two changes be much more impactful in reducing the release time than any amount of module or repo reshuffling?
> 
> Remko 
> 
> (Shameless plug) Every java main() method deserves http://picocli.info
> 
>> On Jan 22, 2018, at 8:14, Ralph Goers <ra...@dslextreme.com> wrote:
>> 
>> I am very much against the idea of a single repo. Yes, you can have multiple projects in the repo but I am not sure how that can be sanely released. I much prefer the model that Maven has taken. They are now using gitbox [1] which seems to allow GitHub to be the primary repo. Every Maven plugin is individually released. Scroll down the link below to the Maven section and you can see all the plugin repos.
>> 
>> The upside to this is that it are: 
>>   1. It is far easier to perform releases of the individual components. 
>>   2. It is much easier to accept plugin contributions.
>> The downsides are:
>>   1. A page like https://maven.apache.org/plugins/ <https://maven.apache.org/plugins/> is needed to keep track of the plugin versions.
>>   2. It could make sense to have log4j-parent and log4j-bom projects. The first to help keep the builds similar and the second to help customers pick up the latest versions. 
>> 
>> Ralph
>> 
>> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/repos/asf>
>> 
>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com> wrote:
>>> 
>>> Hi All,
>>> 
>>> I am writing this to in an effort to understand how we can manage and grow
>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense but more
>>> in the maturing sense.
>>> 
>>> I am prompted to write this by Ralph's comment that log4j-core and the main
>>> repo is too big and that releases take too long make, partially prompted by
>>> my addition of a new module called log4j-jdbc-dbcp2 which currently
>>> contains one small class with a dependency on Apache Commons DBCP 2.
>>> 
>>> I would like to express my gratitude at Ralph's efforts to revitalized
>>> Log4j and performing most release management duties. Thank you Ralph!
>>> 
>>> I can see two main orthogonal issues:
>>> - The size of the git repo.
>>> - The size of the log4j-core module.
>>> 
>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
>>> another 'plugin' repo.
>>> 
>>> I really dislike this idea:
>>> - The plugin repo has never been released. Not that one releases a repo but
>>> you get the point.
>>> - How do you keep things in sync between repos and code when we have no
>>> official 'core' SPI.
>>> 
>>> For my money, we should keep _everything_ in one repo. Good enough for
>>> Google, so good enough for me. What we release out of that repo is a
>>> different story and what I would like to discuss next.
>>> 
>>> This is not the same as Maven plugins IMO but the case could be made for it
>>> I suppose where a lot/most plugins live in their own repos. It is not the
>>> same as with Maven IMO because our plugins rely on log4j-core and it's
>>> guts, for which we only make loose compatibility guarantees -- as opposed
>>> to log4j-core where we are strict(er). Maven OTOH, has a API for plugin
>>> auteurs.
>>> 
>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we have
>>> no 'official' core SPI, so splitting it off into a separate repo would
>>> greatly increase the risk of it falling out of sync. It is just so much
>>> more easier to maintain when it is all in one repo.
>>> 
>>> My proposal is to:
>>> 
>>> - Put everything back into one repo (Chainsaw too?)
>>> - Define a core SPI for plugin writers where we make some statement about
>>> BC, more than the casual 'we try not break stuff.'
>>> - Defining what Log4j project 'components' we have and release based on
>>> that. For example, today, all of the main Log4j repo is one component with
>>> many modules. Chainsaw would be another component of the Log4j project.
>>> Maybe we need to redefine components: API, File, JDBC and so on. A
>>> component can have one of more module
>>> - Got for there.
>>> 
>>> Thoughts? Help me flush this out?
>>> 
>>> Thank you for reading!
>>> Gary
>> 
> 



Re: [log4j] The shape of Log4j

Posted by Remko Popma <re...@gmail.com>.
Log4j2 is not like Maven. Maven (I assume) has a plugin API. Log4j2 modules depend on the internals of log4j-core. 

I agree with Gary that not being able to verify that a core change doesn’t break any modules when they live outside the main project is a valid concern. 

Why are we talking about modules and repos when the issue is that it takes too long to do a release? Maven reports most modules as taking a handful of seconds to build. 

I keep thinking:

1. We’re running the tests twice during a release. Let’s change this. Logically, once should be enough. What prevents us from doing this?

2. Core tests take unnecessarily long. By running some tests in parallel and some other tests without forking I have hope we can run all core tests in ~15 minutes. 

Won’t those two changes be much more impactful in reducing the release time than any amount of module or repo reshuffling?

Remko 

(Shameless plug) Every java main() method deserves http://picocli.info

> On Jan 22, 2018, at 8:14, Ralph Goers <ra...@dslextreme.com> wrote:
> 
> I am very much against the idea of a single repo. Yes, you can have multiple projects in the repo but I am not sure how that can be sanely released. I much prefer the model that Maven has taken. They are now using gitbox [1] which seems to allow GitHub to be the primary repo. Every Maven plugin is individually released. Scroll down the link below to the Maven section and you can see all the plugin repos.
> 
> The upside to this is that it are: 
>    1. It is far easier to perform releases of the individual components. 
>    2. It is much easier to accept plugin contributions.
> The downsides are:
>    1. A page like https://maven.apache.org/plugins/ <https://maven.apache.org/plugins/> is needed to keep track of the plugin versions.
>    2. It could make sense to have log4j-parent and log4j-bom projects. The first to help keep the builds similar and the second to help customers pick up the latest versions. 
> 
> Ralph
> 
> [1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/repos/asf>
> 
>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com> wrote:
>> 
>> Hi All,
>> 
>> I am writing this to in an effort to understand how we can manage and grow
>> Log4j. I use the term 'grow' not in the 'bigger is better' sense but more
>> in the maturing sense.
>> 
>> I am prompted to write this by Ralph's comment that log4j-core and the main
>> repo is too big and that releases take too long make, partially prompted by
>> my addition of a new module called log4j-jdbc-dbcp2 which currently
>> contains one small class with a dependency on Apache Commons DBCP 2.
>> 
>> I would like to express my gratitude at Ralph's efforts to revitalized
>> Log4j and performing most release management duties. Thank you Ralph!
>> 
>> I can see two main orthogonal issues:
>> - The size of the git repo.
>> - The size of the log4j-core module.
>> 
>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
>> another 'plugin' repo.
>> 
>> I really dislike this idea:
>> - The plugin repo has never been released. Not that one releases a repo but
>> you get the point.
>> - How do you keep things in sync between repos and code when we have no
>> official 'core' SPI.
>> 
>> For my money, we should keep _everything_ in one repo. Good enough for
>> Google, so good enough for me. What we release out of that repo is a
>> different story and what I would like to discuss next.
>> 
>> This is not the same as Maven plugins IMO but the case could be made for it
>> I suppose where a lot/most plugins live in their own repos. It is not the
>> same as with Maven IMO because our plugins rely on log4j-core and it's
>> guts, for which we only make loose compatibility guarantees -- as opposed
>> to log4j-core where we are strict(er). Maven OTOH, has a API for plugin
>> auteurs.
>> 
>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we have
>> no 'official' core SPI, so splitting it off into a separate repo would
>> greatly increase the risk of it falling out of sync. It is just so much
>> more easier to maintain when it is all in one repo.
>> 
>> My proposal is to:
>> 
>> - Put everything back into one repo (Chainsaw too?)
>> - Define a core SPI for plugin writers where we make some statement about
>> BC, more than the casual 'we try not break stuff.'
>> - Defining what Log4j project 'components' we have and release based on
>> that. For example, today, all of the main Log4j repo is one component with
>> many modules. Chainsaw would be another component of the Log4j project.
>> Maybe we need to redefine components: API, File, JDBC and so on. A
>> component can have one of more module
>> - Got for there.
>> 
>> Thoughts? Help me flush this out?
>> 
>> Thank you for reading!
>> Gary
> 

Re: [log4j] The shape of Log4j

Posted by Ralph Goers <ra...@dslextreme.com>.
I am very much against the idea of a single repo. Yes, you can have multiple projects in the repo but I am not sure how that can be sanely released. I much prefer the model that Maven has taken. They are now using gitbox [1] which seems to allow GitHub to be the primary repo. Every Maven plugin is individually released. Scroll down the link below to the Maven section and you can see all the plugin repos.

The upside to this is that it are: 
	1. It is far easier to perform releases of the individual components. 
	2. It is much easier to accept plugin contributions.
The downsides are:
	1. A page like https://maven.apache.org/plugins/ <https://maven.apache.org/plugins/> is needed to keep track of the plugin versions.
	2. It could make sense to have log4j-parent and log4j-bom projects. The first to help keep the builds similar and the second to help customers pick up the latest versions. 

Ralph

[1] https://gitbox.apache.org/repos/asf <https://gitbox.apache.org/repos/asf>

> On Jan 21, 2018, at 8:55 AM, Gary Gregory <ga...@gmail.com> wrote:
> 
> Hi All,
> 
> I am writing this to in an effort to understand how we can manage and grow
> Log4j. I use the term 'grow' not in the 'bigger is better' sense but more
> in the maturing sense.
> 
> I am prompted to write this by Ralph's comment that log4j-core and the main
> repo is too big and that releases take too long make, partially prompted by
> my addition of a new module called log4j-jdbc-dbcp2 which currently
> contains one small class with a dependency on Apache Commons DBCP 2.
> 
> I would like to express my gratitude at Ralph's efforts to revitalized
> Log4j and performing most release management duties. Thank you Ralph!
> 
> I can see two main orthogonal issues:
> - The size of the git repo.
> - The size of the log4j-core module.
> 
> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to
> another 'plugin' repo.
> 
> I really dislike this idea:
> - The plugin repo has never been released. Not that one releases a repo but
> you get the point.
> - How do you keep things in sync between repos and code when we have no
> official 'core' SPI.
> 
> For my money, we should keep _everything_ in one repo. Good enough for
> Google, so good enough for me. What we release out of that repo is a
> different story and what I would like to discuss next.
> 
> This is not the same as Maven plugins IMO but the case could be made for it
> I suppose where a lot/most plugins live in their own repos. It is not the
> same as with Maven IMO because our plugins rely on log4j-core and it's
> guts, for which we only make loose compatibility guarantees -- as opposed
> to log4j-core where we are strict(er). Maven OTOH, has a API for plugin
> auteurs.
> 
> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we have
> no 'official' core SPI, so splitting it off into a separate repo would
> greatly increase the risk of it falling out of sync. It is just so much
> more easier to maintain when it is all in one repo.
> 
> My proposal is to:
> 
> - Put everything back into one repo (Chainsaw too?)
> - Define a core SPI for plugin writers where we make some statement about
> BC, more than the casual 'we try not break stuff.'
> - Defining what Log4j project 'components' we have and release based on
> that. For example, today, all of the main Log4j repo is one component with
> many modules. Chainsaw would be another component of the Log4j project.
> Maybe we need to redefine components: API, File, JDBC and so on. A
> component can have one of more module
> - Got for there.
> 
> Thoughts? Help me flush this out?
> 
> Thank you for reading!
> Gary


Re: [log4j] The shape of Log4j

Posted by Jochen Wiedmann <jo...@gmail.com>.
On Sun, Jan 21, 2018 at 4:55 PM, Gary Gregory <ga...@gmail.com> wrote:

> - Put everything back into one repo (Chainsaw too?)

In general, yes. However, Chainsaw would be one step too far. From my
point of view, Chainsaw is a consumer, and not a part of log4j-core.
At the most, split a meaningful part out of Chainsaw (Parsers come to
my mind), and put that back into the core repo. However, things like a
UI are definitely misplaced. (Think of the Linux kernel, which
carefully separates user space from kernel.)

In general, I do think, that several repos should work, too. There are
tools, like Jenkins, which are giving basically the same level of
integration (Commit triggers build, build triggers build, etc.)

Jochen