You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@logging.apache.org by Volkan Yazıcı <vo...@yazi.ci> on 2023/10/02 07:04:54 UTC
Re: Refining product feature set strategy
I give up. Gut-feeling-driven development FTW.
Ralph, quoting your own words, "async and gc-free stuff exploded the
complexity of the code base". Plus, we are yet to reproduce Remko's "log4j
is faster than logback" results on a modern stack, where all of our earlier
attempts failed. Now I find it ironic that you use async/gc-free as an
example to convince me how well our gut-feelings paid off.
All in all, I heard the PMC. We are all converging to a "Ceki Reloaded"
scenario here. I will shut up.
On Sat, Sep 30, 2023 at 12:52 AM Ralph Goers <ra...@dslextreme.com>
wrote:
> I’m sorry, I can’t really agree with much of any of this. Following the
> thoughts being proposed in this thread much of Log4j 2 and even the initial
> work I did to create it would not have seen the light of day. Almost 100%
> of the stuff Matt has done would never have happened.
>
> It is a fact that people come and go from projects. Some stay longer than
> others. We have had PMC members who disappeared the day after they were
> elected to the PMC. Remko arguably made two of the most significant
> contributions in the GC-Free and Async work. Yet he has largely gone
> inactive. It happens. Even after accepting the JsonTemplateLayout and
> Volkan as a committer we had no guarantee he would stick around. We hoped
> he would and he did. Gary and I have each contributed a large number of
> components because they met some need we have/had and we continue to
> support them.
>
> Requiring hoops that must be jumped through before stuff can be accepted
> is just a terrible idea. It does nothing but create a perception that we
> are not open to new contributions and new contributors.
>
> Yes, we can retire things. I don’t think anyone is arguing against
> retiring log4j-cassandra. I haven’t seen any arguments against log4j-kafka.
> That is primarily because they are woefully out of date. However, there is
> nothing wrong with a module that hasn’t had any updates in a long while if
> it is meeting user’s needs. You are never going to know for certain home
> many users there are of a given module. That data is impossible to get. Any
> company that has their own repository manager (which should be most) will
> download a release of an artifact from Maven Central once and it will be
> hard for you to know when they did it. The metrics Sonatype provided came
> from their security product, which not everyone purchases. Yes, you can
> make guesses from those numbers but you can’t actually know how good those
> guesses are.
>
> In short, I am in favor of retiring things when they are no longer
> maintainable. I am completely in favor of adding new components when they
> make sense. IOW, every contribution needs to be considered on its own
> merits.
>
> Ralph
>
>
>
> > On Sep 29, 2023, at 11:32 AM, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > I think Jira is good enough for that, since there is transition from
> state
> > to state that can be used to shepherd issues through. RFC, JEP, all way
> too
> > heavy handed for us IMO.
> >
> > Gary
> >
> >
> > On Fri, Sep 29, 2023, 2:05 PM Matt Sicker <ma...@musigma.org> wrote:
> >
> >> I think it could be valuable for us to establish some form of an RFC
> >> process for proposing and developing major new features. I also want to
> >> avoid being too process-heavy here as that would also disincentivize
> >> contributions. I agree that we should try to be more data-driven to
> >> determine what features and products should have the most attention.
> >>
> >> I do like the idea of having “sample” plugins which are not distributed
> as
> >> binaries, though, which can be used in documentation for examples of
> how to
> >> integrate your own systems. With the flexibility of the plugin system
> here,
> >> we can defer the implementation of more obscure integrations to the end
> >> user.
> >>
> >> I will note that we do already have some level of filtering to what we
> >> include. I’ve proposed numerous features in the past that I didn’t
> pursue
> >> due to reasons raised by others or lack of interest.
> >>
> >>> On Sep 29, 2023, at 2:59 AM, Volkan Yazıcı <vo...@yazi.ci> wrote:
> >>>
> >>> I want to challenge the current way of PMC determining the product
> >> feature
> >>> set and work towards a more sustainable alternative.
> >>>
> >>> Logging Services team...
> >>>
> >>> - delivers mission-critical products that are deployed at the core of
> >>> the world-wide infrastructure (actually, in Mars too)
> >>> - is short on development resources compared to its wide range of
> >>> offerings
> >>> - deals with substantial amount of legacy
> >>> - suffers from knowledge silo
> >>>
> >>> Any IT team in this state would be really conservative on accepting new
> >>> features and try hard to get rid of the bloat. They would react to this
> >>> challenge by first discovering the user base, pinpointing the core
> >> values,
> >>> and then lazer-focusing the limited development resources on to the
> >> crucial
> >>> deliverables. Though we do something totally different, one can even
> say
> >>> the opposite. Below I share excerpts from recent mailing list posts.
> >>>
> >>> *"I had added [X] ... to show how to write plugins and for some
> >>> consulting-related use cases"* – PMC member explaining how feature X
> was
> >>> released
> >>>
> >>> *"... stuff seems like it could be useful"* – PMC member asking to keep
> >>> feature X
> >>>
> >>> *"my employer used [X] ... for anyone implementing ... this would be
> very
> >>> relevant. By archiving this we are basically telling users that they
> >>> cannot use it any more since it will no longer be supported. For that
> >>> reason I am not in favor of [retiring]"* – PMC member asking to keep
> >>> feature X
> >>>
> >>> *"We [the employer] ... use [X]"* – PMC member asking to keep feature X
> >>>
> >>> *"User base [is] ... very low to non-existent."* – PMC member asking to
> >>> keep product X
> >>>
> >>> *"[PMC member] has steadfastly [reacted] ... and until he positively
> says
> >>> he will no longer maintain it I am not willing to override that"* – PMC
> >>> member asking to keep product X because another (one and only) PMC
> member
> >>> reacted on product retirement inquiry
> >>>
> >>> *"Our employers are ... paying customers."* – a PMC member explaining
> why
> >>> we should keep feature X used by an organization employing another PMC
> >>> member
> >>>
> >>>
> >>> I see a pattern in the way we decide to maintain a feature/product:
> >>>
> >>> - It is not data-driven. It is disconnected from its user base. No
> >> usage
> >>> statistics, etc. is shared or used.
> >>> - We serve individuals' and employers' agendas.
> >>> - We underestimate the cost of adding/keeping a feature.
> >>>
> >>> I think this diet is "unhealthy" because:
> >>>
> >>> - It adds up to the bloat. It is yet another component developers need
> >>> to maintain its dependencies, documentation, website, integration with
> >> the
> >>> build system, etc. This bizarrely slows down the development
> >> experience.
> >>> (Ever wondered how much `log4j-osgi` takes during `./mvnw verify`?)
> >>> - It adds up to the attack surface.
> >>> - Features that are supposed to be optional get shipped to users
> >> without
> >>> their consent. (Consider the percentage of the Log4Shell victims that
> >> used
> >>> the JNDI functionality at all.)
> >>> - Scarce development resources get wasted on chores due to the
> >>> excessive landscape.
> >>> - It gives users the wrong impression that the feature/product is
> >>> maintained, though under the hood it is just kept there because a
> >>> privileged group wants so.
> >>>
> >>> I want to refine this approach with your help. To start the
> discussion, I
> >>> propose the following strategy instead:
> >>>
> >>> *"We only accept a feature with data-driven justification."* – Have a
> >> brand
> >>> new idea? Grab yourself a GitHub/GitLab repository that belongs to
> either
> >>> you or your employer, knock yourself out without any ASF/PMC burdens,
> and
> >>> amaze your users. Once the user attraction gets enough gravity, let's
> >>> discuss blessing it as a part of the official Logging Services
> offering.
> >>> Official support channels are always open to assist the developers of
> >> such
> >>> external efforts.
> >>>
> >>> *"We only keep a feature with data-driven justification."* – Do you
> >> think a
> >>> feature needs to be retired? Bring it up for discussion. PMC should
> >>> evaluate the request based on numbers and this process should be
> >>> independent of the individuals'/employers' agendas. If PMC decides to
> >>> retire a feature/product *and* there are people who volunteer to
> continue
> >>> the development on a successor fork outside of ASF, PMC should do their
> >> due
> >>> diligence to point existing users to this fork without any endorsement.
> >>>
> >>>
> >>> I look forward to hearing your thoughts.
> >>
> >>
>
>
Re: Refining product feature set strategy
Posted by Ralph Goers <ra...@dslextreme.com>.
> On Oct 2, 2023, at 12:04 AM, Volkan Yazıcı <vo...@yazi.ci> wrote:
>
> I give up. Gut-feeling-driven development FTW.
>
> Ralph, quoting your own words, "async and gc-free stuff exploded the
> complexity of the code base".
Yes it did. Matt has refactored the Plugin support at least 3 times and each time it has become more complex. That doesn’t mean it was wrong to do it or it was bad. Likewise with gc-free and async. With some effort we can Improve the situation for those if we choose to.
> Plus, we are yet to reproduce Remko's "log4j
> is faster than logback" results on a modern stack, where all of our earlier
> attempts failed.
I never said his results were incorrect. I just said I have never understood how he got them. I couldn’t reproduce them because I never got the full information on how the tests were performed.
> Now I find it ironic that you use async/gc-free as an
> example to convince me how well our gut-feelings paid off.
Umm. Lots of people are doing async logging. I am quite positive that had a beneficial impact on Log4j2.
>
> All in all, I heard the PMC. We are all converging to a "Ceki Reloaded"
> scenario here. I will shut up.
1. I will never encourage you to shut up.
2. I have no idea what you are referring to in regards to Ceki or how anything he has done with his projects relates to this issue.
Ralph