You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@maven.apache.org by Enrico Olivelli <eo...@gmail.com> on 2020/01/28 07:56:29 UTC

Maven Runtime Metrics

Hi,
I would like to work on introducing an explicit support in Maven Core for
recording "metrics".
As far as I know (and I have still limited knowledge about the internals)
you can write an "extension" and you can intercept main lifecycle events,
and using these hooks you can try to record metrics.
But my proposal is from another point of view and I want to go deeper.

We are adding caches, making refactors....all good things, but we should
have a way to track how Maven is behaving in a measurable manner.

Usually in order to capture realtime metrics you are using some Metrics
framework, like Prometheus, Dropwizards....and you attach your process to a
metrics database and capture all of the values.
Then you can process all of the data, realtime and/or offline, you can
create dashboards, you can also fine tune your configuration or change your
code.

These frameworks deal with time series created by capturing  Counters,
Summaries, Gauges.

I would like to introduce an API to be used by Maven Core and by Plugin to
expose internal metrics, like:
- internal caches size
- memory usage
- disk reads/writes
- network traffic
- any kind of specific metric for plugins (like "number of tests executed",
"number of files sent to javac...")
- .... all of the usual stuff you track in order to understand wha's going
on

I am saying a Maven Metrics API because we must not stick to a single
"provider" (say "DropWizard"), it will be a bad choice in the long term.

The API will expose to Maven Core and to plugins a way to get access to
Metrics and report useful values.

@InjectMe
MetricsProvider metrics;

metrics.getCounter("javacFiles").add(....)
metrics.getSummary("downloadTime").add(....)

I don't want to go down into details as this is only a preliminary email
but in my view we could let the user attach an "implementation" just by
dropping in an "extension", in order not to ship with Maven Core a lot of
third party dependencies.

I have some "production" experience with this topic (actually I did
something like that in my company and in ZooKeeper project) so I will be
happy to prepare and share a design document.

With this work we will open up the way to knowing how a Maven build works,
with the 'time dimension', on the local machine of the developer but even
on CI servers.

Any comment is very welcome

Regards
Enrico

Re: Maven Runtime Metrics

Posted by Romain Manni-Bucau <rm...@gmail.com>.

Looks very interesting (and good work btw!), very impatient to see what the
end dump looks like with wagon and potentially local (disk) I/O!
Next step is likely to try to use as an extension a shade of a "common"
impl (I'm thinking to microprofile metrics but could be codehale one) to
have exporters and be able to persist that.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le lun. 13 avr. 2020 à 15:32, Enrico Olivelli <eo...@gmail.com> a
écrit :

> Hi,
> news about the status of my POC:
> 1) I have created a demo Metrics Provider [1]
> 2) I have updated my branch on Maven Studies [2]
>
> It works everything as expected.
> In order to setup an appealing "demo" I have to add "interesting"
> metrics...
>
> Every suggestion is welcome, as my next step I will try to instrument the
> I/O done by Maven itself like local repository access and the model
> building time
>
> Enrico
>
> [1] https://github.com/eolivelli/simplemavenmetrics
> [2] https://github.com/apache/maven-studies/tree/maven-metrics
>
> Il giorno dom 5 apr 2020 alle ore 14:37 Enrico Olivelli <
> eolivelli@gmail.com>
> ha scritto:
>
> > I have managed to use @Inject and the extensions mechanism.
> > Thank you for your advices.
> >
> > I am now implementing some useful "simple" metrics providers (two
> > "extensions") in order to see this stuff working.
> > My idea is to have two interesting implementations:
> > - Dump stats at the end of the CLI session (using local DropWizard
> > counters/summaries)
> > - Send stats to some Metrics Systems: I am using Prometheus.io at work
> > so I will use that system
> >
> > It will be interesting to have some Grafana dashboard that tells be
> > how Maven is working on my laptop or on Jenkins.
> >
> > Once my POC is ready the next step will be to start instrumenting
> > useful places in Maven Core.
> > We should instrument only "useful" parts of Maven Core.
> > Many Metrics libraries provide standard JVM metrics by default.
> > I think it will be mostly up to the plugins to expose use full value.
> >
> > Personally I am interesting in seeing this stuff:
> > - execution time for checkstyle, spotbugs...
> > - execution time for javac
> > - number of network transfers (Wagon HTTP) and bandwidth....
> > - time to write to disk artifacts (to local repository, in 'install',
> > to "target"....)
> >
> > Enrico
> >
> > Il giorno dom 29 mar 2020 alle ore 10:40 Romain Manni-Bucau
> > <rm...@gmail.com> ha scritto:
> > >
> > > It depends how it is done.
> > >
> > > @Inject MetricsSystem ms; will likely fail but if you inject the
> > container
> > > and do the lookup you can handle the absence of the metrics, less
> elegant
> > > but working as well.
> > >
> > > Alternative is to inject a custom compiler component and have 2
> > components
> > > impl (hints being different), one with injected metrics, one with noop
> > > metrics hardcoded.
> > > Depending the current maven version you select the right hint.
> > > Think it is more or less what we do to handle aether change in some
> > earlier
> > > maven version.
> > >
> > > Romain Manni-Bucau
> > > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > > <https://rmannibucau.metawerx.net/> | Old Blog
> > > <http://rmannibucau.wordpress.com> | Github <
> > https://github.com/rmannibucau> |
> > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > > <
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > >
> > >
> > >
> > > Le dim. 29 mars 2020 à 10:33, Enrico Olivelli <eo...@gmail.com> a
> > > écrit :
> > >
> > > > Self answer: I had written the DefaultMetricsSystem file in the wrong
> > > > directory ! (in maven core tests!)
> > > > Sorry fo the noise
> > > >
> > > > I have just one problem.
> > > > Currently I am using Jsr300 to plug the MetricsSystem to Maven Core
> > > > and this would be the expected way for Plugins.
> > > >
> > > > Say now that I want to use my MetricsSystem bean (@Inject
> > > > MetricsSystem) in the Maven Compiler Plugin or in Wagon...
> > > > let's say we do a 3.8.2 Maven Compiler plugin and that we deliver
> this
> > > > MetricsSystem in Maven 3.7.0...
> > > > Will Maven compiler plugin 3.8.2 work on Maven 3.6.x that does not
> > > > bundle the definition of the MetricsSytem ?
> > > >
> > > > Ideally I would like to have a "null" value or a Default NOOP
> > > > implementation when the plugin is not running on Maven 3.7.0
> > > > Will it work ?
> > > >
> > > > I will continue with the POC then I will share my model.
> > > > The change to Maven Core is almost there.
> > > > Now my plan is:
> > > > - Create an independent "Maven Extension" (in my github space, non
> ASF
> > > > by now, to make it clear that this system is totally pluggable with
> > > > extensions) that wraps Dropwizard Metrics and outputs the results to
> > > > file at the end of the build
> > > > - Add some initial metrics in Maven runtime
> > > > - Try with some plugin
> > > >
> > > >
> > > >
> > > >
> > > > Enrico
> > > >
> > > > Il giorno sab 28 mar 2020 alle ore 15:57 Enrico Olivelli
> > > > <eo...@gmail.com> ha scritto:
> > > > >
> > > > > Hi,
> > > > > I have pushed a new version of my prof-of-concept
> > > > >
> > https://github.com/apache/maven-studies/compare/maven-metrics?expand=1
> > > > >
> > > > > I am introducing a MetricsSystem interface and a
> DefaultMetricsSystem
> > > > CDI bean.
> > > > > I am new to sisu/plexsus container.
> > > > > It looks like that DefaultMetricsSystem is getting instantiated in
> > > > > Maven Core tests
> > > > >
> > > > > This is my work:
> > > > >
> > https://github.com/apache/maven-studies/compare/maven-metrics?expand=1
> > > > >
> > > > > But if I run Maven from the command line I get the error below
> > > > >
> > > > > Maybe I missing some config/annotation
> > > > >
> > > > > Any help is appreciated !
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > > eolivelli@LAPTOP-OODVV59Q apache-maven-3.7.0-SNAPSHOT]$ bin/mvn
> > > > > [WARNING] Error injecting: org.apache.maven.DefaultMaven
> > > > > com.google.inject.ProvisionException: Unable to provision, see the
> > > > > following errors:
> > > > >
> > > > > 1) No implementation for org.apache.maven.metrics.MetricsSystem was
> > > > bound.
> > > > >   at ClassRealm[plexus.core, parent: null] (via modules:
> > > > > org.eclipse.sisu.wire.WireModule ->
> > > > > org.eclipse.sisu.plexus.PlexusBindingModule)
> > > > >   while locating
> org.apache.maven.lifecycle.internal.LifecycleStarter
> > > > >   while locating org.apache.maven.DefaultMaven
> > > > >
> > > > > 1 error
> > > > >     at
> > > >
> >
> com.google.inject.internal.InternalProvisionException.toProvisionException
> > > > > (InternalProvisionException.java:226)
> > > > >     at com.google.inject.internal.InjectorImpl$1.get
> > > > (InjectorImpl.java:1053)
> > > > >     at com.google.inject.internal.InjectorImpl.getInstance
> > > > > (InjectorImpl.java:1086)
> > > > >     at org.eclipse.sisu.space.AbstractDeferredClass.get
> > > > > (AbstractDeferredClass.java:48)
> > > > >     at com.google.inject.internal.ProviderInternalFactory.provision
> > > > > (ProviderInternalFactory.java:85)
> > > > >     at
> > > >
> >
> com.google.inject.internal.InternalFactoryToInitializableAdapter.provision
> > > > > (InternalFactoryToInitializableAdapter.java:57)
> > > > >     at com.google.inject.internal.ProviderInternalFactory$1.call
> > > > > (ProviderInternalFactory.java:66)
> > > > >     at
> > > >
> >
> com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision
> > > > > (ProvisionListenerStackCallback.java:112)
> > > > >     at
> org.eclipse.sisu.bean.BeanScheduler$CycleActivator.onProvision
> > > > > (BeanScheduler.java:230)
> > > > >     at
> > > >
> >
> com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision
> > > > > (ProvisionListenerStackCallback.java:120)
> > > > >     at
> > > > com.google.inject.internal.ProvisionListenerStackCallback.provision
> > > > > (ProvisionListenerStackCallback.java:66)
> > > > >     at
> com.google.inject.internal.ProviderInternalFactory.circularGet
> > > > > (ProviderInternalFactory.java:61)
> > > > >     at
> > > > com.google.inject.internal.InternalFactoryToInitializableAdapter.get
> > > > > (InternalFactoryToInitializableAdapter.java:47)
> > > > >     at
> > com.google.inject.internal.ProviderToInternalFactoryAdapter.get
> > > > > (ProviderToInternalFactoryAdapter.java:40)
> > > > >     at com.google.inject.internal.SingletonScope$1.get
> > > > (SingletonScope.java:168)
> > > > >     at
> > com.google.inject.internal.InternalFactoryToProviderAdapter.get
> > > > > (InternalFactoryToProviderAdapter.java:39)
> > > > >     at com.google.inject.internal.InjectorImpl$1.get
> > > > (InjectorImpl.java:1050)
> > > > >     at org.eclipse.sisu.inject.LazyBeanEntry.getValue
> > > > (LazyBeanEntry.java:81)
> > > > >     at org.eclipse.sisu.plexus.LazyPlexusBean.getValue
> > > > (LazyPlexusBean.java:51)
> > > > >     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> > > > > (DefaultPlexusContainer.java:263)
> > > > >     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> > > > > (DefaultPlexusContainer.java:255)
> > > > >     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> > > > > (DefaultPlexusContainer.java:249)
> > > > >     at org.apache.maven.cli.MavenCli.container (MavenCli.java:696)
> > > > >     at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:284)
> > > > >     at org.apache.maven.cli.MavenCli.main (MavenCli.java:195)
> > > > >     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0
> (Native
> > > > Method)
> > > > >     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> > > > > (NativeMethodAccessorImpl.java:62)
> > > > >     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> > > > > (DelegatingMethodAccessorImpl.java:43)
> > > > >     at java.lang.reflect.Method.invoke (Method.java:564)
> > > > >     at
> > org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> > > > > (Launcher.java:282)
> > > > >     at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> > > > > (Launcher.java:225)
> > > > >     at
> > org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> > > > > (Launcher.java:406)
> > > > >     at org.codehaus.plexus.classworlds.launcher.Launcher.main
> > > > > (Launcher.java:347)
> > > > > [ERROR] Error executing Maven.
> > > > > [ERROR] com.google.inject.ProvisionException: Unable to provision,
> > see
> > > > > the following errors:
> > > > >
> > > > > 1) No implementation for org.apache.maven.metrics.MetricsSystem was
> > > > bound.
> > > > >   at ClassRealm[plexus.core, parent: null] (via modules:
> > > > > org.eclipse.sisu.wire.WireModule ->
> > > > > org.eclipse.sisu.plexus.PlexusBindingModule)
> > > > >   while locating
> org.apache.maven.lifecycle.internal.LifecycleStarter
> > > > >   while locating org.apache.maven.DefaultMaven
> > > > >   at ClassRealm[plexus.core, parent: null] (via modules:
> > > > > org.eclipse.sisu.wire.WireModule ->
> > > > > org.eclipse.sisu.plexus.PlexusBindingModule)
> > > > >   while locating org.apache.maven.Maven
> > > > >
> > > > > 1 error
> > > > >       role: org.apache.maven.Maven
> > > > >
> > > > > Il giorno mar 17 mar 2020 alle ore 08:35 Hervé BOUTEMY
> > > > > <he...@free.fr> ha scritto:
> > > > > >
> > > > > > Le samedi 14 mars 2020, 15:14:17 CET Enrico Olivelli a écrit :
> > > > > > > I am starting this work.
> > > > > > > I have pushed a copy of current maven core master to
> > maven-studies.
> > > > > > nice
> > > > > >
> > > > > > [...]
> > > > > > > I don't know how classloading works very well in the case of
> > > > Maven/Plexus.
> > > > > > your approach looks reasonable: you're adding extensions to the
> > "core"
> > > > > > classloader [1]
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Hervé
> > > > > >
> > > > > > [1]
> > https://maven.apache.org/guides/mini/guide-maven-classloading.html
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > >
> > > >
> >
>

Re: Maven Runtime Metrics

Posted by Enrico Olivelli <eo...@gmail.com>.

Hi,
news about the status of my POC:
1) I have created a demo Metrics Provider [1]
2) I have updated my branch on Maven Studies [2]

It works everything as expected.
In order to setup an appealing "demo" I have to add "interesting" metrics...

Every suggestion is welcome, as my next step I will try to instrument the
I/O done by Maven itself like local repository access and the model
building time

Enrico

[1] https://github.com/eolivelli/simplemavenmetrics
[2] https://github.com/apache/maven-studies/tree/maven-metrics

Il giorno dom 5 apr 2020 alle ore 14:37 Enrico Olivelli <eo...@gmail.com>
ha scritto:

> I have managed to use @Inject and the extensions mechanism.
> Thank you for your advices.
>
> I am now implementing some useful "simple" metrics providers (two
> "extensions") in order to see this stuff working.
> My idea is to have two interesting implementations:
> - Dump stats at the end of the CLI session (using local DropWizard
> counters/summaries)
> - Send stats to some Metrics Systems: I am using Prometheus.io at work
> so I will use that system
>
> It will be interesting to have some Grafana dashboard that tells be
> how Maven is working on my laptop or on Jenkins.
>
> Once my POC is ready the next step will be to start instrumenting
> useful places in Maven Core.
> We should instrument only "useful" parts of Maven Core.
> Many Metrics libraries provide standard JVM metrics by default.
> I think it will be mostly up to the plugins to expose use full value.
>
> Personally I am interesting in seeing this stuff:
> - execution time for checkstyle, spotbugs...
> - execution time for javac
> - number of network transfers (Wagon HTTP) and bandwidth....
> - time to write to disk artifacts (to local repository, in 'install',
> to "target"....)
>
> Enrico
>
> Il giorno dom 29 mar 2020 alle ore 10:40 Romain Manni-Bucau
> <rm...@gmail.com> ha scritto:
> >
> > It depends how it is done.
> >
> > @Inject MetricsSystem ms; will likely fail but if you inject the
> container
> > and do the lookup you can handle the absence of the metrics, less elegant
> > but working as well.
> >
> > Alternative is to inject a custom compiler component and have 2
> components
> > impl (hints being different), one with injected metrics, one with noop
> > metrics hardcoded.
> > Depending the current maven version you select the right hint.
> > Think it is more or less what we do to handle aether change in some
> earlier
> > maven version.
> >
> > Romain Manni-Bucau
> > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > <https://rmannibucau.metawerx.net/> | Old Blog
> > <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> >
> >
> > Le dim. 29 mars 2020 à 10:33, Enrico Olivelli <eo...@gmail.com> a
> > écrit :
> >
> > > Self answer: I had written the DefaultMetricsSystem file in the wrong
> > > directory ! (in maven core tests!)
> > > Sorry fo the noise
> > >
> > > I have just one problem.
> > > Currently I am using Jsr300 to plug the MetricsSystem to Maven Core
> > > and this would be the expected way for Plugins.
> > >
> > > Say now that I want to use my MetricsSystem bean (@Inject
> > > MetricsSystem) in the Maven Compiler Plugin or in Wagon...
> > > let's say we do a 3.8.2 Maven Compiler plugin and that we deliver this
> > > MetricsSystem in Maven 3.7.0...
> > > Will Maven compiler plugin 3.8.2 work on Maven 3.6.x that does not
> > > bundle the definition of the MetricsSytem ?
> > >
> > > Ideally I would like to have a "null" value or a Default NOOP
> > > implementation when the plugin is not running on Maven 3.7.0
> > > Will it work ?
> > >
> > > I will continue with the POC then I will share my model.
> > > The change to Maven Core is almost there.
> > > Now my plan is:
> > > - Create an independent "Maven Extension" (in my github space, non ASF
> > > by now, to make it clear that this system is totally pluggable with
> > > extensions) that wraps Dropwizard Metrics and outputs the results to
> > > file at the end of the build
> > > - Add some initial metrics in Maven runtime
> > > - Try with some plugin
> > >
> > >
> > >
> > >
> > > Enrico
> > >
> > > Il giorno sab 28 mar 2020 alle ore 15:57 Enrico Olivelli
> > > <eo...@gmail.com> ha scritto:
> > > >
> > > > Hi,
> > > > I have pushed a new version of my prof-of-concept
> > > >
> https://github.com/apache/maven-studies/compare/maven-metrics?expand=1
> > > >
> > > > I am introducing a MetricsSystem interface and a DefaultMetricsSystem
> > > CDI bean.
> > > > I am new to sisu/plexsus container.
> > > > It looks like that DefaultMetricsSystem is getting instantiated in
> > > > Maven Core tests
> > > >
> > > > This is my work:
> > > >
> https://github.com/apache/maven-studies/compare/maven-metrics?expand=1
> > > >
> > > > But if I run Maven from the command line I get the error below
> > > >
> > > > Maybe I missing some config/annotation
> > > >
> > > > Any help is appreciated !
> > > >
> > > > Enrico
> > > >
> > > >
> > > > eolivelli@LAPTOP-OODVV59Q apache-maven-3.7.0-SNAPSHOT]$ bin/mvn
> > > > [WARNING] Error injecting: org.apache.maven.DefaultMaven
> > > > com.google.inject.ProvisionException: Unable to provision, see the
> > > > following errors:
> > > >
> > > > 1) No implementation for org.apache.maven.metrics.MetricsSystem was
> > > bound.
> > > >   at ClassRealm[plexus.core, parent: null] (via modules:
> > > > org.eclipse.sisu.wire.WireModule ->
> > > > org.eclipse.sisu.plexus.PlexusBindingModule)
> > > >   while locating org.apache.maven.lifecycle.internal.LifecycleStarter
> > > >   while locating org.apache.maven.DefaultMaven
> > > >
> > > > 1 error
> > > >     at
> > >
> com.google.inject.internal.InternalProvisionException.toProvisionException
> > > > (InternalProvisionException.java:226)
> > > >     at com.google.inject.internal.InjectorImpl$1.get
> > > (InjectorImpl.java:1053)
> > > >     at com.google.inject.internal.InjectorImpl.getInstance
> > > > (InjectorImpl.java:1086)
> > > >     at org.eclipse.sisu.space.AbstractDeferredClass.get
> > > > (AbstractDeferredClass.java:48)
> > > >     at com.google.inject.internal.ProviderInternalFactory.provision
> > > > (ProviderInternalFactory.java:85)
> > > >     at
> > >
> com.google.inject.internal.InternalFactoryToInitializableAdapter.provision
> > > > (InternalFactoryToInitializableAdapter.java:57)
> > > >     at com.google.inject.internal.ProviderInternalFactory$1.call
> > > > (ProviderInternalFactory.java:66)
> > > >     at
> > >
> com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision
> > > > (ProvisionListenerStackCallback.java:112)
> > > >     at org.eclipse.sisu.bean.BeanScheduler$CycleActivator.onProvision
> > > > (BeanScheduler.java:230)
> > > >     at
> > >
> com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision
> > > > (ProvisionListenerStackCallback.java:120)
> > > >     at
> > > com.google.inject.internal.ProvisionListenerStackCallback.provision
> > > > (ProvisionListenerStackCallback.java:66)
> > > >     at com.google.inject.internal.ProviderInternalFactory.circularGet
> > > > (ProviderInternalFactory.java:61)
> > > >     at
> > > com.google.inject.internal.InternalFactoryToInitializableAdapter.get
> > > > (InternalFactoryToInitializableAdapter.java:47)
> > > >     at
> com.google.inject.internal.ProviderToInternalFactoryAdapter.get
> > > > (ProviderToInternalFactoryAdapter.java:40)
> > > >     at com.google.inject.internal.SingletonScope$1.get
> > > (SingletonScope.java:168)
> > > >     at
> com.google.inject.internal.InternalFactoryToProviderAdapter.get
> > > > (InternalFactoryToProviderAdapter.java:39)
> > > >     at com.google.inject.internal.InjectorImpl$1.get
> > > (InjectorImpl.java:1050)
> > > >     at org.eclipse.sisu.inject.LazyBeanEntry.getValue
> > > (LazyBeanEntry.java:81)
> > > >     at org.eclipse.sisu.plexus.LazyPlexusBean.getValue
> > > (LazyPlexusBean.java:51)
> > > >     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> > > > (DefaultPlexusContainer.java:263)
> > > >     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> > > > (DefaultPlexusContainer.java:255)
> > > >     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> > > > (DefaultPlexusContainer.java:249)
> > > >     at org.apache.maven.cli.MavenCli.container (MavenCli.java:696)
> > > >     at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:284)
> > > >     at org.apache.maven.cli.MavenCli.main (MavenCli.java:195)
> > > >     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native
> > > Method)
> > > >     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> > > > (NativeMethodAccessorImpl.java:62)
> > > >     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> > > > (DelegatingMethodAccessorImpl.java:43)
> > > >     at java.lang.reflect.Method.invoke (Method.java:564)
> > > >     at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> > > > (Launcher.java:282)
> > > >     at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> > > > (Launcher.java:225)
> > > >     at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> > > > (Launcher.java:406)
> > > >     at org.codehaus.plexus.classworlds.launcher.Launcher.main
> > > > (Launcher.java:347)
> > > > [ERROR] Error executing Maven.
> > > > [ERROR] com.google.inject.ProvisionException: Unable to provision,
> see
> > > > the following errors:
> > > >
> > > > 1) No implementation for org.apache.maven.metrics.MetricsSystem was
> > > bound.
> > > >   at ClassRealm[plexus.core, parent: null] (via modules:
> > > > org.eclipse.sisu.wire.WireModule ->
> > > > org.eclipse.sisu.plexus.PlexusBindingModule)
> > > >   while locating org.apache.maven.lifecycle.internal.LifecycleStarter
> > > >   while locating org.apache.maven.DefaultMaven
> > > >   at ClassRealm[plexus.core, parent: null] (via modules:
> > > > org.eclipse.sisu.wire.WireModule ->
> > > > org.eclipse.sisu.plexus.PlexusBindingModule)
> > > >   while locating org.apache.maven.Maven
> > > >
> > > > 1 error
> > > >       role: org.apache.maven.Maven
> > > >
> > > > Il giorno mar 17 mar 2020 alle ore 08:35 Hervé BOUTEMY
> > > > <he...@free.fr> ha scritto:
> > > > >
> > > > > Le samedi 14 mars 2020, 15:14:17 CET Enrico Olivelli a écrit :
> > > > > > I am starting this work.
> > > > > > I have pushed a copy of current maven core master to
> maven-studies.
> > > > > nice
> > > > >
> > > > > [...]
> > > > > > I don't know how classloading works very well in the case of
> > > Maven/Plexus.
> > > > > your approach looks reasonable: you're adding extensions to the
> "core"
> > > > > classloader [1]
> > > > >
> > > > > Regards,
> > > > >
> > > > > Hervé
> > > > >
> > > > > [1]
> https://maven.apache.org/guides/mini/guide-maven-classloading.html
> > > > >
> > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
>

Re: Maven Runtime Metrics

Posted by Enrico Olivelli <eo...@gmail.com>.

I have managed to use @Inject and the extensions mechanism.
Thank you for your advices.

I am now implementing some useful "simple" metrics providers (two
"extensions") in order to see this stuff working.
My idea is to have two interesting implementations:
- Dump stats at the end of the CLI session (using local DropWizard
counters/summaries)
- Send stats to some Metrics Systems: I am using Prometheus.io at work
so I will use that system

It will be interesting to have some Grafana dashboard that tells be
how Maven is working on my laptop or on Jenkins.

Once my POC is ready the next step will be to start instrumenting
useful places in Maven Core.
We should instrument only "useful" parts of Maven Core.
Many Metrics libraries provide standard JVM metrics by default.
I think it will be mostly up to the plugins to expose use full value.

Personally I am interesting in seeing this stuff:
- execution time for checkstyle, spotbugs...
- execution time for javac
- number of network transfers (Wagon HTTP) and bandwidth....
- time to write to disk artifacts (to local repository, in 'install',
to "target"....)

Enrico

Il giorno dom 29 mar 2020 alle ore 10:40 Romain Manni-Bucau
<rm...@gmail.com> ha scritto:
>
> It depends how it is done.
>
> @Inject MetricsSystem ms; will likely fail but if you inject the container
> and do the lookup you can handle the absence of the metrics, less elegant
> but working as well.
>
> Alternative is to inject a custom compiler component and have 2 components
> impl (hints being different), one with injected metrics, one with noop
> metrics hardcoded.
> Depending the current maven version you select the right hint.
> Think it is more or less what we do to handle aether change in some earlier
> maven version.
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>
>
> Le dim. 29 mars 2020 à 10:33, Enrico Olivelli <eo...@gmail.com> a
> écrit :
>
> > Self answer: I had written the DefaultMetricsSystem file in the wrong
> > directory ! (in maven core tests!)
> > Sorry fo the noise
> >
> > I have just one problem.
> > Currently I am using Jsr300 to plug the MetricsSystem to Maven Core
> > and this would be the expected way for Plugins.
> >
> > Say now that I want to use my MetricsSystem bean (@Inject
> > MetricsSystem) in the Maven Compiler Plugin or in Wagon...
> > let's say we do a 3.8.2 Maven Compiler plugin and that we deliver this
> > MetricsSystem in Maven 3.7.0...
> > Will Maven compiler plugin 3.8.2 work on Maven 3.6.x that does not
> > bundle the definition of the MetricsSytem ?
> >
> > Ideally I would like to have a "null" value or a Default NOOP
> > implementation when the plugin is not running on Maven 3.7.0
> > Will it work ?
> >
> > I will continue with the POC then I will share my model.
> > The change to Maven Core is almost there.
> > Now my plan is:
> > - Create an independent "Maven Extension" (in my github space, non ASF
> > by now, to make it clear that this system is totally pluggable with
> > extensions) that wraps Dropwizard Metrics and outputs the results to
> > file at the end of the build
> > - Add some initial metrics in Maven runtime
> > - Try with some plugin
> >
> >
> >
> >
> > Enrico
> >
> > Il giorno sab 28 mar 2020 alle ore 15:57 Enrico Olivelli
> > <eo...@gmail.com> ha scritto:
> > >
> > > Hi,
> > > I have pushed a new version of my prof-of-concept
> > > https://github.com/apache/maven-studies/compare/maven-metrics?expand=1
> > >
> > > I am introducing a MetricsSystem interface and a DefaultMetricsSystem
> > CDI bean.
> > > I am new to sisu/plexsus container.
> > > It looks like that DefaultMetricsSystem is getting instantiated in
> > > Maven Core tests
> > >
> > > This is my work:
> > > https://github.com/apache/maven-studies/compare/maven-metrics?expand=1
> > >
> > > But if I run Maven from the command line I get the error below
> > >
> > > Maybe I missing some config/annotation
> > >
> > > Any help is appreciated !
> > >
> > > Enrico
> > >
> > >
> > > eolivelli@LAPTOP-OODVV59Q apache-maven-3.7.0-SNAPSHOT]$ bin/mvn
> > > [WARNING] Error injecting: org.apache.maven.DefaultMaven
> > > com.google.inject.ProvisionException: Unable to provision, see the
> > > following errors:
> > >
> > > 1) No implementation for org.apache.maven.metrics.MetricsSystem was
> > bound.
> > >   at ClassRealm[plexus.core, parent: null] (via modules:
> > > org.eclipse.sisu.wire.WireModule ->
> > > org.eclipse.sisu.plexus.PlexusBindingModule)
> > >   while locating org.apache.maven.lifecycle.internal.LifecycleStarter
> > >   while locating org.apache.maven.DefaultMaven
> > >
> > > 1 error
> > >     at
> > com.google.inject.internal.InternalProvisionException.toProvisionException
> > > (InternalProvisionException.java:226)
> > >     at com.google.inject.internal.InjectorImpl$1.get
> > (InjectorImpl.java:1053)
> > >     at com.google.inject.internal.InjectorImpl.getInstance
> > > (InjectorImpl.java:1086)
> > >     at org.eclipse.sisu.space.AbstractDeferredClass.get
> > > (AbstractDeferredClass.java:48)
> > >     at com.google.inject.internal.ProviderInternalFactory.provision
> > > (ProviderInternalFactory.java:85)
> > >     at
> > com.google.inject.internal.InternalFactoryToInitializableAdapter.provision
> > > (InternalFactoryToInitializableAdapter.java:57)
> > >     at com.google.inject.internal.ProviderInternalFactory$1.call
> > > (ProviderInternalFactory.java:66)
> > >     at
> > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision
> > > (ProvisionListenerStackCallback.java:112)
> > >     at org.eclipse.sisu.bean.BeanScheduler$CycleActivator.onProvision
> > > (BeanScheduler.java:230)
> > >     at
> > com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision
> > > (ProvisionListenerStackCallback.java:120)
> > >     at
> > com.google.inject.internal.ProvisionListenerStackCallback.provision
> > > (ProvisionListenerStackCallback.java:66)
> > >     at com.google.inject.internal.ProviderInternalFactory.circularGet
> > > (ProviderInternalFactory.java:61)
> > >     at
> > com.google.inject.internal.InternalFactoryToInitializableAdapter.get
> > > (InternalFactoryToInitializableAdapter.java:47)
> > >     at com.google.inject.internal.ProviderToInternalFactoryAdapter.get
> > > (ProviderToInternalFactoryAdapter.java:40)
> > >     at com.google.inject.internal.SingletonScope$1.get
> > (SingletonScope.java:168)
> > >     at com.google.inject.internal.InternalFactoryToProviderAdapter.get
> > > (InternalFactoryToProviderAdapter.java:39)
> > >     at com.google.inject.internal.InjectorImpl$1.get
> > (InjectorImpl.java:1050)
> > >     at org.eclipse.sisu.inject.LazyBeanEntry.getValue
> > (LazyBeanEntry.java:81)
> > >     at org.eclipse.sisu.plexus.LazyPlexusBean.getValue
> > (LazyPlexusBean.java:51)
> > >     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> > > (DefaultPlexusContainer.java:263)
> > >     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> > > (DefaultPlexusContainer.java:255)
> > >     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> > > (DefaultPlexusContainer.java:249)
> > >     at org.apache.maven.cli.MavenCli.container (MavenCli.java:696)
> > >     at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:284)
> > >     at org.apache.maven.cli.MavenCli.main (MavenCli.java:195)
> > >     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native
> > Method)
> > >     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> > > (NativeMethodAccessorImpl.java:62)
> > >     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> > > (DelegatingMethodAccessorImpl.java:43)
> > >     at java.lang.reflect.Method.invoke (Method.java:564)
> > >     at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> > > (Launcher.java:282)
> > >     at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> > > (Launcher.java:225)
> > >     at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> > > (Launcher.java:406)
> > >     at org.codehaus.plexus.classworlds.launcher.Launcher.main
> > > (Launcher.java:347)
> > > [ERROR] Error executing Maven.
> > > [ERROR] com.google.inject.ProvisionException: Unable to provision, see
> > > the following errors:
> > >
> > > 1) No implementation for org.apache.maven.metrics.MetricsSystem was
> > bound.
> > >   at ClassRealm[plexus.core, parent: null] (via modules:
> > > org.eclipse.sisu.wire.WireModule ->
> > > org.eclipse.sisu.plexus.PlexusBindingModule)
> > >   while locating org.apache.maven.lifecycle.internal.LifecycleStarter
> > >   while locating org.apache.maven.DefaultMaven
> > >   at ClassRealm[plexus.core, parent: null] (via modules:
> > > org.eclipse.sisu.wire.WireModule ->
> > > org.eclipse.sisu.plexus.PlexusBindingModule)
> > >   while locating org.apache.maven.Maven
> > >
> > > 1 error
> > >       role: org.apache.maven.Maven
> > >
> > > Il giorno mar 17 mar 2020 alle ore 08:35 Hervé BOUTEMY
> > > <he...@free.fr> ha scritto:
> > > >
> > > > Le samedi 14 mars 2020, 15:14:17 CET Enrico Olivelli a écrit :
> > > > > I am starting this work.
> > > > > I have pushed a copy of current maven core master to maven-studies.
> > > > nice
> > > >
> > > > [...]
> > > > > I don't know how classloading works very well in the case of
> > Maven/Plexus.
> > > > your approach looks reasonable: you're adding extensions to the "core"
> > > > classloader [1]
> > > >
> > > > Regards,
> > > >
> > > > Hervé
> > > >
> > > > [1] https://maven.apache.org/guides/mini/guide-maven-classloading.html
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org

Re: Maven Runtime Metrics

Posted by Romain Manni-Bucau <rm...@gmail.com>.

It depends how it is done.

@Inject MetricsSystem ms; will likely fail but if you inject the container
and do the lookup you can handle the absence of the metrics, less elegant
but working as well.

Alternative is to inject a custom compiler component and have 2 components
impl (hints being different), one with injected metrics, one with noop
metrics hardcoded.
Depending the current maven version you select the right hint.
Think it is more or less what we do to handle aether change in some earlier
maven version.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le dim. 29 mars 2020 à 10:33, Enrico Olivelli <eo...@gmail.com> a
écrit :

> Self answer: I had written the DefaultMetricsSystem file in the wrong
> directory ! (in maven core tests!)
> Sorry fo the noise
>
> I have just one problem.
> Currently I am using Jsr300 to plug the MetricsSystem to Maven Core
> and this would be the expected way for Plugins.
>
> Say now that I want to use my MetricsSystem bean (@Inject
> MetricsSystem) in the Maven Compiler Plugin or in Wagon...
> let's say we do a 3.8.2 Maven Compiler plugin and that we deliver this
> MetricsSystem in Maven 3.7.0...
> Will Maven compiler plugin 3.8.2 work on Maven 3.6.x that does not
> bundle the definition of the MetricsSytem ?
>
> Ideally I would like to have a "null" value or a Default NOOP
> implementation when the plugin is not running on Maven 3.7.0
> Will it work ?
>
> I will continue with the POC then I will share my model.
> The change to Maven Core is almost there.
> Now my plan is:
> - Create an independent "Maven Extension" (in my github space, non ASF
> by now, to make it clear that this system is totally pluggable with
> extensions) that wraps Dropwizard Metrics and outputs the results to
> file at the end of the build
> - Add some initial metrics in Maven runtime
> - Try with some plugin
>
>
>
>
> Enrico
>
> Il giorno sab 28 mar 2020 alle ore 15:57 Enrico Olivelli
> <eo...@gmail.com> ha scritto:
> >
> > Hi,
> > I have pushed a new version of my prof-of-concept
> > https://github.com/apache/maven-studies/compare/maven-metrics?expand=1
> >
> > I am introducing a MetricsSystem interface and a DefaultMetricsSystem
> CDI bean.
> > I am new to sisu/plexsus container.
> > It looks like that DefaultMetricsSystem is getting instantiated in
> > Maven Core tests
> >
> > This is my work:
> > https://github.com/apache/maven-studies/compare/maven-metrics?expand=1
> >
> > But if I run Maven from the command line I get the error below
> >
> > Maybe I missing some config/annotation
> >
> > Any help is appreciated !
> >
> > Enrico
> >
> >
> > eolivelli@LAPTOP-OODVV59Q apache-maven-3.7.0-SNAPSHOT]$ bin/mvn
> > [WARNING] Error injecting: org.apache.maven.DefaultMaven
> > com.google.inject.ProvisionException: Unable to provision, see the
> > following errors:
> >
> > 1) No implementation for org.apache.maven.metrics.MetricsSystem was
> bound.
> >   at ClassRealm[plexus.core, parent: null] (via modules:
> > org.eclipse.sisu.wire.WireModule ->
> > org.eclipse.sisu.plexus.PlexusBindingModule)
> >   while locating org.apache.maven.lifecycle.internal.LifecycleStarter
> >   while locating org.apache.maven.DefaultMaven
> >
> > 1 error
> >     at
> com.google.inject.internal.InternalProvisionException.toProvisionException
> > (InternalProvisionException.java:226)
> >     at com.google.inject.internal.InjectorImpl$1.get
> (InjectorImpl.java:1053)
> >     at com.google.inject.internal.InjectorImpl.getInstance
> > (InjectorImpl.java:1086)
> >     at org.eclipse.sisu.space.AbstractDeferredClass.get
> > (AbstractDeferredClass.java:48)
> >     at com.google.inject.internal.ProviderInternalFactory.provision
> > (ProviderInternalFactory.java:85)
> >     at
> com.google.inject.internal.InternalFactoryToInitializableAdapter.provision
> > (InternalFactoryToInitializableAdapter.java:57)
> >     at com.google.inject.internal.ProviderInternalFactory$1.call
> > (ProviderInternalFactory.java:66)
> >     at
> com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision
> > (ProvisionListenerStackCallback.java:112)
> >     at org.eclipse.sisu.bean.BeanScheduler$CycleActivator.onProvision
> > (BeanScheduler.java:230)
> >     at
> com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision
> > (ProvisionListenerStackCallback.java:120)
> >     at
> com.google.inject.internal.ProvisionListenerStackCallback.provision
> > (ProvisionListenerStackCallback.java:66)
> >     at com.google.inject.internal.ProviderInternalFactory.circularGet
> > (ProviderInternalFactory.java:61)
> >     at
> com.google.inject.internal.InternalFactoryToInitializableAdapter.get
> > (InternalFactoryToInitializableAdapter.java:47)
> >     at com.google.inject.internal.ProviderToInternalFactoryAdapter.get
> > (ProviderToInternalFactoryAdapter.java:40)
> >     at com.google.inject.internal.SingletonScope$1.get
> (SingletonScope.java:168)
> >     at com.google.inject.internal.InternalFactoryToProviderAdapter.get
> > (InternalFactoryToProviderAdapter.java:39)
> >     at com.google.inject.internal.InjectorImpl$1.get
> (InjectorImpl.java:1050)
> >     at org.eclipse.sisu.inject.LazyBeanEntry.getValue
> (LazyBeanEntry.java:81)
> >     at org.eclipse.sisu.plexus.LazyPlexusBean.getValue
> (LazyPlexusBean.java:51)
> >     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> > (DefaultPlexusContainer.java:263)
> >     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> > (DefaultPlexusContainer.java:255)
> >     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> > (DefaultPlexusContainer.java:249)
> >     at org.apache.maven.cli.MavenCli.container (MavenCli.java:696)
> >     at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:284)
> >     at org.apache.maven.cli.MavenCli.main (MavenCli.java:195)
> >     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native
> Method)
> >     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> > (NativeMethodAccessorImpl.java:62)
> >     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> > (DelegatingMethodAccessorImpl.java:43)
> >     at java.lang.reflect.Method.invoke (Method.java:564)
> >     at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> > (Launcher.java:282)
> >     at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> > (Launcher.java:225)
> >     at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> > (Launcher.java:406)
> >     at org.codehaus.plexus.classworlds.launcher.Launcher.main
> > (Launcher.java:347)
> > [ERROR] Error executing Maven.
> > [ERROR] com.google.inject.ProvisionException: Unable to provision, see
> > the following errors:
> >
> > 1) No implementation for org.apache.maven.metrics.MetricsSystem was
> bound.
> >   at ClassRealm[plexus.core, parent: null] (via modules:
> > org.eclipse.sisu.wire.WireModule ->
> > org.eclipse.sisu.plexus.PlexusBindingModule)
> >   while locating org.apache.maven.lifecycle.internal.LifecycleStarter
> >   while locating org.apache.maven.DefaultMaven
> >   at ClassRealm[plexus.core, parent: null] (via modules:
> > org.eclipse.sisu.wire.WireModule ->
> > org.eclipse.sisu.plexus.PlexusBindingModule)
> >   while locating org.apache.maven.Maven
> >
> > 1 error
> >       role: org.apache.maven.Maven
> >
> > Il giorno mar 17 mar 2020 alle ore 08:35 Hervé BOUTEMY
> > <he...@free.fr> ha scritto:
> > >
> > > Le samedi 14 mars 2020, 15:14:17 CET Enrico Olivelli a écrit :
> > > > I am starting this work.
> > > > I have pushed a copy of current maven core master to maven-studies.
> > > nice
> > >
> > > [...]
> > > > I don't know how classloading works very well in the case of
> Maven/Plexus.
> > > your approach looks reasonable: you're adding extensions to the "core"
> > > classloader [1]
> > >
> > > Regards,
> > >
> > > Hervé
> > >
> > > [1] https://maven.apache.org/guides/mini/guide-maven-classloading.html
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>

Re: Maven Runtime Metrics

Posted by Enrico Olivelli <eo...@gmail.com>.

Self answer: I had written the DefaultMetricsSystem file in the wrong
directory ! (in maven core tests!)
Sorry fo the noise

I have just one problem.
Currently I am using Jsr300 to plug the MetricsSystem to Maven Core
and this would be the expected way for Plugins.

Say now that I want to use my MetricsSystem bean (@Inject
MetricsSystem) in the Maven Compiler Plugin or in Wagon...
let's say we do a 3.8.2 Maven Compiler plugin and that we deliver this
MetricsSystem in Maven 3.7.0...
Will Maven compiler plugin 3.8.2 work on Maven 3.6.x that does not
bundle the definition of the MetricsSytem ?

Ideally I would like to have a "null" value or a Default NOOP
implementation when the plugin is not running on Maven 3.7.0
Will it work ?

I will continue with the POC then I will share my model.
The change to Maven Core is almost there.
Now my plan is:
- Create an independent "Maven Extension" (in my github space, non ASF
by now, to make it clear that this system is totally pluggable with
extensions) that wraps Dropwizard Metrics and outputs the results to
file at the end of the build
- Add some initial metrics in Maven runtime
- Try with some plugin




Enrico

Il giorno sab 28 mar 2020 alle ore 15:57 Enrico Olivelli
<eo...@gmail.com> ha scritto:
>
> Hi,
> I have pushed a new version of my prof-of-concept
> https://github.com/apache/maven-studies/compare/maven-metrics?expand=1
>
> I am introducing a MetricsSystem interface and a DefaultMetricsSystem CDI bean.
> I am new to sisu/plexsus container.
> It looks like that DefaultMetricsSystem is getting instantiated in
> Maven Core tests
>
> This is my work:
> https://github.com/apache/maven-studies/compare/maven-metrics?expand=1
>
> But if I run Maven from the command line I get the error below
>
> Maybe I missing some config/annotation
>
> Any help is appreciated !
>
> Enrico
>
>
> eolivelli@LAPTOP-OODVV59Q apache-maven-3.7.0-SNAPSHOT]$ bin/mvn
> [WARNING] Error injecting: org.apache.maven.DefaultMaven
> com.google.inject.ProvisionException: Unable to provision, see the
> following errors:
>
> 1) No implementation for org.apache.maven.metrics.MetricsSystem was bound.
>   at ClassRealm[plexus.core, parent: null] (via modules:
> org.eclipse.sisu.wire.WireModule ->
> org.eclipse.sisu.plexus.PlexusBindingModule)
>   while locating org.apache.maven.lifecycle.internal.LifecycleStarter
>   while locating org.apache.maven.DefaultMaven
>
> 1 error
>     at com.google.inject.internal.InternalProvisionException.toProvisionException
> (InternalProvisionException.java:226)
>     at com.google.inject.internal.InjectorImpl$1.get (InjectorImpl.java:1053)
>     at com.google.inject.internal.InjectorImpl.getInstance
> (InjectorImpl.java:1086)
>     at org.eclipse.sisu.space.AbstractDeferredClass.get
> (AbstractDeferredClass.java:48)
>     at com.google.inject.internal.ProviderInternalFactory.provision
> (ProviderInternalFactory.java:85)
>     at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision
> (InternalFactoryToInitializableAdapter.java:57)
>     at com.google.inject.internal.ProviderInternalFactory$1.call
> (ProviderInternalFactory.java:66)
>     at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision
> (ProvisionListenerStackCallback.java:112)
>     at org.eclipse.sisu.bean.BeanScheduler$CycleActivator.onProvision
> (BeanScheduler.java:230)
>     at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision
> (ProvisionListenerStackCallback.java:120)
>     at com.google.inject.internal.ProvisionListenerStackCallback.provision
> (ProvisionListenerStackCallback.java:66)
>     at com.google.inject.internal.ProviderInternalFactory.circularGet
> (ProviderInternalFactory.java:61)
>     at com.google.inject.internal.InternalFactoryToInitializableAdapter.get
> (InternalFactoryToInitializableAdapter.java:47)
>     at com.google.inject.internal.ProviderToInternalFactoryAdapter.get
> (ProviderToInternalFactoryAdapter.java:40)
>     at com.google.inject.internal.SingletonScope$1.get (SingletonScope.java:168)
>     at com.google.inject.internal.InternalFactoryToProviderAdapter.get
> (InternalFactoryToProviderAdapter.java:39)
>     at com.google.inject.internal.InjectorImpl$1.get (InjectorImpl.java:1050)
>     at org.eclipse.sisu.inject.LazyBeanEntry.getValue (LazyBeanEntry.java:81)
>     at org.eclipse.sisu.plexus.LazyPlexusBean.getValue (LazyPlexusBean.java:51)
>     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> (DefaultPlexusContainer.java:263)
>     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> (DefaultPlexusContainer.java:255)
>     at org.codehaus.plexus.DefaultPlexusContainer.lookup
> (DefaultPlexusContainer.java:249)
>     at org.apache.maven.cli.MavenCli.container (MavenCli.java:696)
>     at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:284)
>     at org.apache.maven.cli.MavenCli.main (MavenCli.java:195)
>     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
>     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
>     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke (Method.java:564)
>     at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> (Launcher.java:282)
>     at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> (Launcher.java:225)
>     at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> (Launcher.java:406)
>     at org.codehaus.plexus.classworlds.launcher.Launcher.main
> (Launcher.java:347)
> [ERROR] Error executing Maven.
> [ERROR] com.google.inject.ProvisionException: Unable to provision, see
> the following errors:
>
> 1) No implementation for org.apache.maven.metrics.MetricsSystem was bound.
>   at ClassRealm[plexus.core, parent: null] (via modules:
> org.eclipse.sisu.wire.WireModule ->
> org.eclipse.sisu.plexus.PlexusBindingModule)
>   while locating org.apache.maven.lifecycle.internal.LifecycleStarter
>   while locating org.apache.maven.DefaultMaven
>   at ClassRealm[plexus.core, parent: null] (via modules:
> org.eclipse.sisu.wire.WireModule ->
> org.eclipse.sisu.plexus.PlexusBindingModule)
>   while locating org.apache.maven.Maven
>
> 1 error
>       role: org.apache.maven.Maven
>
> Il giorno mar 17 mar 2020 alle ore 08:35 Hervé BOUTEMY
> <he...@free.fr> ha scritto:
> >
> > Le samedi 14 mars 2020, 15:14:17 CET Enrico Olivelli a écrit :
> > > I am starting this work.
> > > I have pushed a copy of current maven core master to maven-studies.
> > nice
> >
> > [...]
> > > I don't know how classloading works very well in the case of Maven/Plexus.
> > your approach looks reasonable: you're adding extensions to the "core"
> > classloader [1]
> >
> > Regards,
> >
> > Hervé
> >
> > [1] https://maven.apache.org/guides/mini/guide-maven-classloading.html
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org

Re: Maven Runtime Metrics

Posted by Enrico Olivelli <eo...@gmail.com>.

Hi,
I have pushed a new version of my prof-of-concept
https://github.com/apache/maven-studies/compare/maven-metrics?expand=1

I am introducing a MetricsSystem interface and a DefaultMetricsSystem CDI bean.
I am new to sisu/plexsus container.
It looks like that DefaultMetricsSystem is getting instantiated in
Maven Core tests

This is my work:
https://github.com/apache/maven-studies/compare/maven-metrics?expand=1

But if I run Maven from the command line I get the error below

Maybe I missing some config/annotation

Any help is appreciated !

Enrico


eolivelli@LAPTOP-OODVV59Q apache-maven-3.7.0-SNAPSHOT]$ bin/mvn
[WARNING] Error injecting: org.apache.maven.DefaultMaven
com.google.inject.ProvisionException: Unable to provision, see the
following errors:

1) No implementation for org.apache.maven.metrics.MetricsSystem was bound.
  at ClassRealm[plexus.core, parent: null] (via modules:
org.eclipse.sisu.wire.WireModule ->
org.eclipse.sisu.plexus.PlexusBindingModule)
  while locating org.apache.maven.lifecycle.internal.LifecycleStarter
  while locating org.apache.maven.DefaultMaven

1 error
    at com.google.inject.internal.InternalProvisionException.toProvisionException
(InternalProvisionException.java:226)
    at com.google.inject.internal.InjectorImpl$1.get (InjectorImpl.java:1053)
    at com.google.inject.internal.InjectorImpl.getInstance
(InjectorImpl.java:1086)
    at org.eclipse.sisu.space.AbstractDeferredClass.get
(AbstractDeferredClass.java:48)
    at com.google.inject.internal.ProviderInternalFactory.provision
(ProviderInternalFactory.java:85)
    at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision
(InternalFactoryToInitializableAdapter.java:57)
    at com.google.inject.internal.ProviderInternalFactory$1.call
(ProviderInternalFactory.java:66)
    at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision
(ProvisionListenerStackCallback.java:112)
    at org.eclipse.sisu.bean.BeanScheduler$CycleActivator.onProvision
(BeanScheduler.java:230)
    at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision
(ProvisionListenerStackCallback.java:120)
    at com.google.inject.internal.ProvisionListenerStackCallback.provision
(ProvisionListenerStackCallback.java:66)
    at com.google.inject.internal.ProviderInternalFactory.circularGet
(ProviderInternalFactory.java:61)
    at com.google.inject.internal.InternalFactoryToInitializableAdapter.get
(InternalFactoryToInitializableAdapter.java:47)
    at com.google.inject.internal.ProviderToInternalFactoryAdapter.get
(ProviderToInternalFactoryAdapter.java:40)
    at com.google.inject.internal.SingletonScope$1.get (SingletonScope.java:168)
    at com.google.inject.internal.InternalFactoryToProviderAdapter.get
(InternalFactoryToProviderAdapter.java:39)
    at com.google.inject.internal.InjectorImpl$1.get (InjectorImpl.java:1050)
    at org.eclipse.sisu.inject.LazyBeanEntry.getValue (LazyBeanEntry.java:81)
    at org.eclipse.sisu.plexus.LazyPlexusBean.getValue (LazyPlexusBean.java:51)
    at org.codehaus.plexus.DefaultPlexusContainer.lookup
(DefaultPlexusContainer.java:263)
    at org.codehaus.plexus.DefaultPlexusContainer.lookup
(DefaultPlexusContainer.java:255)
    at org.codehaus.plexus.DefaultPlexusContainer.lookup
(DefaultPlexusContainer.java:249)
    at org.apache.maven.cli.MavenCli.container (MavenCli.java:696)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:284)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:195)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:564)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
(Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch
(Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
(Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main
(Launcher.java:347)
[ERROR] Error executing Maven.
[ERROR] com.google.inject.ProvisionException: Unable to provision, see
the following errors:

1) No implementation for org.apache.maven.metrics.MetricsSystem was bound.
  at ClassRealm[plexus.core, parent: null] (via modules:
org.eclipse.sisu.wire.WireModule ->
org.eclipse.sisu.plexus.PlexusBindingModule)
  while locating org.apache.maven.lifecycle.internal.LifecycleStarter
  while locating org.apache.maven.DefaultMaven
  at ClassRealm[plexus.core, parent: null] (via modules:
org.eclipse.sisu.wire.WireModule ->
org.eclipse.sisu.plexus.PlexusBindingModule)
  while locating org.apache.maven.Maven

1 error
      role: org.apache.maven.Maven

Il giorno mar 17 mar 2020 alle ore 08:35 Hervé BOUTEMY
<he...@free.fr> ha scritto:
>
> Le samedi 14 mars 2020, 15:14:17 CET Enrico Olivelli a écrit :
> > I am starting this work.
> > I have pushed a copy of current maven core master to maven-studies.
> nice
>
> [...]
> > I don't know how classloading works very well in the case of Maven/Plexus.
> your approach looks reasonable: you're adding extensions to the "core"
> classloader [1]
>
> Regards,
>
> Hervé
>
> [1] https://maven.apache.org/guides/mini/guide-maven-classloading.html
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org

Re: Maven Runtime Metrics

Posted by Hervé BOUTEMY <he...@free.fr>.

Le samedi 14 mars 2020, 15:14:17 CET Enrico Olivelli a écrit :
> I am starting this work.
> I have pushed a copy of current maven core master to maven-studies.
nice

[...]
> I don't know how classloading works very well in the case of Maven/Plexus.
your approach looks reasonable: you're adding extensions to the "core" 
classloader [1]

Regards,

Hervé

[1] https://maven.apache.org/guides/mini/guide-maven-classloading.html



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org

Re: Maven Runtime Metrics

Posted by Romain Manni-Bucau <rm...@gmail.com>.

What's wrong with the IoC?

In a plugin you can do:

 @Inject @Named("...") // Or other equivalent annotations
Metrics metrics;

Then you can pass the reference if you want.

Keep in mind static ref in scoped instances, even loggers, are antipatterns
only justified by perf in early times. How would you distinguish two
plugins with a static instance? How do you reset it between two goals
executions keeping maven up as in mavensh or equivalents? Container (IoC)
gives you this control and enbles to keep the consistency of underlying
data. Alternative is to rebuild an IoC behind static vars which should not
the best option to me.

Happy to help on slack next week, just ping me.

Le sam. 14 mars 2020 à 20:00, Enrico Olivelli <eo...@gmail.com> a
écrit :

> Il Sab 14 Mar 2020, 19:27 Romain Manni-Bucau <rm...@gmail.com> ha
> scritto:
>
> > Hmm, not sure it is a good example, plugins do play IoC game already and
> is
> > the cleanest way to move forward otherwise you will leak the counter or
> > need to play with thread locals which is worse for plugins using this
> > pattern (will forbid plugins to use threads).
> >
> > That said you can still replace all this code by anything you want, even
> an
> > IoC lookup and call using a classfiletransformer in ClassRealm plugin
> > classloader. It is just a way to rewrite original code to something else.
> >
>
> I am not used to IoC.
> I have to spend more time in playing with it.
> If you have some spare cycles to help me maybe I could start quicker.
> Maybe we can switch to slack for these stuff.
>
> If we give the flexibility of a Logger and we are able to respect Maven IoC
> container I feel we will do the best so I am totally +1 on it
>
> Enrico
>
>
>
> > Le sam. 14 mars 2020 à 17:59, Enrico Olivelli <eo...@gmail.com> a
> > écrit :
> >
> > > Il giorno sab 14 mar 2020 alle ore 17:53 Romain Manni-Bucau
> > > <rm...@gmail.com> ha scritto:
> > > >
> > > > Assuming maven core/boot is components, a component registry can
> > > instrument
> > > > plugins and subclassloaders with classfiletransformers so maybe ok to
> > > keep
> > > > it in the IoC.
> > >
> > > Romain,
> > > can please you elaborate more ?
> > >
> > > I am thinking this this case:
> > >
> > > class MyCoolCache {
> > >     private static final Counter counter =
> > > MetricsSystem.getCounter("cacheHits");
> > >     public T get(K key) {
> > >   ....    counter.increment();
> > >   }
> > > }
> > >
> > > I am a developer of a plugin and I want to set some metric to see the
> > > runtime behaviour of my component, I don't want my class to play the
> > > IoC game
> > >
> > > You can see it like a "Logger". Loggers are not subject to IoC because
> > > usually you may want a Logger absolutely everywhere and without
> > > changing/refactoring your code.
> > > If I want to add a "temporary" metric I don't want to need to change
> > > constructors, add getters/setters....
> > >
> > >
> > > Enrico
> > >
> > > >
> > > > Le sam. 14 mars 2020 à 15:14, Enrico Olivelli <eo...@gmail.com>
> a
> > > > écrit :
> > > >
> > > > > I am starting this work.
> > > > > I have pushed a copy of current maven core master to maven-studies.
> > > > >
> > > > > Then I have added my "maven-metrics" module.
> > > > > This module contains only the API to be implemented by Metrics
> > > > > Providers and a No-op implementation.
> > > > >
> > > > > You can find the draft API here.
> > > > >
> > >
> https://github.com/apache/maven-studies/tree/maven-metrics/maven-metrics
> > > > >
> > > > > It is not final it is just the first import to git, I have to see
> > some
> > > > > real world Maven cases before designing the final one.
> > > > >
> > > > > Now I am going to instrument some Maven core subsystem and
> implement
> > a
> > > > > simple Metrics Provider in order to see some data.
> > > > >
> > > > > My very first problem is to boot the MetricsProvider and to give
> > > > > access to it from any Java class and module.
> > > > > Maven uses Dependency Injection and this is cool, but with this
> > > > > approach you will not be able to instrument non-component classes,
> > > > > like internal Caches or Network facilities.
> > > > >
> > > > > I am leaning towards using some kind of "global static reference"
> > > > > (like a pure Java Singleton) to hold a reference to the Metrics
> > > > > subsystem.
> > > > >
> > > > > By default the Metrics Provider will be the No-op implementation,
> so
> > > > > third party tools won't see NULLs.
> > > > >
> > > > > in MavenCli I will "boot" (and manage the whole lifecycle) the
> > Metrics
> > > > > Provider and set this "global reference".
> > > > >
> > > > > The Metrics Provider API will be in the main Maven classloader and
> > the
> > > > > Metrics Provider will have to be loaded in the same
> > > > > context/classloader.
> > > > > Plugins and internal classes will use the API and not directly the
> > > > > provider implementation.
> > > > > I don't know how classloading works very well in the case of
> > > Maven/Plexus.
> > > > >
> > > > > Any suggestion is very appreciated.
> > > > >
> > > > > I have very very limited time for this work, but I see value and I
> > > > > will move forward when I have cycles
> > > > >
> > > > > Cheers
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > >
> > > > > Il giorno ven 31 gen 2020 alle ore 00:27 <he...@free.fr>
> ha
> > > > > scritto:
> > > > > >
> > > > > > maven-studies looks like the right location:
> > > > > > https://github.com/apache/maven-studies
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Hervé
> > > > > >
> > > > > > ----- Mail original -----
> > > > > > De: "Enrico Olivelli" <eo...@gmail.com>
> > > > > > À: "Maven Developers List" <de...@maven.apache.org>
> > > > > > Envoyé: Jeudi 30 Janvier 2020 04:31:14
> > > > > > Objet: Re: Maven Runtime Metrics
> > > > > >
> > > > > > I sound like to draft a prototype.
> > > > > >
> > > > > > Any suggestion about which repository should contain this new
> > Metrics
> > > > > API?
> > > > > >
> > > > > >
> > > > > > Enrico
> > > > > >
> > > > > > Il Mar 28 Gen 2020, 13:54 Romain Manni-Bucau <
> > rmannibucau@gmail.com>
> > > ha
> > > > > > scritto:
> > > > > >
> > > > > > > technically we can shade microprofile, metrics/dropwizard or
> even
> > > > > sirona
> > > > > > > but maven requires quite a different model. A typical example
> is
> > > the
> > > > > expiry
> > > > > > > or weighted model will not fit the reporting maven needs (very
> > long
> > > > > running
> > > > > > > instances vs "batch" like executions) and the weighted impl can
> > be
> > > an
> > > > > issue
> > > > > > > but is spec-ed by MP (so the shaded impl would also have this
> > > > > unexpected
> > > > > > > impl)
> > > > > > >
> > > > > > > What about by just adding counters, enabling to inject them in
> > > mojos -
> > > > > > > maybe with a qualifier? - and then see if we need to move to
> > > something
> > > > > more
> > > > > > > widely used or not (but my past experiences kind of tend to
> show
> > > it is
> > > > > not
> > > > > > > needed).
> > > > > > >
> > > > > > > Romain Manni-Bucau
> > > > > > > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > > > > > > <https://rmannibucau.metawerx.net/> | Old Blog
> > > > > > > <http://rmannibucau.wordpress.com> | Github <
> > > > > > > https://github.com/rmannibucau> |
> > > > > > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > > > > > > <
> > > > > > >
> > > > >
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Le mar. 28 janv. 2020 à 13:42, Enrico Olivelli <
> > > eolivelli@gmail.com> a
> > > > > > > écrit :
> > > > > > >
> > > > > > > > Il Mar 28 Gen 2020, 13:30 Lennart Jörelid <
> > > lennart.jorelid@gmail.com>
> > > > > ha
> > > > > > > > scritto:
> > > > > > > >
> > > > > > > > > Would it be possible to use an existing standard for the
> > > > > specification
> > > > > > > of
> > > > > > > > > such a metrics API?
> > > > > > > > >
> > > > > > > >
> > > > > > > > In my opinion we should have our own model, Maven will impose
> > > this
> > > > > new
> > > > > > > API
> > > > > > > > to plugins and we cannot depend on third party tools as
> Romain
> > > also
> > > > > said
> > > > > > > >
> > > > > > > > Okay for annotations
> > > > > > > >
> > > > > > > > Enrico
> > > > > > > >
> > > > > > > > Something like a shaded version of Microprofile Metrics,
> where
> > > the
> > > > > API
> > > > > > > can
> > > > > > > > > consist entirely of Annotations.
> > > > > > > > > (There are programmatic approaches as well, but I suppose
> we
> > > are
> > > > > > > > interested
> > > > > > > > > in limiting the footprint of metrics within the Maven
> > codebase
> > > as
> > > > > much
> > > > > > > as
> > > > > > > > > possible).
> > > > > > > > >
> > > > > > > > > ... implying, of course, that we would need to make a
> > > > > Maven-specific
> > > > > > > > > implementation as discussed above.
> > > > > > > > >
> > > > > > > > > On Tue, Jan 28, 2020 at 10:17 AM Enrico Olivelli <
> > > > > eolivelli@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Il giorno mar 28 gen 2020 alle ore 09:15 Romain
> > Manni-Bucau <
> > > > > > > > > > rmannibucau@gmail.com> ha scritto:
> > > > > > > > > >
> > > > > > > > > > > +1 generally since all extensions doing that report
> wrong
> > > data
> > > > > > > > > > > -1 to use a mainstream impl if not shaded - it would
> just
> > > > > bring us
> > > > > > > > new
> > > > > > > > > > > conflicts i think.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Totally agree.
> > > > > > > > > > In my view any "impl" will be plugged with an
> "extension".
> > > > > > > > > > We will provide only a default no-op implementation
> > > > > > > > > >
> > > > > > > > > > Enrico
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Side note being: we can start we just counters so no
> lib
> > is
> > > > > fine
> > > > > > > > imho.
> > > > > > > > > > >
> > > > > > > > > > > Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <
> > > > > > > eolivelli@gmail.com>
> > > > > > > > a
> > > > > > > > > > > écrit :
> > > > > > > > > > >
> > > > > > > > > > > > Hi,
> > > > > > > > > > > > I would like to work on introducing an explicit
> support
> > > in
> > > > > Maven
> > > > > > > > Core
> > > > > > > > > > for
> > > > > > > > > > > > recording "metrics".
> > > > > > > > > > > > As far as I know (and I have still limited knowledge
> > > about
> > > > > the
> > > > > > > > > > internals)
> > > > > > > > > > > > you can write an "extension" and you can intercept
> main
> > > > > lifecycle
> > > > > > > > > > events,
> > > > > > > > > > > > and using these hooks you can try to record metrics.
> > > > > > > > > > > > But my proposal is from another point of view and I
> > want
> > > to
> > > > > go
> > > > > > > > > deeper.
> > > > > > > > > > > >
> > > > > > > > > > > > We are adding caches, making refactors....all good
> > > things,
> > > > > but we
> > > > > > > > > > should
> > > > > > > > > > > > have a way to track how Maven is behaving in a
> > measurable
> > > > > manner.
> > > > > > > > > > > >
> > > > > > > > > > > > Usually in order to capture realtime metrics you are
> > > using
> > > > > some
> > > > > > > > > Metrics
> > > > > > > > > > > > framework, like Prometheus, Dropwizards....and you
> > attach
> > > > > your
> > > > > > > > > process
> > > > > > > > > > > to a
> > > > > > > > > > > > metrics database and capture all of the values.
> > > > > > > > > > > > Then you can process all of the data, realtime and/or
> > > > > offline,
> > > > > > > you
> > > > > > > > > can
> > > > > > > > > > > > create dashboards, you can also fine tune your
> > > configuration
> > > > > or
> > > > > > > > > change
> > > > > > > > > > > your
> > > > > > > > > > > > code.
> > > > > > > > > > > >
> > > > > > > > > > > > These frameworks deal with time series created by
> > > capturing
> > > > > > > > > Counters,
> > > > > > > > > > > > Summaries, Gauges.
> > > > > > > > > > > >
> > > > > > > > > > > > I would like to introduce an API to be used by Maven
> > Core
> > > > > and by
> > > > > > > > > Plugin
> > > > > > > > > > > to
> > > > > > > > > > > > expose internal metrics, like:
> > > > > > > > > > > > - internal caches size
> > > > > > > > > > > > - memory usage
> > > > > > > > > > > > - disk reads/writes
> > > > > > > > > > > > - network traffic
> > > > > > > > > > > > - any kind of specific metric for plugins (like
> "number
> > > of
> > > > > tests
> > > > > > > > > > > executed",
> > > > > > > > > > > > "number of files sent to javac...")
> > > > > > > > > > > > - .... all of the usual stuff you track in order to
> > > > > understand
> > > > > > > > wha's
> > > > > > > > > > > going
> > > > > > > > > > > > on
> > > > > > > > > > > >
> > > > > > > > > > > > I am saying a Maven Metrics API because we must not
> > > stick to
> > > > > a
> > > > > > > > single
> > > > > > > > > > > > "provider" (say "DropWizard"), it will be a bad
> choice
> > > in the
> > > > > > > long
> > > > > > > > > > term.
> > > > > > > > > > > >
> > > > > > > > > > > > The API will expose to Maven Core and to plugins a
> way
> > > to get
> > > > > > > > access
> > > > > > > > > to
> > > > > > > > > > > > Metrics and report useful values.
> > > > > > > > > > > >
> > > > > > > > > > > > @InjectMe
> > > > > > > > > > > > MetricsProvider metrics;
> > > > > > > > > > > >
> > > > > > > > > > > > metrics.getCounter("javacFiles").add(....)
> > > > > > > > > > > > metrics.getSummary("downloadTime").add(....)
> > > > > > > > > > > >
> > > > > > > > > > > > I don't want to go down into details as this is only
> a
> > > > > > > preliminary
> > > > > > > > > > email
> > > > > > > > > > > > but in my view we could let the user attach an
> > > > > "implementation"
> > > > > > > > just
> > > > > > > > > by
> > > > > > > > > > > > dropping in an "extension", in order not to ship with
> > > Maven
> > > > > Core
> > > > > > > a
> > > > > > > > > lot
> > > > > > > > > > of
> > > > > > > > > > > > third party dependencies.
> > > > > > > > > > > >
> > > > > > > > > > > > I have some "production" experience with this topic
> > > > > (actually I
> > > > > > > did
> > > > > > > > > > > > something like that in my company and in ZooKeeper
> > > project)
> > > > > so I
> > > > > > > > will
> > > > > > > > > > be
> > > > > > > > > > > > happy to prepare and share a design document.
> > > > > > > > > > > >
> > > > > > > > > > > > With this work we will open up the way to knowing
> how a
> > > Maven
> > > > > > > build
> > > > > > > > > > > works,
> > > > > > > > > > > > with the 'time dimension', on the local machine of
> the
> > > > > developer
> > > > > > > > but
> > > > > > > > > > even
> > > > > > > > > > > > on CI servers.
> > > > > > > > > > > >
> > > > > > > > > > > > Any comment is very welcome
> > > > > > > > > > > >
> > > > > > > > > > > > Regards
> > > > > > > > > > > > Enrico
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > +==============================+
> > > > > > > > > | Bästa hälsningar,
> > > > > > > > > | [sw. "Best regards"]
> > > > > > > > > |
> > > > > > > > > | Lennart Jörelid
> > > > > > > > > | EAI Architect & Integrator
> > > > > > > > > |
> > > > > > > > > | jGuru Europe AB
> > > > > > > > > | Mölnlycke - Kista
> > > > > > > > > |
> > > > > > > > > | Email: lj@jguru.se
> > > > > > > > > | URL:   www.jguru.se
> > > > > > > > > | Phone
> > > > > > > > > | (skype):    jgurueurope
> > > > > > > > > | (intl):     +46 708 507 603
> > > > > > > > > | (domestic): 0708 - 507 603
> > > > > > > > > +==============================+
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > >
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> >
>

Re: Maven Runtime Metrics

Posted by Enrico Olivelli <eo...@gmail.com>.

Il Sab 14 Mar 2020, 19:27 Romain Manni-Bucau <rm...@gmail.com> ha
scritto:

> Hmm, not sure it is a good example, plugins do play IoC game already and is
> the cleanest way to move forward otherwise you will leak the counter or
> need to play with thread locals which is worse for plugins using this
> pattern (will forbid plugins to use threads).
>
> That said you can still replace all this code by anything you want, even an
> IoC lookup and call using a classfiletransformer in ClassRealm plugin
> classloader. It is just a way to rewrite original code to something else.
>

I am not used to IoC.
I have to spend more time in playing with it.
If you have some spare cycles to help me maybe I could start quicker.
Maybe we can switch to slack for these stuff.

If we give the flexibility of a Logger and we are able to respect Maven IoC
container I feel we will do the best so I am totally +1 on it

Enrico



> Le sam. 14 mars 2020 à 17:59, Enrico Olivelli <eo...@gmail.com> a
> écrit :
>
> > Il giorno sab 14 mar 2020 alle ore 17:53 Romain Manni-Bucau
> > <rm...@gmail.com> ha scritto:
> > >
> > > Assuming maven core/boot is components, a component registry can
> > instrument
> > > plugins and subclassloaders with classfiletransformers so maybe ok to
> > keep
> > > it in the IoC.
> >
> > Romain,
> > can please you elaborate more ?
> >
> > I am thinking this this case:
> >
> > class MyCoolCache {
> >     private static final Counter counter =
> > MetricsSystem.getCounter("cacheHits");
> >     public T get(K key) {
> >   ....    counter.increment();
> >   }
> > }
> >
> > I am a developer of a plugin and I want to set some metric to see the
> > runtime behaviour of my component, I don't want my class to play the
> > IoC game
> >
> > You can see it like a "Logger". Loggers are not subject to IoC because
> > usually you may want a Logger absolutely everywhere and without
> > changing/refactoring your code.
> > If I want to add a "temporary" metric I don't want to need to change
> > constructors, add getters/setters....
> >
> >
> > Enrico
> >
> > >
> > > Le sam. 14 mars 2020 à 15:14, Enrico Olivelli <eo...@gmail.com> a
> > > écrit :
> > >
> > > > I am starting this work.
> > > > I have pushed a copy of current maven core master to maven-studies.
> > > >
> > > > Then I have added my "maven-metrics" module.
> > > > This module contains only the API to be implemented by Metrics
> > > > Providers and a No-op implementation.
> > > >
> > > > You can find the draft API here.
> > > >
> > https://github.com/apache/maven-studies/tree/maven-metrics/maven-metrics
> > > >
> > > > It is not final it is just the first import to git, I have to see
> some
> > > > real world Maven cases before designing the final one.
> > > >
> > > > Now I am going to instrument some Maven core subsystem and implement
> a
> > > > simple Metrics Provider in order to see some data.
> > > >
> > > > My very first problem is to boot the MetricsProvider and to give
> > > > access to it from any Java class and module.
> > > > Maven uses Dependency Injection and this is cool, but with this
> > > > approach you will not be able to instrument non-component classes,
> > > > like internal Caches or Network facilities.
> > > >
> > > > I am leaning towards using some kind of "global static reference"
> > > > (like a pure Java Singleton) to hold a reference to the Metrics
> > > > subsystem.
> > > >
> > > > By default the Metrics Provider will be the No-op implementation, so
> > > > third party tools won't see NULLs.
> > > >
> > > > in MavenCli I will "boot" (and manage the whole lifecycle) the
> Metrics
> > > > Provider and set this "global reference".
> > > >
> > > > The Metrics Provider API will be in the main Maven classloader and
> the
> > > > Metrics Provider will have to be loaded in the same
> > > > context/classloader.
> > > > Plugins and internal classes will use the API and not directly the
> > > > provider implementation.
> > > > I don't know how classloading works very well in the case of
> > Maven/Plexus.
> > > >
> > > > Any suggestion is very appreciated.
> > > >
> > > > I have very very limited time for this work, but I see value and I
> > > > will move forward when I have cycles
> > > >
> > > > Cheers
> > > >
> > > > Enrico
> > > >
> > > >
> > > >
> > > > Il giorno ven 31 gen 2020 alle ore 00:27 <he...@free.fr> ha
> > > > scritto:
> > > > >
> > > > > maven-studies looks like the right location:
> > > > > https://github.com/apache/maven-studies
> > > > >
> > > > > Regards,
> > > > >
> > > > > Hervé
> > > > >
> > > > > ----- Mail original -----
> > > > > De: "Enrico Olivelli" <eo...@gmail.com>
> > > > > À: "Maven Developers List" <de...@maven.apache.org>
> > > > > Envoyé: Jeudi 30 Janvier 2020 04:31:14
> > > > > Objet: Re: Maven Runtime Metrics
> > > > >
> > > > > I sound like to draft a prototype.
> > > > >
> > > > > Any suggestion about which repository should contain this new
> Metrics
> > > > API?
> > > > >
> > > > >
> > > > > Enrico
> > > > >
> > > > > Il Mar 28 Gen 2020, 13:54 Romain Manni-Bucau <
> rmannibucau@gmail.com>
> > ha
> > > > > scritto:
> > > > >
> > > > > > technically we can shade microprofile, metrics/dropwizard or even
> > > > sirona
> > > > > > but maven requires quite a different model. A typical example is
> > the
> > > > expiry
> > > > > > or weighted model will not fit the reporting maven needs (very
> long
> > > > running
> > > > > > instances vs "batch" like executions) and the weighted impl can
> be
> > an
> > > > issue
> > > > > > but is spec-ed by MP (so the shaded impl would also have this
> > > > unexpected
> > > > > > impl)
> > > > > >
> > > > > > What about by just adding counters, enabling to inject them in
> > mojos -
> > > > > > maybe with a qualifier? - and then see if we need to move to
> > something
> > > > more
> > > > > > widely used or not (but my past experiences kind of tend to show
> > it is
> > > > not
> > > > > > needed).
> > > > > >
> > > > > > Romain Manni-Bucau
> > > > > > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > > > > > <https://rmannibucau.metawerx.net/> | Old Blog
> > > > > > <http://rmannibucau.wordpress.com> | Github <
> > > > > > https://github.com/rmannibucau> |
> > > > > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > > > > > <
> > > > > >
> > > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Le mar. 28 janv. 2020 à 13:42, Enrico Olivelli <
> > eolivelli@gmail.com> a
> > > > > > écrit :
> > > > > >
> > > > > > > Il Mar 28 Gen 2020, 13:30 Lennart Jörelid <
> > lennart.jorelid@gmail.com>
> > > > ha
> > > > > > > scritto:
> > > > > > >
> > > > > > > > Would it be possible to use an existing standard for the
> > > > specification
> > > > > > of
> > > > > > > > such a metrics API?
> > > > > > > >
> > > > > > >
> > > > > > > In my opinion we should have our own model, Maven will impose
> > this
> > > > new
> > > > > > API
> > > > > > > to plugins and we cannot depend on third party tools as Romain
> > also
> > > > said
> > > > > > >
> > > > > > > Okay for annotations
> > > > > > >
> > > > > > > Enrico
> > > > > > >
> > > > > > > Something like a shaded version of Microprofile Metrics, where
> > the
> > > > API
> > > > > > can
> > > > > > > > consist entirely of Annotations.
> > > > > > > > (There are programmatic approaches as well, but I suppose we
> > are
> > > > > > > interested
> > > > > > > > in limiting the footprint of metrics within the Maven
> codebase
> > as
> > > > much
> > > > > > as
> > > > > > > > possible).
> > > > > > > >
> > > > > > > > ... implying, of course, that we would need to make a
> > > > Maven-specific
> > > > > > > > implementation as discussed above.
> > > > > > > >
> > > > > > > > On Tue, Jan 28, 2020 at 10:17 AM Enrico Olivelli <
> > > > eolivelli@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Il giorno mar 28 gen 2020 alle ore 09:15 Romain
> Manni-Bucau <
> > > > > > > > > rmannibucau@gmail.com> ha scritto:
> > > > > > > > >
> > > > > > > > > > +1 generally since all extensions doing that report wrong
> > data
> > > > > > > > > > -1 to use a mainstream impl if not shaded - it would just
> > > > bring us
> > > > > > > new
> > > > > > > > > > conflicts i think.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Totally agree.
> > > > > > > > > In my view any "impl" will be plugged with an "extension".
> > > > > > > > > We will provide only a default no-op implementation
> > > > > > > > >
> > > > > > > > > Enrico
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > Side note being: we can start we just counters so no lib
> is
> > > > fine
> > > > > > > imho.
> > > > > > > > > >
> > > > > > > > > > Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <
> > > > > > eolivelli@gmail.com>
> > > > > > > a
> > > > > > > > > > écrit :
> > > > > > > > > >
> > > > > > > > > > > Hi,
> > > > > > > > > > > I would like to work on introducing an explicit support
> > in
> > > > Maven
> > > > > > > Core
> > > > > > > > > for
> > > > > > > > > > > recording "metrics".
> > > > > > > > > > > As far as I know (and I have still limited knowledge
> > about
> > > > the
> > > > > > > > > internals)
> > > > > > > > > > > you can write an "extension" and you can intercept main
> > > > lifecycle
> > > > > > > > > events,
> > > > > > > > > > > and using these hooks you can try to record metrics.
> > > > > > > > > > > But my proposal is from another point of view and I
> want
> > to
> > > > go
> > > > > > > > deeper.
> > > > > > > > > > >
> > > > > > > > > > > We are adding caches, making refactors....all good
> > things,
> > > > but we
> > > > > > > > > should
> > > > > > > > > > > have a way to track how Maven is behaving in a
> measurable
> > > > manner.
> > > > > > > > > > >
> > > > > > > > > > > Usually in order to capture realtime metrics you are
> > using
> > > > some
> > > > > > > > Metrics
> > > > > > > > > > > framework, like Prometheus, Dropwizards....and you
> attach
> > > > your
> > > > > > > > process
> > > > > > > > > > to a
> > > > > > > > > > > metrics database and capture all of the values.
> > > > > > > > > > > Then you can process all of the data, realtime and/or
> > > > offline,
> > > > > > you
> > > > > > > > can
> > > > > > > > > > > create dashboards, you can also fine tune your
> > configuration
> > > > or
> > > > > > > > change
> > > > > > > > > > your
> > > > > > > > > > > code.
> > > > > > > > > > >
> > > > > > > > > > > These frameworks deal with time series created by
> > capturing
> > > > > > > > Counters,
> > > > > > > > > > > Summaries, Gauges.
> > > > > > > > > > >
> > > > > > > > > > > I would like to introduce an API to be used by Maven
> Core
> > > > and by
> > > > > > > > Plugin
> > > > > > > > > > to
> > > > > > > > > > > expose internal metrics, like:
> > > > > > > > > > > - internal caches size
> > > > > > > > > > > - memory usage
> > > > > > > > > > > - disk reads/writes
> > > > > > > > > > > - network traffic
> > > > > > > > > > > - any kind of specific metric for plugins (like "number
> > of
> > > > tests
> > > > > > > > > > executed",
> > > > > > > > > > > "number of files sent to javac...")
> > > > > > > > > > > - .... all of the usual stuff you track in order to
> > > > understand
> > > > > > > wha's
> > > > > > > > > > going
> > > > > > > > > > > on
> > > > > > > > > > >
> > > > > > > > > > > I am saying a Maven Metrics API because we must not
> > stick to
> > > > a
> > > > > > > single
> > > > > > > > > > > "provider" (say "DropWizard"), it will be a bad choice
> > in the
> > > > > > long
> > > > > > > > > term.
> > > > > > > > > > >
> > > > > > > > > > > The API will expose to Maven Core and to plugins a way
> > to get
> > > > > > > access
> > > > > > > > to
> > > > > > > > > > > Metrics and report useful values.
> > > > > > > > > > >
> > > > > > > > > > > @InjectMe
> > > > > > > > > > > MetricsProvider metrics;
> > > > > > > > > > >
> > > > > > > > > > > metrics.getCounter("javacFiles").add(....)
> > > > > > > > > > > metrics.getSummary("downloadTime").add(....)
> > > > > > > > > > >
> > > > > > > > > > > I don't want to go down into details as this is only a
> > > > > > preliminary
> > > > > > > > > email
> > > > > > > > > > > but in my view we could let the user attach an
> > > > "implementation"
> > > > > > > just
> > > > > > > > by
> > > > > > > > > > > dropping in an "extension", in order not to ship with
> > Maven
> > > > Core
> > > > > > a
> > > > > > > > lot
> > > > > > > > > of
> > > > > > > > > > > third party dependencies.
> > > > > > > > > > >
> > > > > > > > > > > I have some "production" experience with this topic
> > > > (actually I
> > > > > > did
> > > > > > > > > > > something like that in my company and in ZooKeeper
> > project)
> > > > so I
> > > > > > > will
> > > > > > > > > be
> > > > > > > > > > > happy to prepare and share a design document.
> > > > > > > > > > >
> > > > > > > > > > > With this work we will open up the way to knowing how a
> > Maven
> > > > > > build
> > > > > > > > > > works,
> > > > > > > > > > > with the 'time dimension', on the local machine of the
> > > > developer
> > > > > > > but
> > > > > > > > > even
> > > > > > > > > > > on CI servers.
> > > > > > > > > > >
> > > > > > > > > > > Any comment is very welcome
> > > > > > > > > > >
> > > > > > > > > > > Regards
> > > > > > > > > > > Enrico
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > --
> > > > > > > > +==============================+
> > > > > > > > | Bästa hälsningar,
> > > > > > > > | [sw. "Best regards"]
> > > > > > > > |
> > > > > > > > | Lennart Jörelid
> > > > > > > > | EAI Architect & Integrator
> > > > > > > > |
> > > > > > > > | jGuru Europe AB
> > > > > > > > | Mölnlycke - Kista
> > > > > > > > |
> > > > > > > > | Email: lj@jguru.se
> > > > > > > > | URL:   www.jguru.se
> > > > > > > > | Phone
> > > > > > > > | (skype):    jgurueurope
> > > > > > > > | (intl):     +46 708 507 603
> > > > > > > > | (domestic): 0708 - 507 603
> > > > > > > > +==============================+
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>

Re: Maven Runtime Metrics

Posted by Romain Manni-Bucau <rm...@gmail.com>.

Hmm, not sure it is a good example, plugins do play IoC game already and is
the cleanest way to move forward otherwise you will leak the counter or
need to play with thread locals which is worse for plugins using this
pattern (will forbid plugins to use threads).

That said you can still replace all this code by anything you want, even an
IoC lookup and call using a classfiletransformer in ClassRealm plugin
classloader. It is just a way to rewrite original code to something else.

Le sam. 14 mars 2020 à 17:59, Enrico Olivelli <eo...@gmail.com> a
écrit :

> Il giorno sab 14 mar 2020 alle ore 17:53 Romain Manni-Bucau
> <rm...@gmail.com> ha scritto:
> >
> > Assuming maven core/boot is components, a component registry can
> instrument
> > plugins and subclassloaders with classfiletransformers so maybe ok to
> keep
> > it in the IoC.
>
> Romain,
> can please you elaborate more ?
>
> I am thinking this this case:
>
> class MyCoolCache {
>     private static final Counter counter =
> MetricsSystem.getCounter("cacheHits");
>     public T get(K key) {
>   ....    counter.increment();
>   }
> }
>
> I am a developer of a plugin and I want to set some metric to see the
> runtime behaviour of my component, I don't want my class to play the
> IoC game
>
> You can see it like a "Logger". Loggers are not subject to IoC because
> usually you may want a Logger absolutely everywhere and without
> changing/refactoring your code.
> If I want to add a "temporary" metric I don't want to need to change
> constructors, add getters/setters....
>
>
> Enrico
>
> >
> > Le sam. 14 mars 2020 à 15:14, Enrico Olivelli <eo...@gmail.com> a
> > écrit :
> >
> > > I am starting this work.
> > > I have pushed a copy of current maven core master to maven-studies.
> > >
> > > Then I have added my "maven-metrics" module.
> > > This module contains only the API to be implemented by Metrics
> > > Providers and a No-op implementation.
> > >
> > > You can find the draft API here.
> > >
> https://github.com/apache/maven-studies/tree/maven-metrics/maven-metrics
> > >
> > > It is not final it is just the first import to git, I have to see some
> > > real world Maven cases before designing the final one.
> > >
> > > Now I am going to instrument some Maven core subsystem and implement a
> > > simple Metrics Provider in order to see some data.
> > >
> > > My very first problem is to boot the MetricsProvider and to give
> > > access to it from any Java class and module.
> > > Maven uses Dependency Injection and this is cool, but with this
> > > approach you will not be able to instrument non-component classes,
> > > like internal Caches or Network facilities.
> > >
> > > I am leaning towards using some kind of "global static reference"
> > > (like a pure Java Singleton) to hold a reference to the Metrics
> > > subsystem.
> > >
> > > By default the Metrics Provider will be the No-op implementation, so
> > > third party tools won't see NULLs.
> > >
> > > in MavenCli I will "boot" (and manage the whole lifecycle) the Metrics
> > > Provider and set this "global reference".
> > >
> > > The Metrics Provider API will be in the main Maven classloader and the
> > > Metrics Provider will have to be loaded in the same
> > > context/classloader.
> > > Plugins and internal classes will use the API and not directly the
> > > provider implementation.
> > > I don't know how classloading works very well in the case of
> Maven/Plexus.
> > >
> > > Any suggestion is very appreciated.
> > >
> > > I have very very limited time for this work, but I see value and I
> > > will move forward when I have cycles
> > >
> > > Cheers
> > >
> > > Enrico
> > >
> > >
> > >
> > > Il giorno ven 31 gen 2020 alle ore 00:27 <he...@free.fr> ha
> > > scritto:
> > > >
> > > > maven-studies looks like the right location:
> > > > https://github.com/apache/maven-studies
> > > >
> > > > Regards,
> > > >
> > > > Hervé
> > > >
> > > > ----- Mail original -----
> > > > De: "Enrico Olivelli" <eo...@gmail.com>
> > > > À: "Maven Developers List" <de...@maven.apache.org>
> > > > Envoyé: Jeudi 30 Janvier 2020 04:31:14
> > > > Objet: Re: Maven Runtime Metrics
> > > >
> > > > I sound like to draft a prototype.
> > > >
> > > > Any suggestion about which repository should contain this new Metrics
> > > API?
> > > >
> > > >
> > > > Enrico
> > > >
> > > > Il Mar 28 Gen 2020, 13:54 Romain Manni-Bucau <rm...@gmail.com>
> ha
> > > > scritto:
> > > >
> > > > > technically we can shade microprofile, metrics/dropwizard or even
> > > sirona
> > > > > but maven requires quite a different model. A typical example is
> the
> > > expiry
> > > > > or weighted model will not fit the reporting maven needs (very long
> > > running
> > > > > instances vs "batch" like executions) and the weighted impl can be
> an
> > > issue
> > > > > but is spec-ed by MP (so the shaded impl would also have this
> > > unexpected
> > > > > impl)
> > > > >
> > > > > What about by just adding counters, enabling to inject them in
> mojos -
> > > > > maybe with a qualifier? - and then see if we need to move to
> something
> > > more
> > > > > widely used or not (but my past experiences kind of tend to show
> it is
> > > not
> > > > > needed).
> > > > >
> > > > > Romain Manni-Bucau
> > > > > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > > > > <https://rmannibucau.metawerx.net/> | Old Blog
> > > > > <http://rmannibucau.wordpress.com> | Github <
> > > > > https://github.com/rmannibucau> |
> > > > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > > > > <
> > > > >
> > >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > > > >
> > > > >
> > > > >
> > > > > Le mar. 28 janv. 2020 à 13:42, Enrico Olivelli <
> eolivelli@gmail.com> a
> > > > > écrit :
> > > > >
> > > > > > Il Mar 28 Gen 2020, 13:30 Lennart Jörelid <
> lennart.jorelid@gmail.com>
> > > ha
> > > > > > scritto:
> > > > > >
> > > > > > > Would it be possible to use an existing standard for the
> > > specification
> > > > > of
> > > > > > > such a metrics API?
> > > > > > >
> > > > > >
> > > > > > In my opinion we should have our own model, Maven will impose
> this
> > > new
> > > > > API
> > > > > > to plugins and we cannot depend on third party tools as Romain
> also
> > > said
> > > > > >
> > > > > > Okay for annotations
> > > > > >
> > > > > > Enrico
> > > > > >
> > > > > > Something like a shaded version of Microprofile Metrics, where
> the
> > > API
> > > > > can
> > > > > > > consist entirely of Annotations.
> > > > > > > (There are programmatic approaches as well, but I suppose we
> are
> > > > > > interested
> > > > > > > in limiting the footprint of metrics within the Maven codebase
> as
> > > much
> > > > > as
> > > > > > > possible).
> > > > > > >
> > > > > > > ... implying, of course, that we would need to make a
> > > Maven-specific
> > > > > > > implementation as discussed above.
> > > > > > >
> > > > > > > On Tue, Jan 28, 2020 at 10:17 AM Enrico Olivelli <
> > > eolivelli@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Il giorno mar 28 gen 2020 alle ore 09:15 Romain Manni-Bucau <
> > > > > > > > rmannibucau@gmail.com> ha scritto:
> > > > > > > >
> > > > > > > > > +1 generally since all extensions doing that report wrong
> data
> > > > > > > > > -1 to use a mainstream impl if not shaded - it would just
> > > bring us
> > > > > > new
> > > > > > > > > conflicts i think.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Totally agree.
> > > > > > > > In my view any "impl" will be plugged with an "extension".
> > > > > > > > We will provide only a default no-op implementation
> > > > > > > >
> > > > > > > > Enrico
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > Side note being: we can start we just counters so no lib is
> > > fine
> > > > > > imho.
> > > > > > > > >
> > > > > > > > > Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <
> > > > > eolivelli@gmail.com>
> > > > > > a
> > > > > > > > > écrit :
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > > I would like to work on introducing an explicit support
> in
> > > Maven
> > > > > > Core
> > > > > > > > for
> > > > > > > > > > recording "metrics".
> > > > > > > > > > As far as I know (and I have still limited knowledge
> about
> > > the
> > > > > > > > internals)
> > > > > > > > > > you can write an "extension" and you can intercept main
> > > lifecycle
> > > > > > > > events,
> > > > > > > > > > and using these hooks you can try to record metrics.
> > > > > > > > > > But my proposal is from another point of view and I want
> to
> > > go
> > > > > > > deeper.
> > > > > > > > > >
> > > > > > > > > > We are adding caches, making refactors....all good
> things,
> > > but we
> > > > > > > > should
> > > > > > > > > > have a way to track how Maven is behaving in a measurable
> > > manner.
> > > > > > > > > >
> > > > > > > > > > Usually in order to capture realtime metrics you are
> using
> > > some
> > > > > > > Metrics
> > > > > > > > > > framework, like Prometheus, Dropwizards....and you attach
> > > your
> > > > > > > process
> > > > > > > > > to a
> > > > > > > > > > metrics database and capture all of the values.
> > > > > > > > > > Then you can process all of the data, realtime and/or
> > > offline,
> > > > > you
> > > > > > > can
> > > > > > > > > > create dashboards, you can also fine tune your
> configuration
> > > or
> > > > > > > change
> > > > > > > > > your
> > > > > > > > > > code.
> > > > > > > > > >
> > > > > > > > > > These frameworks deal with time series created by
> capturing
> > > > > > > Counters,
> > > > > > > > > > Summaries, Gauges.
> > > > > > > > > >
> > > > > > > > > > I would like to introduce an API to be used by Maven Core
> > > and by
> > > > > > > Plugin
> > > > > > > > > to
> > > > > > > > > > expose internal metrics, like:
> > > > > > > > > > - internal caches size
> > > > > > > > > > - memory usage
> > > > > > > > > > - disk reads/writes
> > > > > > > > > > - network traffic
> > > > > > > > > > - any kind of specific metric for plugins (like "number
> of
> > > tests
> > > > > > > > > executed",
> > > > > > > > > > "number of files sent to javac...")
> > > > > > > > > > - .... all of the usual stuff you track in order to
> > > understand
> > > > > > wha's
> > > > > > > > > going
> > > > > > > > > > on
> > > > > > > > > >
> > > > > > > > > > I am saying a Maven Metrics API because we must not
> stick to
> > > a
> > > > > > single
> > > > > > > > > > "provider" (say "DropWizard"), it will be a bad choice
> in the
> > > > > long
> > > > > > > > term.
> > > > > > > > > >
> > > > > > > > > > The API will expose to Maven Core and to plugins a way
> to get
> > > > > > access
> > > > > > > to
> > > > > > > > > > Metrics and report useful values.
> > > > > > > > > >
> > > > > > > > > > @InjectMe
> > > > > > > > > > MetricsProvider metrics;
> > > > > > > > > >
> > > > > > > > > > metrics.getCounter("javacFiles").add(....)
> > > > > > > > > > metrics.getSummary("downloadTime").add(....)
> > > > > > > > > >
> > > > > > > > > > I don't want to go down into details as this is only a
> > > > > preliminary
> > > > > > > > email
> > > > > > > > > > but in my view we could let the user attach an
> > > "implementation"
> > > > > > just
> > > > > > > by
> > > > > > > > > > dropping in an "extension", in order not to ship with
> Maven
> > > Core
> > > > > a
> > > > > > > lot
> > > > > > > > of
> > > > > > > > > > third party dependencies.
> > > > > > > > > >
> > > > > > > > > > I have some "production" experience with this topic
> > > (actually I
> > > > > did
> > > > > > > > > > something like that in my company and in ZooKeeper
> project)
> > > so I
> > > > > > will
> > > > > > > > be
> > > > > > > > > > happy to prepare and share a design document.
> > > > > > > > > >
> > > > > > > > > > With this work we will open up the way to knowing how a
> Maven
> > > > > build
> > > > > > > > > works,
> > > > > > > > > > with the 'time dimension', on the local machine of the
> > > developer
> > > > > > but
> > > > > > > > even
> > > > > > > > > > on CI servers.
> > > > > > > > > >
> > > > > > > > > > Any comment is very welcome
> > > > > > > > > >
> > > > > > > > > > Regards
> > > > > > > > > > Enrico
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > --
> > > > > > > +==============================+
> > > > > > > | Bästa hälsningar,
> > > > > > > | [sw. "Best regards"]
> > > > > > > |
> > > > > > > | Lennart Jörelid
> > > > > > > | EAI Architect & Integrator
> > > > > > > |
> > > > > > > | jGuru Europe AB
> > > > > > > | Mölnlycke - Kista
> > > > > > > |
> > > > > > > | Email: lj@jguru.se
> > > > > > > | URL:   www.jguru.se
> > > > > > > | Phone
> > > > > > > | (skype):    jgurueurope
> > > > > > > | (intl):     +46 708 507 603
> > > > > > > | (domestic): 0708 - 507 603
> > > > > > > +==============================+
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > > For additional commands, e-mail: dev-help@maven.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>

Re: Maven Runtime Metrics

Posted by Enrico Olivelli <eo...@gmail.com>.

Il giorno sab 14 mar 2020 alle ore 17:53 Romain Manni-Bucau
<rm...@gmail.com> ha scritto:
>
> Assuming maven core/boot is components, a component registry can instrument
> plugins and subclassloaders with classfiletransformers so maybe ok to keep
> it in the IoC.

Romain,
can please you elaborate more ?

I am thinking this this case:

class MyCoolCache {
    private static final Counter counter =
MetricsSystem.getCounter("cacheHits");
    public T get(K key) {
  ....    counter.increment();
  }
}

I am a developer of a plugin and I want to set some metric to see the
runtime behaviour of my component, I don't want my class to play the
IoC game

You can see it like a "Logger". Loggers are not subject to IoC because
usually you may want a Logger absolutely everywhere and without
changing/refactoring your code.
If I want to add a "temporary" metric I don't want to need to change
constructors, add getters/setters....


Enrico

>
> Le sam. 14 mars 2020 à 15:14, Enrico Olivelli <eo...@gmail.com> a
> écrit :
>
> > I am starting this work.
> > I have pushed a copy of current maven core master to maven-studies.
> >
> > Then I have added my "maven-metrics" module.
> > This module contains only the API to be implemented by Metrics
> > Providers and a No-op implementation.
> >
> > You can find the draft API here.
> > https://github.com/apache/maven-studies/tree/maven-metrics/maven-metrics
> >
> > It is not final it is just the first import to git, I have to see some
> > real world Maven cases before designing the final one.
> >
> > Now I am going to instrument some Maven core subsystem and implement a
> > simple Metrics Provider in order to see some data.
> >
> > My very first problem is to boot the MetricsProvider and to give
> > access to it from any Java class and module.
> > Maven uses Dependency Injection and this is cool, but with this
> > approach you will not be able to instrument non-component classes,
> > like internal Caches or Network facilities.
> >
> > I am leaning towards using some kind of "global static reference"
> > (like a pure Java Singleton) to hold a reference to the Metrics
> > subsystem.
> >
> > By default the Metrics Provider will be the No-op implementation, so
> > third party tools won't see NULLs.
> >
> > in MavenCli I will "boot" (and manage the whole lifecycle) the Metrics
> > Provider and set this "global reference".
> >
> > The Metrics Provider API will be in the main Maven classloader and the
> > Metrics Provider will have to be loaded in the same
> > context/classloader.
> > Plugins and internal classes will use the API and not directly the
> > provider implementation.
> > I don't know how classloading works very well in the case of Maven/Plexus.
> >
> > Any suggestion is very appreciated.
> >
> > I have very very limited time for this work, but I see value and I
> > will move forward when I have cycles
> >
> > Cheers
> >
> > Enrico
> >
> >
> >
> > Il giorno ven 31 gen 2020 alle ore 00:27 <he...@free.fr> ha
> > scritto:
> > >
> > > maven-studies looks like the right location:
> > > https://github.com/apache/maven-studies
> > >
> > > Regards,
> > >
> > > Hervé
> > >
> > > ----- Mail original -----
> > > De: "Enrico Olivelli" <eo...@gmail.com>
> > > À: "Maven Developers List" <de...@maven.apache.org>
> > > Envoyé: Jeudi 30 Janvier 2020 04:31:14
> > > Objet: Re: Maven Runtime Metrics
> > >
> > > I sound like to draft a prototype.
> > >
> > > Any suggestion about which repository should contain this new Metrics
> > API?
> > >
> > >
> > > Enrico
> > >
> > > Il Mar 28 Gen 2020, 13:54 Romain Manni-Bucau <rm...@gmail.com> ha
> > > scritto:
> > >
> > > > technically we can shade microprofile, metrics/dropwizard or even
> > sirona
> > > > but maven requires quite a different model. A typical example is the
> > expiry
> > > > or weighted model will not fit the reporting maven needs (very long
> > running
> > > > instances vs "batch" like executions) and the weighted impl can be an
> > issue
> > > > but is spec-ed by MP (so the shaded impl would also have this
> > unexpected
> > > > impl)
> > > >
> > > > What about by just adding counters, enabling to inject them in mojos -
> > > > maybe with a qualifier? - and then see if we need to move to something
> > more
> > > > widely used or not (but my past experiences kind of tend to show it is
> > not
> > > > needed).
> > > >
> > > > Romain Manni-Bucau
> > > > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > > > <https://rmannibucau.metawerx.net/> | Old Blog
> > > > <http://rmannibucau.wordpress.com> | Github <
> > > > https://github.com/rmannibucau> |
> > > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > > > <
> > > >
> > https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > > >
> > > >
> > > >
> > > > Le mar. 28 janv. 2020 à 13:42, Enrico Olivelli <eo...@gmail.com> a
> > > > écrit :
> > > >
> > > > > Il Mar 28 Gen 2020, 13:30 Lennart Jörelid <le...@gmail.com>
> > ha
> > > > > scritto:
> > > > >
> > > > > > Would it be possible to use an existing standard for the
> > specification
> > > > of
> > > > > > such a metrics API?
> > > > > >
> > > > >
> > > > > In my opinion we should have our own model, Maven will impose this
> > new
> > > > API
> > > > > to plugins and we cannot depend on third party tools as Romain also
> > said
> > > > >
> > > > > Okay for annotations
> > > > >
> > > > > Enrico
> > > > >
> > > > > Something like a shaded version of Microprofile Metrics, where the
> > API
> > > > can
> > > > > > consist entirely of Annotations.
> > > > > > (There are programmatic approaches as well, but I suppose we are
> > > > > interested
> > > > > > in limiting the footprint of metrics within the Maven codebase as
> > much
> > > > as
> > > > > > possible).
> > > > > >
> > > > > > ... implying, of course, that we would need to make a
> > Maven-specific
> > > > > > implementation as discussed above.
> > > > > >
> > > > > > On Tue, Jan 28, 2020 at 10:17 AM Enrico Olivelli <
> > eolivelli@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Il giorno mar 28 gen 2020 alle ore 09:15 Romain Manni-Bucau <
> > > > > > > rmannibucau@gmail.com> ha scritto:
> > > > > > >
> > > > > > > > +1 generally since all extensions doing that report wrong data
> > > > > > > > -1 to use a mainstream impl if not shaded - it would just
> > bring us
> > > > > new
> > > > > > > > conflicts i think.
> > > > > > > >
> > > > > > >
> > > > > > > Totally agree.
> > > > > > > In my view any "impl" will be plugged with an "extension".
> > > > > > > We will provide only a default no-op implementation
> > > > > > >
> > > > > > > Enrico
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Side note being: we can start we just counters so no lib is
> > fine
> > > > > imho.
> > > > > > > >
> > > > > > > > Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <
> > > > eolivelli@gmail.com>
> > > > > a
> > > > > > > > écrit :
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > > I would like to work on introducing an explicit support in
> > Maven
> > > > > Core
> > > > > > > for
> > > > > > > > > recording "metrics".
> > > > > > > > > As far as I know (and I have still limited knowledge about
> > the
> > > > > > > internals)
> > > > > > > > > you can write an "extension" and you can intercept main
> > lifecycle
> > > > > > > events,
> > > > > > > > > and using these hooks you can try to record metrics.
> > > > > > > > > But my proposal is from another point of view and I want to
> > go
> > > > > > deeper.
> > > > > > > > >
> > > > > > > > > We are adding caches, making refactors....all good things,
> > but we
> > > > > > > should
> > > > > > > > > have a way to track how Maven is behaving in a measurable
> > manner.
> > > > > > > > >
> > > > > > > > > Usually in order to capture realtime metrics you are using
> > some
> > > > > > Metrics
> > > > > > > > > framework, like Prometheus, Dropwizards....and you attach
> > your
> > > > > > process
> > > > > > > > to a
> > > > > > > > > metrics database and capture all of the values.
> > > > > > > > > Then you can process all of the data, realtime and/or
> > offline,
> > > > you
> > > > > > can
> > > > > > > > > create dashboards, you can also fine tune your configuration
> > or
> > > > > > change
> > > > > > > > your
> > > > > > > > > code.
> > > > > > > > >
> > > > > > > > > These frameworks deal with time series created by capturing
> > > > > > Counters,
> > > > > > > > > Summaries, Gauges.
> > > > > > > > >
> > > > > > > > > I would like to introduce an API to be used by Maven Core
> > and by
> > > > > > Plugin
> > > > > > > > to
> > > > > > > > > expose internal metrics, like:
> > > > > > > > > - internal caches size
> > > > > > > > > - memory usage
> > > > > > > > > - disk reads/writes
> > > > > > > > > - network traffic
> > > > > > > > > - any kind of specific metric for plugins (like "number of
> > tests
> > > > > > > > executed",
> > > > > > > > > "number of files sent to javac...")
> > > > > > > > > - .... all of the usual stuff you track in order to
> > understand
> > > > > wha's
> > > > > > > > going
> > > > > > > > > on
> > > > > > > > >
> > > > > > > > > I am saying a Maven Metrics API because we must not stick to
> > a
> > > > > single
> > > > > > > > > "provider" (say "DropWizard"), it will be a bad choice in the
> > > > long
> > > > > > > term.
> > > > > > > > >
> > > > > > > > > The API will expose to Maven Core and to plugins a way to get
> > > > > access
> > > > > > to
> > > > > > > > > Metrics and report useful values.
> > > > > > > > >
> > > > > > > > > @InjectMe
> > > > > > > > > MetricsProvider metrics;
> > > > > > > > >
> > > > > > > > > metrics.getCounter("javacFiles").add(....)
> > > > > > > > > metrics.getSummary("downloadTime").add(....)
> > > > > > > > >
> > > > > > > > > I don't want to go down into details as this is only a
> > > > preliminary
> > > > > > > email
> > > > > > > > > but in my view we could let the user attach an
> > "implementation"
> > > > > just
> > > > > > by
> > > > > > > > > dropping in an "extension", in order not to ship with Maven
> > Core
> > > > a
> > > > > > lot
> > > > > > > of
> > > > > > > > > third party dependencies.
> > > > > > > > >
> > > > > > > > > I have some "production" experience with this topic
> > (actually I
> > > > did
> > > > > > > > > something like that in my company and in ZooKeeper project)
> > so I
> > > > > will
> > > > > > > be
> > > > > > > > > happy to prepare and share a design document.
> > > > > > > > >
> > > > > > > > > With this work we will open up the way to knowing how a Maven
> > > > build
> > > > > > > > works,
> > > > > > > > > with the 'time dimension', on the local machine of the
> > developer
> > > > > but
> > > > > > > even
> > > > > > > > > on CI servers.
> > > > > > > > >
> > > > > > > > > Any comment is very welcome
> > > > > > > > >
> > > > > > > > > Regards
> > > > > > > > > Enrico
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > --
> > > > > > +==============================+
> > > > > > | Bästa hälsningar,
> > > > > > | [sw. "Best regards"]
> > > > > > |
> > > > > > | Lennart Jörelid
> > > > > > | EAI Architect & Integrator
> > > > > > |
> > > > > > | jGuru Europe AB
> > > > > > | Mölnlycke - Kista
> > > > > > |
> > > > > > | Email: lj@jguru.se
> > > > > > | URL:   www.jguru.se
> > > > > > | Phone
> > > > > > | (skype):    jgurueurope
> > > > > > | (intl):     +46 708 507 603
> > > > > > | (domestic): 0708 - 507 603
> > > > > > +==============================+
> > > > > >
> > > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org

Re: Maven Runtime Metrics

Posted by Romain Manni-Bucau <rm...@gmail.com>.

Assuming maven core/boot is components, a component registry can instrument
plugins and subclassloaders with classfiletransformers so maybe ok to keep
it in the IoC.

Le sam. 14 mars 2020 à 15:14, Enrico Olivelli <eo...@gmail.com> a
écrit :

> I am starting this work.
> I have pushed a copy of current maven core master to maven-studies.
>
> Then I have added my "maven-metrics" module.
> This module contains only the API to be implemented by Metrics
> Providers and a No-op implementation.
>
> You can find the draft API here.
> https://github.com/apache/maven-studies/tree/maven-metrics/maven-metrics
>
> It is not final it is just the first import to git, I have to see some
> real world Maven cases before designing the final one.
>
> Now I am going to instrument some Maven core subsystem and implement a
> simple Metrics Provider in order to see some data.
>
> My very first problem is to boot the MetricsProvider and to give
> access to it from any Java class and module.
> Maven uses Dependency Injection and this is cool, but with this
> approach you will not be able to instrument non-component classes,
> like internal Caches or Network facilities.
>
> I am leaning towards using some kind of "global static reference"
> (like a pure Java Singleton) to hold a reference to the Metrics
> subsystem.
>
> By default the Metrics Provider will be the No-op implementation, so
> third party tools won't see NULLs.
>
> in MavenCli I will "boot" (and manage the whole lifecycle) the Metrics
> Provider and set this "global reference".
>
> The Metrics Provider API will be in the main Maven classloader and the
> Metrics Provider will have to be loaded in the same
> context/classloader.
> Plugins and internal classes will use the API and not directly the
> provider implementation.
> I don't know how classloading works very well in the case of Maven/Plexus.
>
> Any suggestion is very appreciated.
>
> I have very very limited time for this work, but I see value and I
> will move forward when I have cycles
>
> Cheers
>
> Enrico
>
>
>
> Il giorno ven 31 gen 2020 alle ore 00:27 <he...@free.fr> ha
> scritto:
> >
> > maven-studies looks like the right location:
> > https://github.com/apache/maven-studies
> >
> > Regards,
> >
> > Hervé
> >
> > ----- Mail original -----
> > De: "Enrico Olivelli" <eo...@gmail.com>
> > À: "Maven Developers List" <de...@maven.apache.org>
> > Envoyé: Jeudi 30 Janvier 2020 04:31:14
> > Objet: Re: Maven Runtime Metrics
> >
> > I sound like to draft a prototype.
> >
> > Any suggestion about which repository should contain this new Metrics
> API?
> >
> >
> > Enrico
> >
> > Il Mar 28 Gen 2020, 13:54 Romain Manni-Bucau <rm...@gmail.com> ha
> > scritto:
> >
> > > technically we can shade microprofile, metrics/dropwizard or even
> sirona
> > > but maven requires quite a different model. A typical example is the
> expiry
> > > or weighted model will not fit the reporting maven needs (very long
> running
> > > instances vs "batch" like executions) and the weighted impl can be an
> issue
> > > but is spec-ed by MP (so the shaded impl would also have this
> unexpected
> > > impl)
> > >
> > > What about by just adding counters, enabling to inject them in mojos -
> > > maybe with a qualifier? - and then see if we need to move to something
> more
> > > widely used or not (but my past experiences kind of tend to show it is
> not
> > > needed).
> > >
> > > Romain Manni-Bucau
> > > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > > <https://rmannibucau.metawerx.net/> | Old Blog
> > > <http://rmannibucau.wordpress.com> | Github <
> > > https://github.com/rmannibucau> |
> > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > > <
> > >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > >
> > >
> > >
> > > Le mar. 28 janv. 2020 à 13:42, Enrico Olivelli <eo...@gmail.com> a
> > > écrit :
> > >
> > > > Il Mar 28 Gen 2020, 13:30 Lennart Jörelid <le...@gmail.com>
> ha
> > > > scritto:
> > > >
> > > > > Would it be possible to use an existing standard for the
> specification
> > > of
> > > > > such a metrics API?
> > > > >
> > > >
> > > > In my opinion we should have our own model, Maven will impose this
> new
> > > API
> > > > to plugins and we cannot depend on third party tools as Romain also
> said
> > > >
> > > > Okay for annotations
> > > >
> > > > Enrico
> > > >
> > > > Something like a shaded version of Microprofile Metrics, where the
> API
> > > can
> > > > > consist entirely of Annotations.
> > > > > (There are programmatic approaches as well, but I suppose we are
> > > > interested
> > > > > in limiting the footprint of metrics within the Maven codebase as
> much
> > > as
> > > > > possible).
> > > > >
> > > > > ... implying, of course, that we would need to make a
> Maven-specific
> > > > > implementation as discussed above.
> > > > >
> > > > > On Tue, Jan 28, 2020 at 10:17 AM Enrico Olivelli <
> eolivelli@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Il giorno mar 28 gen 2020 alle ore 09:15 Romain Manni-Bucau <
> > > > > > rmannibucau@gmail.com> ha scritto:
> > > > > >
> > > > > > > +1 generally since all extensions doing that report wrong data
> > > > > > > -1 to use a mainstream impl if not shaded - it would just
> bring us
> > > > new
> > > > > > > conflicts i think.
> > > > > > >
> > > > > >
> > > > > > Totally agree.
> > > > > > In my view any "impl" will be plugged with an "extension".
> > > > > > We will provide only a default no-op implementation
> > > > > >
> > > > > > Enrico
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Side note being: we can start we just counters so no lib is
> fine
> > > > imho.
> > > > > > >
> > > > > > > Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <
> > > eolivelli@gmail.com>
> > > > a
> > > > > > > écrit :
> > > > > > >
> > > > > > > > Hi,
> > > > > > > > I would like to work on introducing an explicit support in
> Maven
> > > > Core
> > > > > > for
> > > > > > > > recording "metrics".
> > > > > > > > As far as I know (and I have still limited knowledge about
> the
> > > > > > internals)
> > > > > > > > you can write an "extension" and you can intercept main
> lifecycle
> > > > > > events,
> > > > > > > > and using these hooks you can try to record metrics.
> > > > > > > > But my proposal is from another point of view and I want to
> go
> > > > > deeper.
> > > > > > > >
> > > > > > > > We are adding caches, making refactors....all good things,
> but we
> > > > > > should
> > > > > > > > have a way to track how Maven is behaving in a measurable
> manner.
> > > > > > > >
> > > > > > > > Usually in order to capture realtime metrics you are using
> some
> > > > > Metrics
> > > > > > > > framework, like Prometheus, Dropwizards....and you attach
> your
> > > > > process
> > > > > > > to a
> > > > > > > > metrics database and capture all of the values.
> > > > > > > > Then you can process all of the data, realtime and/or
> offline,
> > > you
> > > > > can
> > > > > > > > create dashboards, you can also fine tune your configuration
> or
> > > > > change
> > > > > > > your
> > > > > > > > code.
> > > > > > > >
> > > > > > > > These frameworks deal with time series created by capturing
> > > > > Counters,
> > > > > > > > Summaries, Gauges.
> > > > > > > >
> > > > > > > > I would like to introduce an API to be used by Maven Core
> and by
> > > > > Plugin
> > > > > > > to
> > > > > > > > expose internal metrics, like:
> > > > > > > > - internal caches size
> > > > > > > > - memory usage
> > > > > > > > - disk reads/writes
> > > > > > > > - network traffic
> > > > > > > > - any kind of specific metric for plugins (like "number of
> tests
> > > > > > > executed",
> > > > > > > > "number of files sent to javac...")
> > > > > > > > - .... all of the usual stuff you track in order to
> understand
> > > > wha's
> > > > > > > going
> > > > > > > > on
> > > > > > > >
> > > > > > > > I am saying a Maven Metrics API because we must not stick to
> a
> > > > single
> > > > > > > > "provider" (say "DropWizard"), it will be a bad choice in the
> > > long
> > > > > > term.
> > > > > > > >
> > > > > > > > The API will expose to Maven Core and to plugins a way to get
> > > > access
> > > > > to
> > > > > > > > Metrics and report useful values.
> > > > > > > >
> > > > > > > > @InjectMe
> > > > > > > > MetricsProvider metrics;
> > > > > > > >
> > > > > > > > metrics.getCounter("javacFiles").add(....)
> > > > > > > > metrics.getSummary("downloadTime").add(....)
> > > > > > > >
> > > > > > > > I don't want to go down into details as this is only a
> > > preliminary
> > > > > > email
> > > > > > > > but in my view we could let the user attach an
> "implementation"
> > > > just
> > > > > by
> > > > > > > > dropping in an "extension", in order not to ship with Maven
> Core
> > > a
> > > > > lot
> > > > > > of
> > > > > > > > third party dependencies.
> > > > > > > >
> > > > > > > > I have some "production" experience with this topic
> (actually I
> > > did
> > > > > > > > something like that in my company and in ZooKeeper project)
> so I
> > > > will
> > > > > > be
> > > > > > > > happy to prepare and share a design document.
> > > > > > > >
> > > > > > > > With this work we will open up the way to knowing how a Maven
> > > build
> > > > > > > works,
> > > > > > > > with the 'time dimension', on the local machine of the
> developer
> > > > but
> > > > > > even
> > > > > > > > on CI servers.
> > > > > > > >
> > > > > > > > Any comment is very welcome
> > > > > > > >
> > > > > > > > Regards
> > > > > > > > Enrico
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > --
> > > > > +==============================+
> > > > > | Bästa hälsningar,
> > > > > | [sw. "Best regards"]
> > > > > |
> > > > > | Lennart Jörelid
> > > > > | EAI Architect & Integrator
> > > > > |
> > > > > | jGuru Europe AB
> > > > > | Mölnlycke - Kista
> > > > > |
> > > > > | Email: lj@jguru.se
> > > > > | URL:   www.jguru.se
> > > > > | Phone
> > > > > | (skype):    jgurueurope
> > > > > | (intl):     +46 708 507 603
> > > > > | (domestic): 0708 - 507 603
> > > > > +==============================+
> > > > >
> > > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>

Re: Maven Runtime Metrics

Posted by Enrico Olivelli <eo...@gmail.com>.

I am starting this work.
I have pushed a copy of current maven core master to maven-studies.

Then I have added my "maven-metrics" module.
This module contains only the API to be implemented by Metrics
Providers and a No-op implementation.

You can find the draft API here.
https://github.com/apache/maven-studies/tree/maven-metrics/maven-metrics

It is not final it is just the first import to git, I have to see some
real world Maven cases before designing the final one.

Now I am going to instrument some Maven core subsystem and implement a
simple Metrics Provider in order to see some data.

My very first problem is to boot the MetricsProvider and to give
access to it from any Java class and module.
Maven uses Dependency Injection and this is cool, but with this
approach you will not be able to instrument non-component classes,
like internal Caches or Network facilities.

I am leaning towards using some kind of "global static reference"
(like a pure Java Singleton) to hold a reference to the Metrics
subsystem.

By default the Metrics Provider will be the No-op implementation, so
third party tools won't see NULLs.

in MavenCli I will "boot" (and manage the whole lifecycle) the Metrics
Provider and set this "global reference".

The Metrics Provider API will be in the main Maven classloader and the
Metrics Provider will have to be loaded in the same
context/classloader.
Plugins and internal classes will use the API and not directly the
provider implementation.
I don't know how classloading works very well in the case of Maven/Plexus.

Any suggestion is very appreciated.

I have very very limited time for this work, but I see value and I
will move forward when I have cycles

Cheers

Enrico



Il giorno ven 31 gen 2020 alle ore 00:27 <he...@free.fr> ha scritto:
>
> maven-studies looks like the right location:
> https://github.com/apache/maven-studies
>
> Regards,
>
> Hervé
>
> ----- Mail original -----
> De: "Enrico Olivelli" <eo...@gmail.com>
> À: "Maven Developers List" <de...@maven.apache.org>
> Envoyé: Jeudi 30 Janvier 2020 04:31:14
> Objet: Re: Maven Runtime Metrics
>
> I sound like to draft a prototype.
>
> Any suggestion about which repository should contain this new Metrics API?
>
>
> Enrico
>
> Il Mar 28 Gen 2020, 13:54 Romain Manni-Bucau <rm...@gmail.com> ha
> scritto:
>
> > technically we can shade microprofile, metrics/dropwizard or even sirona
> > but maven requires quite a different model. A typical example is the expiry
> > or weighted model will not fit the reporting maven needs (very long running
> > instances vs "batch" like executions) and the weighted impl can be an issue
> > but is spec-ed by MP (so the shaded impl would also have this unexpected
> > impl)
> >
> > What about by just adding counters, enabling to inject them in mojos -
> > maybe with a qualifier? - and then see if we need to move to something more
> > widely used or not (but my past experiences kind of tend to show it is not
> > needed).
> >
> > Romain Manni-Bucau
> > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > <https://rmannibucau.metawerx.net/> | Old Blog
> > <http://rmannibucau.wordpress.com> | Github <
> > https://github.com/rmannibucau> |
> > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > <
> > https://www.packtpub.com/application-development/java-ee-8-high-performance
> > >
> >
> >
> > Le mar. 28 janv. 2020 à 13:42, Enrico Olivelli <eo...@gmail.com> a
> > écrit :
> >
> > > Il Mar 28 Gen 2020, 13:30 Lennart Jörelid <le...@gmail.com> ha
> > > scritto:
> > >
> > > > Would it be possible to use an existing standard for the specification
> > of
> > > > such a metrics API?
> > > >
> > >
> > > In my opinion we should have our own model, Maven will impose this new
> > API
> > > to plugins and we cannot depend on third party tools as Romain also said
> > >
> > > Okay for annotations
> > >
> > > Enrico
> > >
> > > Something like a shaded version of Microprofile Metrics, where the API
> > can
> > > > consist entirely of Annotations.
> > > > (There are programmatic approaches as well, but I suppose we are
> > > interested
> > > > in limiting the footprint of metrics within the Maven codebase as much
> > as
> > > > possible).
> > > >
> > > > ... implying, of course, that we would need to make a Maven-specific
> > > > implementation as discussed above.
> > > >
> > > > On Tue, Jan 28, 2020 at 10:17 AM Enrico Olivelli <eo...@gmail.com>
> > > > wrote:
> > > >
> > > > > Il giorno mar 28 gen 2020 alle ore 09:15 Romain Manni-Bucau <
> > > > > rmannibucau@gmail.com> ha scritto:
> > > > >
> > > > > > +1 generally since all extensions doing that report wrong data
> > > > > > -1 to use a mainstream impl if not shaded - it would just bring us
> > > new
> > > > > > conflicts i think.
> > > > > >
> > > > >
> > > > > Totally agree.
> > > > > In my view any "impl" will be plugged with an "extension".
> > > > > We will provide only a default no-op implementation
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > >
> > > > > > Side note being: we can start we just counters so no lib is fine
> > > imho.
> > > > > >
> > > > > > Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <
> > eolivelli@gmail.com>
> > > a
> > > > > > écrit :
> > > > > >
> > > > > > > Hi,
> > > > > > > I would like to work on introducing an explicit support in Maven
> > > Core
> > > > > for
> > > > > > > recording "metrics".
> > > > > > > As far as I know (and I have still limited knowledge about the
> > > > > internals)
> > > > > > > you can write an "extension" and you can intercept main lifecycle
> > > > > events,
> > > > > > > and using these hooks you can try to record metrics.
> > > > > > > But my proposal is from another point of view and I want to go
> > > > deeper.
> > > > > > >
> > > > > > > We are adding caches, making refactors....all good things, but we
> > > > > should
> > > > > > > have a way to track how Maven is behaving in a measurable manner.
> > > > > > >
> > > > > > > Usually in order to capture realtime metrics you are using some
> > > > Metrics
> > > > > > > framework, like Prometheus, Dropwizards....and you attach your
> > > > process
> > > > > > to a
> > > > > > > metrics database and capture all of the values.
> > > > > > > Then you can process all of the data, realtime and/or offline,
> > you
> > > > can
> > > > > > > create dashboards, you can also fine tune your configuration or
> > > > change
> > > > > > your
> > > > > > > code.
> > > > > > >
> > > > > > > These frameworks deal with time series created by capturing
> > > > Counters,
> > > > > > > Summaries, Gauges.
> > > > > > >
> > > > > > > I would like to introduce an API to be used by Maven Core and by
> > > > Plugin
> > > > > > to
> > > > > > > expose internal metrics, like:
> > > > > > > - internal caches size
> > > > > > > - memory usage
> > > > > > > - disk reads/writes
> > > > > > > - network traffic
> > > > > > > - any kind of specific metric for plugins (like "number of tests
> > > > > > executed",
> > > > > > > "number of files sent to javac...")
> > > > > > > - .... all of the usual stuff you track in order to understand
> > > wha's
> > > > > > going
> > > > > > > on
> > > > > > >
> > > > > > > I am saying a Maven Metrics API because we must not stick to a
> > > single
> > > > > > > "provider" (say "DropWizard"), it will be a bad choice in the
> > long
> > > > > term.
> > > > > > >
> > > > > > > The API will expose to Maven Core and to plugins a way to get
> > > access
> > > > to
> > > > > > > Metrics and report useful values.
> > > > > > >
> > > > > > > @InjectMe
> > > > > > > MetricsProvider metrics;
> > > > > > >
> > > > > > > metrics.getCounter("javacFiles").add(....)
> > > > > > > metrics.getSummary("downloadTime").add(....)
> > > > > > >
> > > > > > > I don't want to go down into details as this is only a
> > preliminary
> > > > > email
> > > > > > > but in my view we could let the user attach an "implementation"
> > > just
> > > > by
> > > > > > > dropping in an "extension", in order not to ship with Maven Core
> > a
> > > > lot
> > > > > of
> > > > > > > third party dependencies.
> > > > > > >
> > > > > > > I have some "production" experience with this topic (actually I
> > did
> > > > > > > something like that in my company and in ZooKeeper project) so I
> > > will
> > > > > be
> > > > > > > happy to prepare and share a design document.
> > > > > > >
> > > > > > > With this work we will open up the way to knowing how a Maven
> > build
> > > > > > works,
> > > > > > > with the 'time dimension', on the local machine of the developer
> > > but
> > > > > even
> > > > > > > on CI servers.
> > > > > > >
> > > > > > > Any comment is very welcome
> > > > > > >
> > > > > > > Regards
> > > > > > > Enrico
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > --
> > > > +==============================+
> > > > | Bästa hälsningar,
> > > > | [sw. "Best regards"]
> > > > |
> > > > | Lennart Jörelid
> > > > | EAI Architect & Integrator
> > > > |
> > > > | jGuru Europe AB
> > > > | Mölnlycke - Kista
> > > > |
> > > > | Email: lj@jguru.se
> > > > | URL:   www.jguru.se
> > > > | Phone
> > > > | (skype):    jgurueurope
> > > > | (intl):     +46 708 507 603
> > > > | (domestic): 0708 - 507 603
> > > > +==============================+
> > > >
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org

Re: Maven Runtime Metrics

Posted by he...@free.fr.

maven-studies looks like the right location:
https://github.com/apache/maven-studies

Regards,

Hervé

----- Mail original -----
De: "Enrico Olivelli" <eo...@gmail.com>
À: "Maven Developers List" <de...@maven.apache.org>
Envoyé: Jeudi 30 Janvier 2020 04:31:14
Objet: Re: Maven Runtime Metrics

I sound like to draft a prototype.

Any suggestion about which repository should contain this new Metrics API?


Enrico

Il Mar 28 Gen 2020, 13:54 Romain Manni-Bucau <rm...@gmail.com> ha
scritto:

> technically we can shade microprofile, metrics/dropwizard or even sirona
> but maven requires quite a different model. A typical example is the expiry
> or weighted model will not fit the reporting maven needs (very long running
> instances vs "batch" like executions) and the weighted impl can be an issue
> but is spec-ed by MP (so the shaded impl would also have this unexpected
> impl)
>
> What about by just adding counters, enabling to inject them in mojos -
> maybe with a qualifier? - and then see if we need to move to something more
> widely used or not (but my past experiences kind of tend to show it is not
> needed).
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le mar. 28 janv. 2020 à 13:42, Enrico Olivelli <eo...@gmail.com> a
> écrit :
>
> > Il Mar 28 Gen 2020, 13:30 Lennart Jörelid <le...@gmail.com> ha
> > scritto:
> >
> > > Would it be possible to use an existing standard for the specification
> of
> > > such a metrics API?
> > >
> >
> > In my opinion we should have our own model, Maven will impose this new
> API
> > to plugins and we cannot depend on third party tools as Romain also said
> >
> > Okay for annotations
> >
> > Enrico
> >
> > Something like a shaded version of Microprofile Metrics, where the API
> can
> > > consist entirely of Annotations.
> > > (There are programmatic approaches as well, but I suppose we are
> > interested
> > > in limiting the footprint of metrics within the Maven codebase as much
> as
> > > possible).
> > >
> > > ... implying, of course, that we would need to make a Maven-specific
> > > implementation as discussed above.
> > >
> > > On Tue, Jan 28, 2020 at 10:17 AM Enrico Olivelli <eo...@gmail.com>
> > > wrote:
> > >
> > > > Il giorno mar 28 gen 2020 alle ore 09:15 Romain Manni-Bucau <
> > > > rmannibucau@gmail.com> ha scritto:
> > > >
> > > > > +1 generally since all extensions doing that report wrong data
> > > > > -1 to use a mainstream impl if not shaded - it would just bring us
> > new
> > > > > conflicts i think.
> > > > >
> > > >
> > > > Totally agree.
> > > > In my view any "impl" will be plugged with an "extension".
> > > > We will provide only a default no-op implementation
> > > >
> > > > Enrico
> > > >
> > > >
> > > >
> > > > > Side note being: we can start we just counters so no lib is fine
> > imho.
> > > > >
> > > > > Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <
> eolivelli@gmail.com>
> > a
> > > > > écrit :
> > > > >
> > > > > > Hi,
> > > > > > I would like to work on introducing an explicit support in Maven
> > Core
> > > > for
> > > > > > recording "metrics".
> > > > > > As far as I know (and I have still limited knowledge about the
> > > > internals)
> > > > > > you can write an "extension" and you can intercept main lifecycle
> > > > events,
> > > > > > and using these hooks you can try to record metrics.
> > > > > > But my proposal is from another point of view and I want to go
> > > deeper.
> > > > > >
> > > > > > We are adding caches, making refactors....all good things, but we
> > > > should
> > > > > > have a way to track how Maven is behaving in a measurable manner.
> > > > > >
> > > > > > Usually in order to capture realtime metrics you are using some
> > > Metrics
> > > > > > framework, like Prometheus, Dropwizards....and you attach your
> > > process
> > > > > to a
> > > > > > metrics database and capture all of the values.
> > > > > > Then you can process all of the data, realtime and/or offline,
> you
> > > can
> > > > > > create dashboards, you can also fine tune your configuration or
> > > change
> > > > > your
> > > > > > code.
> > > > > >
> > > > > > These frameworks deal with time series created by capturing
> > > Counters,
> > > > > > Summaries, Gauges.
> > > > > >
> > > > > > I would like to introduce an API to be used by Maven Core and by
> > > Plugin
> > > > > to
> > > > > > expose internal metrics, like:
> > > > > > - internal caches size
> > > > > > - memory usage
> > > > > > - disk reads/writes
> > > > > > - network traffic
> > > > > > - any kind of specific metric for plugins (like "number of tests
> > > > > executed",
> > > > > > "number of files sent to javac...")
> > > > > > - .... all of the usual stuff you track in order to understand
> > wha's
> > > > > going
> > > > > > on
> > > > > >
> > > > > > I am saying a Maven Metrics API because we must not stick to a
> > single
> > > > > > "provider" (say "DropWizard"), it will be a bad choice in the
> long
> > > > term.
> > > > > >
> > > > > > The API will expose to Maven Core and to plugins a way to get
> > access
> > > to
> > > > > > Metrics and report useful values.
> > > > > >
> > > > > > @InjectMe
> > > > > > MetricsProvider metrics;
> > > > > >
> > > > > > metrics.getCounter("javacFiles").add(....)
> > > > > > metrics.getSummary("downloadTime").add(....)
> > > > > >
> > > > > > I don't want to go down into details as this is only a
> preliminary
> > > > email
> > > > > > but in my view we could let the user attach an "implementation"
> > just
> > > by
> > > > > > dropping in an "extension", in order not to ship with Maven Core
> a
> > > lot
> > > > of
> > > > > > third party dependencies.
> > > > > >
> > > > > > I have some "production" experience with this topic (actually I
> did
> > > > > > something like that in my company and in ZooKeeper project) so I
> > will
> > > > be
> > > > > > happy to prepare and share a design document.
> > > > > >
> > > > > > With this work we will open up the way to knowing how a Maven
> build
> > > > > works,
> > > > > > with the 'time dimension', on the local machine of the developer
> > but
> > > > even
> > > > > > on CI servers.
> > > > > >
> > > > > > Any comment is very welcome
> > > > > >
> > > > > > Regards
> > > > > > Enrico
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > --
> > > +==============================+
> > > | Bästa hälsningar,
> > > | [sw. "Best regards"]
> > > |
> > > | Lennart Jörelid
> > > | EAI Architect & Integrator
> > > |
> > > | jGuru Europe AB
> > > | Mölnlycke - Kista
> > > |
> > > | Email: lj@jguru.se
> > > | URL:   www.jguru.se
> > > | Phone
> > > | (skype):    jgurueurope
> > > | (intl):     +46 708 507 603
> > > | (domestic): 0708 - 507 603
> > > +==============================+
> > >
> >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org

Re: Maven Runtime Metrics

Posted by Enrico Olivelli <eo...@gmail.com>.

I sound like to draft a prototype.

Any suggestion about which repository should contain this new Metrics API?


Enrico

Il Mar 28 Gen 2020, 13:54 Romain Manni-Bucau <rm...@gmail.com> ha
scritto:

> technically we can shade microprofile, metrics/dropwizard or even sirona
> but maven requires quite a different model. A typical example is the expiry
> or weighted model will not fit the reporting maven needs (very long running
> instances vs "batch" like executions) and the weighted impl can be an issue
> but is spec-ed by MP (so the shaded impl would also have this unexpected
> impl)
>
> What about by just adding counters, enabling to inject them in mojos -
> maybe with a qualifier? - and then see if we need to move to something more
> widely used or not (but my past experiences kind of tend to show it is not
> needed).
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le mar. 28 janv. 2020 à 13:42, Enrico Olivelli <eo...@gmail.com> a
> écrit :
>
> > Il Mar 28 Gen 2020, 13:30 Lennart Jörelid <le...@gmail.com> ha
> > scritto:
> >
> > > Would it be possible to use an existing standard for the specification
> of
> > > such a metrics API?
> > >
> >
> > In my opinion we should have our own model, Maven will impose this new
> API
> > to plugins and we cannot depend on third party tools as Romain also said
> >
> > Okay for annotations
> >
> > Enrico
> >
> > Something like a shaded version of Microprofile Metrics, where the API
> can
> > > consist entirely of Annotations.
> > > (There are programmatic approaches as well, but I suppose we are
> > interested
> > > in limiting the footprint of metrics within the Maven codebase as much
> as
> > > possible).
> > >
> > > ... implying, of course, that we would need to make a Maven-specific
> > > implementation as discussed above.
> > >
> > > On Tue, Jan 28, 2020 at 10:17 AM Enrico Olivelli <eo...@gmail.com>
> > > wrote:
> > >
> > > > Il giorno mar 28 gen 2020 alle ore 09:15 Romain Manni-Bucau <
> > > > rmannibucau@gmail.com> ha scritto:
> > > >
> > > > > +1 generally since all extensions doing that report wrong data
> > > > > -1 to use a mainstream impl if not shaded - it would just bring us
> > new
> > > > > conflicts i think.
> > > > >
> > > >
> > > > Totally agree.
> > > > In my view any "impl" will be plugged with an "extension".
> > > > We will provide only a default no-op implementation
> > > >
> > > > Enrico
> > > >
> > > >
> > > >
> > > > > Side note being: we can start we just counters so no lib is fine
> > imho.
> > > > >
> > > > > Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <
> eolivelli@gmail.com>
> > a
> > > > > écrit :
> > > > >
> > > > > > Hi,
> > > > > > I would like to work on introducing an explicit support in Maven
> > Core
> > > > for
> > > > > > recording "metrics".
> > > > > > As far as I know (and I have still limited knowledge about the
> > > > internals)
> > > > > > you can write an "extension" and you can intercept main lifecycle
> > > > events,
> > > > > > and using these hooks you can try to record metrics.
> > > > > > But my proposal is from another point of view and I want to go
> > > deeper.
> > > > > >
> > > > > > We are adding caches, making refactors....all good things, but we
> > > > should
> > > > > > have a way to track how Maven is behaving in a measurable manner.
> > > > > >
> > > > > > Usually in order to capture realtime metrics you are using some
> > > Metrics
> > > > > > framework, like Prometheus, Dropwizards....and you attach your
> > > process
> > > > > to a
> > > > > > metrics database and capture all of the values.
> > > > > > Then you can process all of the data, realtime and/or offline,
> you
> > > can
> > > > > > create dashboards, you can also fine tune your configuration or
> > > change
> > > > > your
> > > > > > code.
> > > > > >
> > > > > > These frameworks deal with time series created by capturing
> > > Counters,
> > > > > > Summaries, Gauges.
> > > > > >
> > > > > > I would like to introduce an API to be used by Maven Core and by
> > > Plugin
> > > > > to
> > > > > > expose internal metrics, like:
> > > > > > - internal caches size
> > > > > > - memory usage
> > > > > > - disk reads/writes
> > > > > > - network traffic
> > > > > > - any kind of specific metric for plugins (like "number of tests
> > > > > executed",
> > > > > > "number of files sent to javac...")
> > > > > > - .... all of the usual stuff you track in order to understand
> > wha's
> > > > > going
> > > > > > on
> > > > > >
> > > > > > I am saying a Maven Metrics API because we must not stick to a
> > single
> > > > > > "provider" (say "DropWizard"), it will be a bad choice in the
> long
> > > > term.
> > > > > >
> > > > > > The API will expose to Maven Core and to plugins a way to get
> > access
> > > to
> > > > > > Metrics and report useful values.
> > > > > >
> > > > > > @InjectMe
> > > > > > MetricsProvider metrics;
> > > > > >
> > > > > > metrics.getCounter("javacFiles").add(....)
> > > > > > metrics.getSummary("downloadTime").add(....)
> > > > > >
> > > > > > I don't want to go down into details as this is only a
> preliminary
> > > > email
> > > > > > but in my view we could let the user attach an "implementation"
> > just
> > > by
> > > > > > dropping in an "extension", in order not to ship with Maven Core
> a
> > > lot
> > > > of
> > > > > > third party dependencies.
> > > > > >
> > > > > > I have some "production" experience with this topic (actually I
> did
> > > > > > something like that in my company and in ZooKeeper project) so I
> > will
> > > > be
> > > > > > happy to prepare and share a design document.
> > > > > >
> > > > > > With this work we will open up the way to knowing how a Maven
> build
> > > > > works,
> > > > > > with the 'time dimension', on the local machine of the developer
> > but
> > > > even
> > > > > > on CI servers.
> > > > > >
> > > > > > Any comment is very welcome
> > > > > >
> > > > > > Regards
> > > > > > Enrico
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > --
> > > +==============================+
> > > | Bästa hälsningar,
> > > | [sw. "Best regards"]
> > > |
> > > | Lennart Jörelid
> > > | EAI Architect & Integrator
> > > |
> > > | jGuru Europe AB
> > > | Mölnlycke - Kista
> > > |
> > > | Email: lj@jguru.se
> > > | URL:   www.jguru.se
> > > | Phone
> > > | (skype):    jgurueurope
> > > | (intl):     +46 708 507 603
> > > | (domestic): 0708 - 507 603
> > > +==============================+
> > >
> >
>

Re: Maven Runtime Metrics

Posted by Romain Manni-Bucau <rm...@gmail.com>.

technically we can shade microprofile, metrics/dropwizard or even sirona
but maven requires quite a different model. A typical example is the expiry
or weighted model will not fit the reporting maven needs (very long running
instances vs "batch" like executions) and the weighted impl can be an issue
but is spec-ed by MP (so the shaded impl would also have this unexpected
impl)

What about by just adding counters, enabling to inject them in mojos -
maybe with a qualifier? - and then see if we need to move to something more
widely used or not (but my past experiences kind of tend to show it is not
needed).

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mar. 28 janv. 2020 à 13:42, Enrico Olivelli <eo...@gmail.com> a
écrit :

> Il Mar 28 Gen 2020, 13:30 Lennart Jörelid <le...@gmail.com> ha
> scritto:
>
> > Would it be possible to use an existing standard for the specification of
> > such a metrics API?
> >
>
> In my opinion we should have our own model, Maven will impose this new API
> to plugins and we cannot depend on third party tools as Romain also said
>
> Okay for annotations
>
> Enrico
>
> Something like a shaded version of Microprofile Metrics, where the API can
> > consist entirely of Annotations.
> > (There are programmatic approaches as well, but I suppose we are
> interested
> > in limiting the footprint of metrics within the Maven codebase as much as
> > possible).
> >
> > ... implying, of course, that we would need to make a Maven-specific
> > implementation as discussed above.
> >
> > On Tue, Jan 28, 2020 at 10:17 AM Enrico Olivelli <eo...@gmail.com>
> > wrote:
> >
> > > Il giorno mar 28 gen 2020 alle ore 09:15 Romain Manni-Bucau <
> > > rmannibucau@gmail.com> ha scritto:
> > >
> > > > +1 generally since all extensions doing that report wrong data
> > > > -1 to use a mainstream impl if not shaded - it would just bring us
> new
> > > > conflicts i think.
> > > >
> > >
> > > Totally agree.
> > > In my view any "impl" will be plugged with an "extension".
> > > We will provide only a default no-op implementation
> > >
> > > Enrico
> > >
> > >
> > >
> > > > Side note being: we can start we just counters so no lib is fine
> imho.
> > > >
> > > > Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <eo...@gmail.com>
> a
> > > > écrit :
> > > >
> > > > > Hi,
> > > > > I would like to work on introducing an explicit support in Maven
> Core
> > > for
> > > > > recording "metrics".
> > > > > As far as I know (and I have still limited knowledge about the
> > > internals)
> > > > > you can write an "extension" and you can intercept main lifecycle
> > > events,
> > > > > and using these hooks you can try to record metrics.
> > > > > But my proposal is from another point of view and I want to go
> > deeper.
> > > > >
> > > > > We are adding caches, making refactors....all good things, but we
> > > should
> > > > > have a way to track how Maven is behaving in a measurable manner.
> > > > >
> > > > > Usually in order to capture realtime metrics you are using some
> > Metrics
> > > > > framework, like Prometheus, Dropwizards....and you attach your
> > process
> > > > to a
> > > > > metrics database and capture all of the values.
> > > > > Then you can process all of the data, realtime and/or offline, you
> > can
> > > > > create dashboards, you can also fine tune your configuration or
> > change
> > > > your
> > > > > code.
> > > > >
> > > > > These frameworks deal with time series created by capturing
> > Counters,
> > > > > Summaries, Gauges.
> > > > >
> > > > > I would like to introduce an API to be used by Maven Core and by
> > Plugin
> > > > to
> > > > > expose internal metrics, like:
> > > > > - internal caches size
> > > > > - memory usage
> > > > > - disk reads/writes
> > > > > - network traffic
> > > > > - any kind of specific metric for plugins (like "number of tests
> > > > executed",
> > > > > "number of files sent to javac...")
> > > > > - .... all of the usual stuff you track in order to understand
> wha's
> > > > going
> > > > > on
> > > > >
> > > > > I am saying a Maven Metrics API because we must not stick to a
> single
> > > > > "provider" (say "DropWizard"), it will be a bad choice in the long
> > > term.
> > > > >
> > > > > The API will expose to Maven Core and to plugins a way to get
> access
> > to
> > > > > Metrics and report useful values.
> > > > >
> > > > > @InjectMe
> > > > > MetricsProvider metrics;
> > > > >
> > > > > metrics.getCounter("javacFiles").add(....)
> > > > > metrics.getSummary("downloadTime").add(....)
> > > > >
> > > > > I don't want to go down into details as this is only a preliminary
> > > email
> > > > > but in my view we could let the user attach an "implementation"
> just
> > by
> > > > > dropping in an "extension", in order not to ship with Maven Core a
> > lot
> > > of
> > > > > third party dependencies.
> > > > >
> > > > > I have some "production" experience with this topic (actually I did
> > > > > something like that in my company and in ZooKeeper project) so I
> will
> > > be
> > > > > happy to prepare and share a design document.
> > > > >
> > > > > With this work we will open up the way to knowing how a Maven build
> > > > works,
> > > > > with the 'time dimension', on the local machine of the developer
> but
> > > even
> > > > > on CI servers.
> > > > >
> > > > > Any comment is very welcome
> > > > >
> > > > > Regards
> > > > > Enrico
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > --
> > +==============================+
> > | Bästa hälsningar,
> > | [sw. "Best regards"]
> > |
> > | Lennart Jörelid
> > | EAI Architect & Integrator
> > |
> > | jGuru Europe AB
> > | Mölnlycke - Kista
> > |
> > | Email: lj@jguru.se
> > | URL:   www.jguru.se
> > | Phone
> > | (skype):    jgurueurope
> > | (intl):     +46 708 507 603
> > | (domestic): 0708 - 507 603
> > +==============================+
> >
>

Re: Maven Runtime Metrics

Posted by Enrico Olivelli <eo...@gmail.com>.

Il Mar 28 Gen 2020, 13:30 Lennart Jörelid <le...@gmail.com> ha
scritto:

> Would it be possible to use an existing standard for the specification of
> such a metrics API?
>

In my opinion we should have our own model, Maven will impose this new API
to plugins and we cannot depend on third party tools as Romain also said

Okay for annotations

Enrico

Something like a shaded version of Microprofile Metrics, where the API can
> consist entirely of Annotations.
> (There are programmatic approaches as well, but I suppose we are interested
> in limiting the footprint of metrics within the Maven codebase as much as
> possible).
>
> ... implying, of course, that we would need to make a Maven-specific
> implementation as discussed above.
>
> On Tue, Jan 28, 2020 at 10:17 AM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
> > Il giorno mar 28 gen 2020 alle ore 09:15 Romain Manni-Bucau <
> > rmannibucau@gmail.com> ha scritto:
> >
> > > +1 generally since all extensions doing that report wrong data
> > > -1 to use a mainstream impl if not shaded - it would just bring us new
> > > conflicts i think.
> > >
> >
> > Totally agree.
> > In my view any "impl" will be plugged with an "extension".
> > We will provide only a default no-op implementation
> >
> > Enrico
> >
> >
> >
> > > Side note being: we can start we just counters so no lib is fine imho.
> > >
> > > Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <eo...@gmail.com> a
> > > écrit :
> > >
> > > > Hi,
> > > > I would like to work on introducing an explicit support in Maven Core
> > for
> > > > recording "metrics".
> > > > As far as I know (and I have still limited knowledge about the
> > internals)
> > > > you can write an "extension" and you can intercept main lifecycle
> > events,
> > > > and using these hooks you can try to record metrics.
> > > > But my proposal is from another point of view and I want to go
> deeper.
> > > >
> > > > We are adding caches, making refactors....all good things, but we
> > should
> > > > have a way to track how Maven is behaving in a measurable manner.
> > > >
> > > > Usually in order to capture realtime metrics you are using some
> Metrics
> > > > framework, like Prometheus, Dropwizards....and you attach your
> process
> > > to a
> > > > metrics database and capture all of the values.
> > > > Then you can process all of the data, realtime and/or offline, you
> can
> > > > create dashboards, you can also fine tune your configuration or
> change
> > > your
> > > > code.
> > > >
> > > > These frameworks deal with time series created by capturing
> Counters,
> > > > Summaries, Gauges.
> > > >
> > > > I would like to introduce an API to be used by Maven Core and by
> Plugin
> > > to
> > > > expose internal metrics, like:
> > > > - internal caches size
> > > > - memory usage
> > > > - disk reads/writes
> > > > - network traffic
> > > > - any kind of specific metric for plugins (like "number of tests
> > > executed",
> > > > "number of files sent to javac...")
> > > > - .... all of the usual stuff you track in order to understand wha's
> > > going
> > > > on
> > > >
> > > > I am saying a Maven Metrics API because we must not stick to a single
> > > > "provider" (say "DropWizard"), it will be a bad choice in the long
> > term.
> > > >
> > > > The API will expose to Maven Core and to plugins a way to get access
> to
> > > > Metrics and report useful values.
> > > >
> > > > @InjectMe
> > > > MetricsProvider metrics;
> > > >
> > > > metrics.getCounter("javacFiles").add(....)
> > > > metrics.getSummary("downloadTime").add(....)
> > > >
> > > > I don't want to go down into details as this is only a preliminary
> > email
> > > > but in my view we could let the user attach an "implementation" just
> by
> > > > dropping in an "extension", in order not to ship with Maven Core a
> lot
> > of
> > > > third party dependencies.
> > > >
> > > > I have some "production" experience with this topic (actually I did
> > > > something like that in my company and in ZooKeeper project) so I will
> > be
> > > > happy to prepare and share a design document.
> > > >
> > > > With this work we will open up the way to knowing how a Maven build
> > > works,
> > > > with the 'time dimension', on the local machine of the developer but
> > even
> > > > on CI servers.
> > > >
> > > > Any comment is very welcome
> > > >
> > > > Regards
> > > > Enrico
> > > >
> > >
> >
>
>
> --
>
> --
> +==============================+
> | Bästa hälsningar,
> | [sw. "Best regards"]
> |
> | Lennart Jörelid
> | EAI Architect & Integrator
> |
> | jGuru Europe AB
> | Mölnlycke - Kista
> |
> | Email: lj@jguru.se
> | URL:   www.jguru.se
> | Phone
> | (skype):    jgurueurope
> | (intl):     +46 708 507 603
> | (domestic): 0708 - 507 603
> +==============================+
>

Re: Maven Runtime Metrics

Posted by Lennart Jörelid <le...@gmail.com>.

Would it be possible to use an existing standard for the specification of
such a metrics API?
Something like a shaded version of Microprofile Metrics, where the API can
consist entirely of Annotations.
(There are programmatic approaches as well, but I suppose we are interested
in limiting the footprint of metrics within the Maven codebase as much as
possible).

... implying, of course, that we would need to make a Maven-specific
implementation as discussed above.

On Tue, Jan 28, 2020 at 10:17 AM Enrico Olivelli <eo...@gmail.com>
wrote:

> Il giorno mar 28 gen 2020 alle ore 09:15 Romain Manni-Bucau <
> rmannibucau@gmail.com> ha scritto:
>
> > +1 generally since all extensions doing that report wrong data
> > -1 to use a mainstream impl if not shaded - it would just bring us new
> > conflicts i think.
> >
>
> Totally agree.
> In my view any "impl" will be plugged with an "extension".
> We will provide only a default no-op implementation
>
> Enrico
>
>
>
> > Side note being: we can start we just counters so no lib is fine imho.
> >
> > Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <eo...@gmail.com> a
> > écrit :
> >
> > > Hi,
> > > I would like to work on introducing an explicit support in Maven Core
> for
> > > recording "metrics".
> > > As far as I know (and I have still limited knowledge about the
> internals)
> > > you can write an "extension" and you can intercept main lifecycle
> events,
> > > and using these hooks you can try to record metrics.
> > > But my proposal is from another point of view and I want to go deeper.
> > >
> > > We are adding caches, making refactors....all good things, but we
> should
> > > have a way to track how Maven is behaving in a measurable manner.
> > >
> > > Usually in order to capture realtime metrics you are using some Metrics
> > > framework, like Prometheus, Dropwizards....and you attach your process
> > to a
> > > metrics database and capture all of the values.
> > > Then you can process all of the data, realtime and/or offline, you can
> > > create dashboards, you can also fine tune your configuration or change
> > your
> > > code.
> > >
> > > These frameworks deal with time series created by capturing  Counters,
> > > Summaries, Gauges.
> > >
> > > I would like to introduce an API to be used by Maven Core and by Plugin
> > to
> > > expose internal metrics, like:
> > > - internal caches size
> > > - memory usage
> > > - disk reads/writes
> > > - network traffic
> > > - any kind of specific metric for plugins (like "number of tests
> > executed",
> > > "number of files sent to javac...")
> > > - .... all of the usual stuff you track in order to understand wha's
> > going
> > > on
> > >
> > > I am saying a Maven Metrics API because we must not stick to a single
> > > "provider" (say "DropWizard"), it will be a bad choice in the long
> term.
> > >
> > > The API will expose to Maven Core and to plugins a way to get access to
> > > Metrics and report useful values.
> > >
> > > @InjectMe
> > > MetricsProvider metrics;
> > >
> > > metrics.getCounter("javacFiles").add(....)
> > > metrics.getSummary("downloadTime").add(....)
> > >
> > > I don't want to go down into details as this is only a preliminary
> email
> > > but in my view we could let the user attach an "implementation" just by
> > > dropping in an "extension", in order not to ship with Maven Core a lot
> of
> > > third party dependencies.
> > >
> > > I have some "production" experience with this topic (actually I did
> > > something like that in my company and in ZooKeeper project) so I will
> be
> > > happy to prepare and share a design document.
> > >
> > > With this work we will open up the way to knowing how a Maven build
> > works,
> > > with the 'time dimension', on the local machine of the developer but
> even
> > > on CI servers.
> > >
> > > Any comment is very welcome
> > >
> > > Regards
> > > Enrico
> > >
> >
>


-- 

--
+==============================+
| Bästa hälsningar,
| [sw. "Best regards"]
|
| Lennart Jörelid
| EAI Architect & Integrator
|
| jGuru Europe AB
| Mölnlycke - Kista
|
| Email: lj@jguru.se
| URL:   www.jguru.se
| Phone
| (skype):    jgurueurope
| (intl):     +46 708 507 603
| (domestic): 0708 - 507 603
+==============================+

Re: Maven Runtime Metrics

Posted by Enrico Olivelli <eo...@gmail.com>.

Il giorno mar 28 gen 2020 alle ore 09:15 Romain Manni-Bucau <
rmannibucau@gmail.com> ha scritto:

> +1 generally since all extensions doing that report wrong data
> -1 to use a mainstream impl if not shaded - it would just bring us new
> conflicts i think.
>

Totally agree.
In my view any "impl" will be plugged with an "extension".
We will provide only a default no-op implementation

Enrico



> Side note being: we can start we just counters so no lib is fine imho.
>
> Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <eo...@gmail.com> a
> écrit :
>
> > Hi,
> > I would like to work on introducing an explicit support in Maven Core for
> > recording "metrics".
> > As far as I know (and I have still limited knowledge about the internals)
> > you can write an "extension" and you can intercept main lifecycle events,
> > and using these hooks you can try to record metrics.
> > But my proposal is from another point of view and I want to go deeper.
> >
> > We are adding caches, making refactors....all good things, but we should
> > have a way to track how Maven is behaving in a measurable manner.
> >
> > Usually in order to capture realtime metrics you are using some Metrics
> > framework, like Prometheus, Dropwizards....and you attach your process
> to a
> > metrics database and capture all of the values.
> > Then you can process all of the data, realtime and/or offline, you can
> > create dashboards, you can also fine tune your configuration or change
> your
> > code.
> >
> > These frameworks deal with time series created by capturing  Counters,
> > Summaries, Gauges.
> >
> > I would like to introduce an API to be used by Maven Core and by Plugin
> to
> > expose internal metrics, like:
> > - internal caches size
> > - memory usage
> > - disk reads/writes
> > - network traffic
> > - any kind of specific metric for plugins (like "number of tests
> executed",
> > "number of files sent to javac...")
> > - .... all of the usual stuff you track in order to understand wha's
> going
> > on
> >
> > I am saying a Maven Metrics API because we must not stick to a single
> > "provider" (say "DropWizard"), it will be a bad choice in the long term.
> >
> > The API will expose to Maven Core and to plugins a way to get access to
> > Metrics and report useful values.
> >
> > @InjectMe
> > MetricsProvider metrics;
> >
> > metrics.getCounter("javacFiles").add(....)
> > metrics.getSummary("downloadTime").add(....)
> >
> > I don't want to go down into details as this is only a preliminary email
> > but in my view we could let the user attach an "implementation" just by
> > dropping in an "extension", in order not to ship with Maven Core a lot of
> > third party dependencies.
> >
> > I have some "production" experience with this topic (actually I did
> > something like that in my company and in ZooKeeper project) so I will be
> > happy to prepare and share a design document.
> >
> > With this work we will open up the way to knowing how a Maven build
> works,
> > with the 'time dimension', on the local machine of the developer but even
> > on CI servers.
> >
> > Any comment is very welcome
> >
> > Regards
> > Enrico
> >
>

Re: Maven Runtime Metrics

Posted by Romain Manni-Bucau <rm...@gmail.com>.

+1 generally since all extensions doing that report wrong data
-1 to use a mainstream impl if not shaded - it would just bring us new
conflicts i think.
Side note being: we can start we just counters so no lib is fine imho.

Le mar. 28 janv. 2020 à 08:56, Enrico Olivelli <eo...@gmail.com> a
écrit :

> Hi,
> I would like to work on introducing an explicit support in Maven Core for
> recording "metrics".
> As far as I know (and I have still limited knowledge about the internals)
> you can write an "extension" and you can intercept main lifecycle events,
> and using these hooks you can try to record metrics.
> But my proposal is from another point of view and I want to go deeper.
>
> We are adding caches, making refactors....all good things, but we should
> have a way to track how Maven is behaving in a measurable manner.
>
> Usually in order to capture realtime metrics you are using some Metrics
> framework, like Prometheus, Dropwizards....and you attach your process to a
> metrics database and capture all of the values.
> Then you can process all of the data, realtime and/or offline, you can
> create dashboards, you can also fine tune your configuration or change your
> code.
>
> These frameworks deal with time series created by capturing  Counters,
> Summaries, Gauges.
>
> I would like to introduce an API to be used by Maven Core and by Plugin to
> expose internal metrics, like:
> - internal caches size
> - memory usage
> - disk reads/writes
> - network traffic
> - any kind of specific metric for plugins (like "number of tests executed",
> "number of files sent to javac...")
> - .... all of the usual stuff you track in order to understand wha's going
> on
>
> I am saying a Maven Metrics API because we must not stick to a single
> "provider" (say "DropWizard"), it will be a bad choice in the long term.
>
> The API will expose to Maven Core and to plugins a way to get access to
> Metrics and report useful values.
>
> @InjectMe
> MetricsProvider metrics;
>
> metrics.getCounter("javacFiles").add(....)
> metrics.getSummary("downloadTime").add(....)
>
> I don't want to go down into details as this is only a preliminary email
> but in my view we could let the user attach an "implementation" just by
> dropping in an "extension", in order not to ship with Maven Core a lot of
> third party dependencies.
>
> I have some "production" experience with this topic (actually I did
> something like that in my company and in ZooKeeper project) so I will be
> happy to prepare and share a design document.
>
> With this work we will open up the way to knowing how a Maven build works,
> with the 'time dimension', on the local machine of the developer but even
> on CI servers.
>
> Any comment is very welcome
>
> Regards
> Enrico
>