You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by Clebert Suconic <cl...@gmail.com> on 2022/11/04 20:51:48 UTC
DISCUSS: Memory Leak Test...
I have a pet project that I have kept on hold for many years... but I
see a need in Artemis... and perhaps I could use this as a push to
make it happen..
https://github.com/clebertsuconic/d-sect
The project is a JVMTI C Agent, that I can use to fetch objects,
references and do some reports. It's a lot simpler to write tests to
avoid memory leaks with this.. and you can do things like this:
@Test
void test() {
... do your test thing.. and then.
JVMTIInterface jvmti = new JVMTIInterface();
Object[] objects = jvmti.getAllObjects("mypackage.MyClass");
Assert.assertEquals("My class is leaking", objects.length);
}
There are reporting methods that will show what would eventually be
holding the references.
What would be a good way to consume this in ActiveMQ Artemis? We
should add a few tests like this but I'm not sure how we would consume
the library in there?
--
Clebert Suconic
Re: DISCUSS: Memory Leak Test...
Posted by Clebert Suconic <cl...@gmail.com>.
I meant to also send this JIRA:
https://issues.sonatype.org/browse/OSSRH-88708
On Fri, Feb 10, 2023 at 4:24 PM Clebert Suconic
<cl...@gmail.com> wrote:
>
> Just for the record: the public repo has been created already. I need
> to make a first release which will happen soon.
>
>
> The project has been renamed as check-leak (Thanks Gary for the
> name... I like it!)
>
> https://github.com/check-leak/check-leak
>
> On Wed, Feb 8, 2023 at 10:21 PM Clebert Suconic
> <cl...@gmail.com> wrote:
> >
> > sorry.. correct URI for the project:
> >
> > https://github.com/d-leak/d-leak
> >
> > On Wed, Feb 8, 2023 at 10:20 PM Clebert Suconic
> > <cl...@gmail.com> wrote:
> > >
> > > So.. this is it!!
> > >
> > > https://github.com/apache/activemq-artemis/pull/4363
> > >
> > >
> > > It is using jitpack.io (Thanks Domenico)...
> > >
> > >
> > > I have created a maven plugin that will install the required library,
> > > and the new project ./test/leak-tests is working seamlessly... You
> > > just run the test.. and that's it.
> > >
> > >
> > > I have the project hosted at github.com/d-leak/dleak
> > >
> > >
> > > I still need to do some cleanup on the API, and add a README. but the
> > > version 0.2 is already functional for what I needed here.
> > >
> > > Le me know what you guys think of this..
> > >
> > > On Wed, Feb 8, 2023 at 5:50 PM Clebert Suconic
> > > <cl...@gmail.com> wrote:
> > > >
> > > > Actually I misunderstood GitHub packages. I thought it would allow public access and I just realized Domenico said it doesn’t.
> > > >
> > > >
> > > > Now I’m scared of Jitpack. I don’t need to do anything !!! Just reference the package and it will work !!!!!!???
> > > >
> > > >
> > > > I guess I will start with jitpack for now.
> > > >
> > > > On Wed, Feb 8, 2023 at 3:28 PM Clebert Suconic <cl...@gmail.com> wrote:
> > > >>
> > > >> Just an update... I did some work on this again...
> > > >>
> > > >>
> > > >> and instead of using sonatype, it would be easy enough to use github
> > > >> maven packaging...
> > > >>
> > > >>
> > > >> and this is working like a charm actually:
> > > >>
> > > >> https://github.com/d-leak
> > > >>
> > > >>
> > > >>
> > > >> I am writing tests that are validating
> > > >>
> > > >> JVMInterface jvmti = new JVMTIInterface();
> > > >> Assert.assertEquals(0,
> > > >> jvmti.getAllObjects("packageName.ServerConnection").length);
> > > >>
> > > >>
> > > >> If someone ever forget to clear a reference again, this test will catch that...
> > > >>
> > > >>
> > > >> I hope more someone else will make use of this, because it feels cool :)
> > > >>
> > > >> On Wed, Nov 9, 2022 at 6:25 PM Arthur Naseef <ar...@amlinv.com> wrote:
> > > >> >
> > > >> > Yeah, Sontatype OSSRH is a little involved to get going, but not too bad.
> > > >> > I use it (for example
> > > >> > https://mvnrepository.com/artifact/com.artnaseef/correlation-id-utils).
> > > >> >
> > > >> > On Mon, Nov 7, 2022 at 4:20 AM Robbie Gemmell <ro...@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > You can release stuff to Maven Central via Sonatype OSSRH,
> > > >> > > https://central.sonatype.org/publish/publish-guide/
> > > >> > >
> > > >> > > For example by using your GitHub Pages details for the groupid
> > > >> > > coordinates i.e. io.github.your-id, among other options:
> > > >> > >
> > > >> > > https://central.sonatype.org/publish/requirements/coordinates/#supported-code-hosting-services-for-personal-groupid
> > > >> > >
> > > >> > > On Sun, 6 Nov 2022 at 10:10, Domenico Francesco Bruscino
> > > >> > > <br...@gmail.com> wrote:
> > > >> > > >
> > > >> > > > JitPack is free for open-source projects and it doesn't require
> > > >> > > > authentication to download artifacts, see https://jitpack.io/
> > > >> > > >
> > > >> > > > Another free alternative for open-source projects is GitHub Packages but
> > > >> > > it
> > > >> > > > requires authentication, see
> > > >> > > >
> > > >> > > https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry
> > > >> > > >
> > > >> > > >
> > > >> > > > On Sat, 5 Nov 2022 at 17:15, Clebert Suconic <cl...@gmail.com>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > getAllReachableObjects is somewhat easy to implement in C.
> > > >> > > > >
> > > >> > > > > I will try to pursue my own implementation.. and if we fail we have a
> > > >> > > > > failback on the jetbrains route.
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > there are some reporting things I'm doing to show where are eventual
> > > >> > > leaks.
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > I worked on a test with Justin, and we asserted noObjects in the heap
> > > >> > > > > and it worked like a charm.
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > I just need to find a way to deploy a maven repo somewhere.
> > > >> > > > >
> > > >> > > > > On Sat, Nov 5, 2022 at 11:13 AM Jiri Daněk <jd...@redhat.com> wrote:
> > > >> > > > > >
> > > >> > > > > > On Sat, Nov 5, 2022 at 4:00 PM Clebert Suconic <
> > > >> > > > > clebert.suconic@gmail.com>
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > There are of course tools that would been written before. I have
> > > >> > > not
> > > >> > > > > > > seen anything that would be as simple as I'm proposing (with the
> > > >> > > > > > > exception of the .NET where you can get a list of objects).
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > > > Yes! It is weird that C#, Python, JavaScript, all have a simple way
> > > >> > > to
> > > >> > > > > list
> > > >> > > > > > heap objects, only in Java it seems soo hard to do.
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > > What I'm doing is based on the JBoss Profiler that I wrote many
> > > >> > > years
> > > >> > > > > > > ago (and I left it rotten and dead)... so this would revive the
> > > >> > > simple
> > > >> > > > > > > parts just for the JUnit Tests.
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > > > Yeah, I linked it in my previous mail. I was thinking that the code
> > > >> > > looks
> > > >> > > > > > somewhat similar, except it's in C and not C++...
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > > You just get a simple list of Objects before and later.
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > Regarding the criticism to JMVTI.. I have it already written,
> > > >> > > besides
> > > >> > > > > > > any tooling around Java and Debugging will require C dev...
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > the project itself already exists.. and I think it's viable. I am
> > > >> > > > > > > looking for options to host it, while you seem to be looking for
> > > >> > > > > > > counter arguments for having it done?
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > > > I'm just writing up the results of my explorations. Initially I
> > > >> > > hoped to
> > > >> > > > > > learn what is possible. Best case would be I found something that's
> > > >> > > > > already
> > > >> > > > > > written and hosted, and that does not include native code (think all
> > > >> > > > > those
> > > >> > > > > > possible build/compatibility issues).... that did not really
> > > >> > > happen...
> > > >> > > > > >
> > > >> > > > > > Some tests can be written using the weakref approach, but that has
> > > >> > > > > limited
> > > >> > > > > > use for only specific scenarios. The dumping and reading of hprof
> > > >> > > file
> > > >> > > > > > might work, but I did not find a good Java implementation (Shark is
> > > >> > > in
> > > >> > > > > > Kotlin). The jetbrains JVMTI library looks reasonable, on a first
> > > >> > > sight,
> > > >> > > > > > though.
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > > if you give me an alternative in Java where i can do
> > > >> > > > > > > jvmti.getallObject(clazz); and eventually list the references for
> > > >> > > it..
> > > >> > > > > > > perhaps I could just use that option instead. Such thing will be
> > > >> > > > > > > written in C anyways.
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > > > https://github.com/JetBrains/debugger-memory-agent
> > > >> > > > > >
> > > >> > > > > > MemoryAgent ma = MemoryAgent.get();
> > > >> > > > > > if (ma == null) { /* handle failure to load */ }
> > > >> > > > > > ma.getAllReachableObjects(null, clazz);
> > > >> > > > > > --
> > > >> > > > > > Mit freundlichen Grüßen / Kind regards
> > > >> > > > > > Jiri Daněk
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > --
> > > >> > > > > Clebert Suconic
> > > >> > > > >
> > > >> > >
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Clebert Suconic
> > > >
> > > > --
> > > > Clebert Suconic
> > >
> > >
> > >
> > > --
> > > Clebert Suconic
> >
> >
> >
> > --
> > Clebert Suconic
>
>
>
> --
> Clebert Suconic
--
Clebert Suconic
Re: DISCUSS: Memory Leak Test...
Posted by Clebert Suconic <cl...@gmail.com>.
Just for the record: the public repo has been created already. I need
to make a first release which will happen soon.
The project has been renamed as check-leak (Thanks Gary for the
name... I like it!)
https://github.com/check-leak/check-leak
On Wed, Feb 8, 2023 at 10:21 PM Clebert Suconic
<cl...@gmail.com> wrote:
>
> sorry.. correct URI for the project:
>
> https://github.com/d-leak/d-leak
>
> On Wed, Feb 8, 2023 at 10:20 PM Clebert Suconic
> <cl...@gmail.com> wrote:
> >
> > So.. this is it!!
> >
> > https://github.com/apache/activemq-artemis/pull/4363
> >
> >
> > It is using jitpack.io (Thanks Domenico)...
> >
> >
> > I have created a maven plugin that will install the required library,
> > and the new project ./test/leak-tests is working seamlessly... You
> > just run the test.. and that's it.
> >
> >
> > I have the project hosted at github.com/d-leak/dleak
> >
> >
> > I still need to do some cleanup on the API, and add a README. but the
> > version 0.2 is already functional for what I needed here.
> >
> > Le me know what you guys think of this..
> >
> > On Wed, Feb 8, 2023 at 5:50 PM Clebert Suconic
> > <cl...@gmail.com> wrote:
> > >
> > > Actually I misunderstood GitHub packages. I thought it would allow public access and I just realized Domenico said it doesn’t.
> > >
> > >
> > > Now I’m scared of Jitpack. I don’t need to do anything !!! Just reference the package and it will work !!!!!!???
> > >
> > >
> > > I guess I will start with jitpack for now.
> > >
> > > On Wed, Feb 8, 2023 at 3:28 PM Clebert Suconic <cl...@gmail.com> wrote:
> > >>
> > >> Just an update... I did some work on this again...
> > >>
> > >>
> > >> and instead of using sonatype, it would be easy enough to use github
> > >> maven packaging...
> > >>
> > >>
> > >> and this is working like a charm actually:
> > >>
> > >> https://github.com/d-leak
> > >>
> > >>
> > >>
> > >> I am writing tests that are validating
> > >>
> > >> JVMInterface jvmti = new JVMTIInterface();
> > >> Assert.assertEquals(0,
> > >> jvmti.getAllObjects("packageName.ServerConnection").length);
> > >>
> > >>
> > >> If someone ever forget to clear a reference again, this test will catch that...
> > >>
> > >>
> > >> I hope more someone else will make use of this, because it feels cool :)
> > >>
> > >> On Wed, Nov 9, 2022 at 6:25 PM Arthur Naseef <ar...@amlinv.com> wrote:
> > >> >
> > >> > Yeah, Sontatype OSSRH is a little involved to get going, but not too bad.
> > >> > I use it (for example
> > >> > https://mvnrepository.com/artifact/com.artnaseef/correlation-id-utils).
> > >> >
> > >> > On Mon, Nov 7, 2022 at 4:20 AM Robbie Gemmell <ro...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > You can release stuff to Maven Central via Sonatype OSSRH,
> > >> > > https://central.sonatype.org/publish/publish-guide/
> > >> > >
> > >> > > For example by using your GitHub Pages details for the groupid
> > >> > > coordinates i.e. io.github.your-id, among other options:
> > >> > >
> > >> > > https://central.sonatype.org/publish/requirements/coordinates/#supported-code-hosting-services-for-personal-groupid
> > >> > >
> > >> > > On Sun, 6 Nov 2022 at 10:10, Domenico Francesco Bruscino
> > >> > > <br...@gmail.com> wrote:
> > >> > > >
> > >> > > > JitPack is free for open-source projects and it doesn't require
> > >> > > > authentication to download artifacts, see https://jitpack.io/
> > >> > > >
> > >> > > > Another free alternative for open-source projects is GitHub Packages but
> > >> > > it
> > >> > > > requires authentication, see
> > >> > > >
> > >> > > https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry
> > >> > > >
> > >> > > >
> > >> > > > On Sat, 5 Nov 2022 at 17:15, Clebert Suconic <cl...@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > getAllReachableObjects is somewhat easy to implement in C.
> > >> > > > >
> > >> > > > > I will try to pursue my own implementation.. and if we fail we have a
> > >> > > > > failback on the jetbrains route.
> > >> > > > >
> > >> > > > >
> > >> > > > > there are some reporting things I'm doing to show where are eventual
> > >> > > leaks.
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > I worked on a test with Justin, and we asserted noObjects in the heap
> > >> > > > > and it worked like a charm.
> > >> > > > >
> > >> > > > >
> > >> > > > > I just need to find a way to deploy a maven repo somewhere.
> > >> > > > >
> > >> > > > > On Sat, Nov 5, 2022 at 11:13 AM Jiri Daněk <jd...@redhat.com> wrote:
> > >> > > > > >
> > >> > > > > > On Sat, Nov 5, 2022 at 4:00 PM Clebert Suconic <
> > >> > > > > clebert.suconic@gmail.com>
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > There are of course tools that would been written before. I have
> > >> > > not
> > >> > > > > > > seen anything that would be as simple as I'm proposing (with the
> > >> > > > > > > exception of the .NET where you can get a list of objects).
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > > Yes! It is weird that C#, Python, JavaScript, all have a simple way
> > >> > > to
> > >> > > > > list
> > >> > > > > > heap objects, only in Java it seems soo hard to do.
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > > What I'm doing is based on the JBoss Profiler that I wrote many
> > >> > > years
> > >> > > > > > > ago (and I left it rotten and dead)... so this would revive the
> > >> > > simple
> > >> > > > > > > parts just for the JUnit Tests.
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > > Yeah, I linked it in my previous mail. I was thinking that the code
> > >> > > looks
> > >> > > > > > somewhat similar, except it's in C and not C++...
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > > You just get a simple list of Objects before and later.
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > Regarding the criticism to JMVTI.. I have it already written,
> > >> > > besides
> > >> > > > > > > any tooling around Java and Debugging will require C dev...
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > the project itself already exists.. and I think it's viable. I am
> > >> > > > > > > looking for options to host it, while you seem to be looking for
> > >> > > > > > > counter arguments for having it done?
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > > I'm just writing up the results of my explorations. Initially I
> > >> > > hoped to
> > >> > > > > > learn what is possible. Best case would be I found something that's
> > >> > > > > already
> > >> > > > > > written and hosted, and that does not include native code (think all
> > >> > > > > those
> > >> > > > > > possible build/compatibility issues).... that did not really
> > >> > > happen...
> > >> > > > > >
> > >> > > > > > Some tests can be written using the weakref approach, but that has
> > >> > > > > limited
> > >> > > > > > use for only specific scenarios. The dumping and reading of hprof
> > >> > > file
> > >> > > > > > might work, but I did not find a good Java implementation (Shark is
> > >> > > in
> > >> > > > > > Kotlin). The jetbrains JVMTI library looks reasonable, on a first
> > >> > > sight,
> > >> > > > > > though.
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > > if you give me an alternative in Java where i can do
> > >> > > > > > > jvmti.getallObject(clazz); and eventually list the references for
> > >> > > it..
> > >> > > > > > > perhaps I could just use that option instead. Such thing will be
> > >> > > > > > > written in C anyways.
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > > https://github.com/JetBrains/debugger-memory-agent
> > >> > > > > >
> > >> > > > > > MemoryAgent ma = MemoryAgent.get();
> > >> > > > > > if (ma == null) { /* handle failure to load */ }
> > >> > > > > > ma.getAllReachableObjects(null, clazz);
> > >> > > > > > --
> > >> > > > > > Mit freundlichen Grüßen / Kind regards
> > >> > > > > > Jiri Daněk
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > Clebert Suconic
> > >> > > > >
> > >> > >
> > >>
> > >>
> > >>
> > >> --
> > >> Clebert Suconic
> > >
> > > --
> > > Clebert Suconic
> >
> >
> >
> > --
> > Clebert Suconic
>
>
>
> --
> Clebert Suconic
--
Clebert Suconic
Re: DISCUSS: Memory Leak Test...
Posted by Clebert Suconic <cl...@gmail.com>.
sorry.. correct URI for the project:
https://github.com/d-leak/d-leak
On Wed, Feb 8, 2023 at 10:20 PM Clebert Suconic
<cl...@gmail.com> wrote:
>
> So.. this is it!!
>
> https://github.com/apache/activemq-artemis/pull/4363
>
>
> It is using jitpack.io (Thanks Domenico)...
>
>
> I have created a maven plugin that will install the required library,
> and the new project ./test/leak-tests is working seamlessly... You
> just run the test.. and that's it.
>
>
> I have the project hosted at github.com/d-leak/dleak
>
>
> I still need to do some cleanup on the API, and add a README. but the
> version 0.2 is already functional for what I needed here.
>
> Le me know what you guys think of this..
>
> On Wed, Feb 8, 2023 at 5:50 PM Clebert Suconic
> <cl...@gmail.com> wrote:
> >
> > Actually I misunderstood GitHub packages. I thought it would allow public access and I just realized Domenico said it doesn’t.
> >
> >
> > Now I’m scared of Jitpack. I don’t need to do anything !!! Just reference the package and it will work !!!!!!???
> >
> >
> > I guess I will start with jitpack for now.
> >
> > On Wed, Feb 8, 2023 at 3:28 PM Clebert Suconic <cl...@gmail.com> wrote:
> >>
> >> Just an update... I did some work on this again...
> >>
> >>
> >> and instead of using sonatype, it would be easy enough to use github
> >> maven packaging...
> >>
> >>
> >> and this is working like a charm actually:
> >>
> >> https://github.com/d-leak
> >>
> >>
> >>
> >> I am writing tests that are validating
> >>
> >> JVMInterface jvmti = new JVMTIInterface();
> >> Assert.assertEquals(0,
> >> jvmti.getAllObjects("packageName.ServerConnection").length);
> >>
> >>
> >> If someone ever forget to clear a reference again, this test will catch that...
> >>
> >>
> >> I hope more someone else will make use of this, because it feels cool :)
> >>
> >> On Wed, Nov 9, 2022 at 6:25 PM Arthur Naseef <ar...@amlinv.com> wrote:
> >> >
> >> > Yeah, Sontatype OSSRH is a little involved to get going, but not too bad.
> >> > I use it (for example
> >> > https://mvnrepository.com/artifact/com.artnaseef/correlation-id-utils).
> >> >
> >> > On Mon, Nov 7, 2022 at 4:20 AM Robbie Gemmell <ro...@gmail.com>
> >> > wrote:
> >> >
> >> > > You can release stuff to Maven Central via Sonatype OSSRH,
> >> > > https://central.sonatype.org/publish/publish-guide/
> >> > >
> >> > > For example by using your GitHub Pages details for the groupid
> >> > > coordinates i.e. io.github.your-id, among other options:
> >> > >
> >> > > https://central.sonatype.org/publish/requirements/coordinates/#supported-code-hosting-services-for-personal-groupid
> >> > >
> >> > > On Sun, 6 Nov 2022 at 10:10, Domenico Francesco Bruscino
> >> > > <br...@gmail.com> wrote:
> >> > > >
> >> > > > JitPack is free for open-source projects and it doesn't require
> >> > > > authentication to download artifacts, see https://jitpack.io/
> >> > > >
> >> > > > Another free alternative for open-source projects is GitHub Packages but
> >> > > it
> >> > > > requires authentication, see
> >> > > >
> >> > > https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry
> >> > > >
> >> > > >
> >> > > > On Sat, 5 Nov 2022 at 17:15, Clebert Suconic <cl...@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > getAllReachableObjects is somewhat easy to implement in C.
> >> > > > >
> >> > > > > I will try to pursue my own implementation.. and if we fail we have a
> >> > > > > failback on the jetbrains route.
> >> > > > >
> >> > > > >
> >> > > > > there are some reporting things I'm doing to show where are eventual
> >> > > leaks.
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > I worked on a test with Justin, and we asserted noObjects in the heap
> >> > > > > and it worked like a charm.
> >> > > > >
> >> > > > >
> >> > > > > I just need to find a way to deploy a maven repo somewhere.
> >> > > > >
> >> > > > > On Sat, Nov 5, 2022 at 11:13 AM Jiri Daněk <jd...@redhat.com> wrote:
> >> > > > > >
> >> > > > > > On Sat, Nov 5, 2022 at 4:00 PM Clebert Suconic <
> >> > > > > clebert.suconic@gmail.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > There are of course tools that would been written before. I have
> >> > > not
> >> > > > > > > seen anything that would be as simple as I'm proposing (with the
> >> > > > > > > exception of the .NET where you can get a list of objects).
> >> > > > > > >
> >> > > > > >
> >> > > > > > Yes! It is weird that C#, Python, JavaScript, all have a simple way
> >> > > to
> >> > > > > list
> >> > > > > > heap objects, only in Java it seems soo hard to do.
> >> > > > > >
> >> > > > > >
> >> > > > > > > What I'm doing is based on the JBoss Profiler that I wrote many
> >> > > years
> >> > > > > > > ago (and I left it rotten and dead)... so this would revive the
> >> > > simple
> >> > > > > > > parts just for the JUnit Tests.
> >> > > > > > >
> >> > > > > >
> >> > > > > > Yeah, I linked it in my previous mail. I was thinking that the code
> >> > > looks
> >> > > > > > somewhat similar, except it's in C and not C++...
> >> > > > > >
> >> > > > > >
> >> > > > > > > You just get a simple list of Objects before and later.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > Regarding the criticism to JMVTI.. I have it already written,
> >> > > besides
> >> > > > > > > any tooling around Java and Debugging will require C dev...
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > the project itself already exists.. and I think it's viable. I am
> >> > > > > > > looking for options to host it, while you seem to be looking for
> >> > > > > > > counter arguments for having it done?
> >> > > > > > >
> >> > > > > >
> >> > > > > > I'm just writing up the results of my explorations. Initially I
> >> > > hoped to
> >> > > > > > learn what is possible. Best case would be I found something that's
> >> > > > > already
> >> > > > > > written and hosted, and that does not include native code (think all
> >> > > > > those
> >> > > > > > possible build/compatibility issues).... that did not really
> >> > > happen...
> >> > > > > >
> >> > > > > > Some tests can be written using the weakref approach, but that has
> >> > > > > limited
> >> > > > > > use for only specific scenarios. The dumping and reading of hprof
> >> > > file
> >> > > > > > might work, but I did not find a good Java implementation (Shark is
> >> > > in
> >> > > > > > Kotlin). The jetbrains JVMTI library looks reasonable, on a first
> >> > > sight,
> >> > > > > > though.
> >> > > > > >
> >> > > > > >
> >> > > > > > > if you give me an alternative in Java where i can do
> >> > > > > > > jvmti.getallObject(clazz); and eventually list the references for
> >> > > it..
> >> > > > > > > perhaps I could just use that option instead. Such thing will be
> >> > > > > > > written in C anyways.
> >> > > > > > >
> >> > > > > >
> >> > > > > > https://github.com/JetBrains/debugger-memory-agent
> >> > > > > >
> >> > > > > > MemoryAgent ma = MemoryAgent.get();
> >> > > > > > if (ma == null) { /* handle failure to load */ }
> >> > > > > > ma.getAllReachableObjects(null, clazz);
> >> > > > > > --
> >> > > > > > Mit freundlichen Grüßen / Kind regards
> >> > > > > > Jiri Daněk
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Clebert Suconic
> >> > > > >
> >> > >
> >>
> >>
> >>
> >> --
> >> Clebert Suconic
> >
> > --
> > Clebert Suconic
>
>
>
> --
> Clebert Suconic
--
Clebert Suconic
Re: DISCUSS: Memory Leak Test...
Posted by Clebert Suconic <cl...@gmail.com>.
So.. this is it!!
https://github.com/apache/activemq-artemis/pull/4363
It is using jitpack.io (Thanks Domenico)...
I have created a maven plugin that will install the required library,
and the new project ./test/leak-tests is working seamlessly... You
just run the test.. and that's it.
I have the project hosted at github.com/d-leak/dleak
I still need to do some cleanup on the API, and add a README. but the
version 0.2 is already functional for what I needed here.
Le me know what you guys think of this..
On Wed, Feb 8, 2023 at 5:50 PM Clebert Suconic
<cl...@gmail.com> wrote:
>
> Actually I misunderstood GitHub packages. I thought it would allow public access and I just realized Domenico said it doesn’t.
>
>
> Now I’m scared of Jitpack. I don’t need to do anything !!! Just reference the package and it will work !!!!!!???
>
>
> I guess I will start with jitpack for now.
>
> On Wed, Feb 8, 2023 at 3:28 PM Clebert Suconic <cl...@gmail.com> wrote:
>>
>> Just an update... I did some work on this again...
>>
>>
>> and instead of using sonatype, it would be easy enough to use github
>> maven packaging...
>>
>>
>> and this is working like a charm actually:
>>
>> https://github.com/d-leak
>>
>>
>>
>> I am writing tests that are validating
>>
>> JVMInterface jvmti = new JVMTIInterface();
>> Assert.assertEquals(0,
>> jvmti.getAllObjects("packageName.ServerConnection").length);
>>
>>
>> If someone ever forget to clear a reference again, this test will catch that...
>>
>>
>> I hope more someone else will make use of this, because it feels cool :)
>>
>> On Wed, Nov 9, 2022 at 6:25 PM Arthur Naseef <ar...@amlinv.com> wrote:
>> >
>> > Yeah, Sontatype OSSRH is a little involved to get going, but not too bad.
>> > I use it (for example
>> > https://mvnrepository.com/artifact/com.artnaseef/correlation-id-utils).
>> >
>> > On Mon, Nov 7, 2022 at 4:20 AM Robbie Gemmell <ro...@gmail.com>
>> > wrote:
>> >
>> > > You can release stuff to Maven Central via Sonatype OSSRH,
>> > > https://central.sonatype.org/publish/publish-guide/
>> > >
>> > > For example by using your GitHub Pages details for the groupid
>> > > coordinates i.e. io.github.your-id, among other options:
>> > >
>> > > https://central.sonatype.org/publish/requirements/coordinates/#supported-code-hosting-services-for-personal-groupid
>> > >
>> > > On Sun, 6 Nov 2022 at 10:10, Domenico Francesco Bruscino
>> > > <br...@gmail.com> wrote:
>> > > >
>> > > > JitPack is free for open-source projects and it doesn't require
>> > > > authentication to download artifacts, see https://jitpack.io/
>> > > >
>> > > > Another free alternative for open-source projects is GitHub Packages but
>> > > it
>> > > > requires authentication, see
>> > > >
>> > > https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry
>> > > >
>> > > >
>> > > > On Sat, 5 Nov 2022 at 17:15, Clebert Suconic <cl...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > getAllReachableObjects is somewhat easy to implement in C.
>> > > > >
>> > > > > I will try to pursue my own implementation.. and if we fail we have a
>> > > > > failback on the jetbrains route.
>> > > > >
>> > > > >
>> > > > > there are some reporting things I'm doing to show where are eventual
>> > > leaks.
>> > > > >
>> > > > >
>> > > > >
>> > > > > I worked on a test with Justin, and we asserted noObjects in the heap
>> > > > > and it worked like a charm.
>> > > > >
>> > > > >
>> > > > > I just need to find a way to deploy a maven repo somewhere.
>> > > > >
>> > > > > On Sat, Nov 5, 2022 at 11:13 AM Jiri Daněk <jd...@redhat.com> wrote:
>> > > > > >
>> > > > > > On Sat, Nov 5, 2022 at 4:00 PM Clebert Suconic <
>> > > > > clebert.suconic@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > There are of course tools that would been written before. I have
>> > > not
>> > > > > > > seen anything that would be as simple as I'm proposing (with the
>> > > > > > > exception of the .NET where you can get a list of objects).
>> > > > > > >
>> > > > > >
>> > > > > > Yes! It is weird that C#, Python, JavaScript, all have a simple way
>> > > to
>> > > > > list
>> > > > > > heap objects, only in Java it seems soo hard to do.
>> > > > > >
>> > > > > >
>> > > > > > > What I'm doing is based on the JBoss Profiler that I wrote many
>> > > years
>> > > > > > > ago (and I left it rotten and dead)... so this would revive the
>> > > simple
>> > > > > > > parts just for the JUnit Tests.
>> > > > > > >
>> > > > > >
>> > > > > > Yeah, I linked it in my previous mail. I was thinking that the code
>> > > looks
>> > > > > > somewhat similar, except it's in C and not C++...
>> > > > > >
>> > > > > >
>> > > > > > > You just get a simple list of Objects before and later.
>> > > > > > >
>> > > > > > >
>> > > > > > > Regarding the criticism to JMVTI.. I have it already written,
>> > > besides
>> > > > > > > any tooling around Java and Debugging will require C dev...
>> > > > > > >
>> > > > > > >
>> > > > > > > the project itself already exists.. and I think it's viable. I am
>> > > > > > > looking for options to host it, while you seem to be looking for
>> > > > > > > counter arguments for having it done?
>> > > > > > >
>> > > > > >
>> > > > > > I'm just writing up the results of my explorations. Initially I
>> > > hoped to
>> > > > > > learn what is possible. Best case would be I found something that's
>> > > > > already
>> > > > > > written and hosted, and that does not include native code (think all
>> > > > > those
>> > > > > > possible build/compatibility issues).... that did not really
>> > > happen...
>> > > > > >
>> > > > > > Some tests can be written using the weakref approach, but that has
>> > > > > limited
>> > > > > > use for only specific scenarios. The dumping and reading of hprof
>> > > file
>> > > > > > might work, but I did not find a good Java implementation (Shark is
>> > > in
>> > > > > > Kotlin). The jetbrains JVMTI library looks reasonable, on a first
>> > > sight,
>> > > > > > though.
>> > > > > >
>> > > > > >
>> > > > > > > if you give me an alternative in Java where i can do
>> > > > > > > jvmti.getallObject(clazz); and eventually list the references for
>> > > it..
>> > > > > > > perhaps I could just use that option instead. Such thing will be
>> > > > > > > written in C anyways.
>> > > > > > >
>> > > > > >
>> > > > > > https://github.com/JetBrains/debugger-memory-agent
>> > > > > >
>> > > > > > MemoryAgent ma = MemoryAgent.get();
>> > > > > > if (ma == null) { /* handle failure to load */ }
>> > > > > > ma.getAllReachableObjects(null, clazz);
>> > > > > > --
>> > > > > > Mit freundlichen Grüßen / Kind regards
>> > > > > > Jiri Daněk
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Clebert Suconic
>> > > > >
>> > >
>>
>>
>>
>> --
>> Clebert Suconic
>
> --
> Clebert Suconic
--
Clebert Suconic
Re: DISCUSS: Memory Leak Test...
Posted by Clebert Suconic <cl...@gmail.com>.
Actually I misunderstood GitHub packages. I thought it would allow public
access and I just realized Domenico said it doesn’t.
Now I’m scared of Jitpack. I don’t need to do anything !!! Just reference
the package and it will work !!!!!!???
I guess I will start with jitpack for now.
On Wed, Feb 8, 2023 at 3:28 PM Clebert Suconic <cl...@gmail.com>
wrote:
> Just an update... I did some work on this again...
>
>
> and instead of using sonatype, it would be easy enough to use github
> maven packaging...
>
>
> and this is working like a charm actually:
>
> https://github.com/d-leak
>
>
>
> I am writing tests that are validating
>
> JVMInterface jvmti = new JVMTIInterface();
> Assert.assertEquals(0,
> jvmti.getAllObjects("packageName.ServerConnection").length);
>
>
> If someone ever forget to clear a reference again, this test will catch
> that...
>
>
> I hope more someone else will make use of this, because it feels cool :)
>
> On Wed, Nov 9, 2022 at 6:25 PM Arthur Naseef <ar...@amlinv.com> wrote:
> >
> > Yeah, Sontatype OSSRH is a little involved to get going, but not too bad.
> > I use it (for example
> > https://mvnrepository.com/artifact/com.artnaseef/correlation-id-utils).
> >
> > On Mon, Nov 7, 2022 at 4:20 AM Robbie Gemmell <ro...@gmail.com>
> > wrote:
> >
> > > You can release stuff to Maven Central via Sonatype OSSRH,
> > > https://central.sonatype.org/publish/publish-guide/
> > >
> > > For example by using your GitHub Pages details for the groupid
> > > coordinates i.e. io.github.your-id, among other options:
> > >
> > >
> https://central.sonatype.org/publish/requirements/coordinates/#supported-code-hosting-services-for-personal-groupid
> > >
> > > On Sun, 6 Nov 2022 at 10:10, Domenico Francesco Bruscino
> > > <br...@gmail.com> wrote:
> > > >
> > > > JitPack is free for open-source projects and it doesn't require
> > > > authentication to download artifacts, see https://jitpack.io/
> > > >
> > > > Another free alternative for open-source projects is GitHub Packages
> but
> > > it
> > > > requires authentication, see
> > > >
> > >
> https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry
> > > >
> > > >
> > > > On Sat, 5 Nov 2022 at 17:15, Clebert Suconic <
> clebert.suconic@gmail.com>
> > > > wrote:
> > > >
> > > > > getAllReachableObjects is somewhat easy to implement in C.
> > > > >
> > > > > I will try to pursue my own implementation.. and if we fail we
> have a
> > > > > failback on the jetbrains route.
> > > > >
> > > > >
> > > > > there are some reporting things I'm doing to show where are
> eventual
> > > leaks.
> > > > >
> > > > >
> > > > >
> > > > > I worked on a test with Justin, and we asserted noObjects in the
> heap
> > > > > and it worked like a charm.
> > > > >
> > > > >
> > > > > I just need to find a way to deploy a maven repo somewhere.
> > > > >
> > > > > On Sat, Nov 5, 2022 at 11:13 AM Jiri Daněk <jd...@redhat.com>
> wrote:
> > > > > >
> > > > > > On Sat, Nov 5, 2022 at 4:00 PM Clebert Suconic <
> > > > > clebert.suconic@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > There are of course tools that would been written before. I
> have
> > > not
> > > > > > > seen anything that would be as simple as I'm proposing (with
> the
> > > > > > > exception of the .NET where you can get a list of objects).
> > > > > > >
> > > > > >
> > > > > > Yes! It is weird that C#, Python, JavaScript, all have a simple
> way
> > > to
> > > > > list
> > > > > > heap objects, only in Java it seems soo hard to do.
> > > > > >
> > > > > >
> > > > > > > What I'm doing is based on the JBoss Profiler that I wrote many
> > > years
> > > > > > > ago (and I left it rotten and dead)... so this would revive the
> > > simple
> > > > > > > parts just for the JUnit Tests.
> > > > > > >
> > > > > >
> > > > > > Yeah, I linked it in my previous mail. I was thinking that the
> code
> > > looks
> > > > > > somewhat similar, except it's in C and not C++...
> > > > > >
> > > > > >
> > > > > > > You just get a simple list of Objects before and later.
> > > > > > >
> > > > > > >
> > > > > > > Regarding the criticism to JMVTI.. I have it already written,
> > > besides
> > > > > > > any tooling around Java and Debugging will require C dev...
> > > > > > >
> > > > > > >
> > > > > > > the project itself already exists.. and I think it's viable. I
> am
> > > > > > > looking for options to host it, while you seem to be looking
> for
> > > > > > > counter arguments for having it done?
> > > > > > >
> > > > > >
> > > > > > I'm just writing up the results of my explorations. Initially I
> > > hoped to
> > > > > > learn what is possible. Best case would be I found something
> that's
> > > > > already
> > > > > > written and hosted, and that does not include native code (think
> all
> > > > > those
> > > > > > possible build/compatibility issues).... that did not really
> > > happen...
> > > > > >
> > > > > > Some tests can be written using the weakref approach, but that
> has
> > > > > limited
> > > > > > use for only specific scenarios. The dumping and reading of hprof
> > > file
> > > > > > might work, but I did not find a good Java implementation (Shark
> is
> > > in
> > > > > > Kotlin). The jetbrains JVMTI library looks reasonable, on a first
> > > sight,
> > > > > > though.
> > > > > >
> > > > > >
> > > > > > > if you give me an alternative in Java where i can do
> > > > > > > jvmti.getallObject(clazz); and eventually list the references
> for
> > > it..
> > > > > > > perhaps I could just use that option instead. Such thing will
> be
> > > > > > > written in C anyways.
> > > > > > >
> > > > > >
> > > > > > https://github.com/JetBrains/debugger-memory-agent
> > > > > >
> > > > > > MemoryAgent ma = MemoryAgent.get();
> > > > > > if (ma == null) { /* handle failure to load */ }
> > > > > > ma.getAllReachableObjects(null, clazz);
> > > > > > --
> > > > > > Mit freundlichen Grüßen / Kind regards
> > > > > > Jiri Daněk
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Clebert Suconic
> > > > >
> > >
>
>
>
> --
> Clebert Suconic
>
--
Clebert Suconic
Re: DISCUSS: Memory Leak Test...
Posted by Clebert Suconic <cl...@gmail.com>.
Just an update... I did some work on this again...
and instead of using sonatype, it would be easy enough to use github
maven packaging...
and this is working like a charm actually:
https://github.com/d-leak
I am writing tests that are validating
JVMInterface jvmti = new JVMTIInterface();
Assert.assertEquals(0,
jvmti.getAllObjects("packageName.ServerConnection").length);
If someone ever forget to clear a reference again, this test will catch that...
I hope more someone else will make use of this, because it feels cool :)
On Wed, Nov 9, 2022 at 6:25 PM Arthur Naseef <ar...@amlinv.com> wrote:
>
> Yeah, Sontatype OSSRH is a little involved to get going, but not too bad.
> I use it (for example
> https://mvnrepository.com/artifact/com.artnaseef/correlation-id-utils).
>
> On Mon, Nov 7, 2022 at 4:20 AM Robbie Gemmell <ro...@gmail.com>
> wrote:
>
> > You can release stuff to Maven Central via Sonatype OSSRH,
> > https://central.sonatype.org/publish/publish-guide/
> >
> > For example by using your GitHub Pages details for the groupid
> > coordinates i.e. io.github.your-id, among other options:
> >
> > https://central.sonatype.org/publish/requirements/coordinates/#supported-code-hosting-services-for-personal-groupid
> >
> > On Sun, 6 Nov 2022 at 10:10, Domenico Francesco Bruscino
> > <br...@gmail.com> wrote:
> > >
> > > JitPack is free for open-source projects and it doesn't require
> > > authentication to download artifacts, see https://jitpack.io/
> > >
> > > Another free alternative for open-source projects is GitHub Packages but
> > it
> > > requires authentication, see
> > >
> > https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry
> > >
> > >
> > > On Sat, 5 Nov 2022 at 17:15, Clebert Suconic <cl...@gmail.com>
> > > wrote:
> > >
> > > > getAllReachableObjects is somewhat easy to implement in C.
> > > >
> > > > I will try to pursue my own implementation.. and if we fail we have a
> > > > failback on the jetbrains route.
> > > >
> > > >
> > > > there are some reporting things I'm doing to show where are eventual
> > leaks.
> > > >
> > > >
> > > >
> > > > I worked on a test with Justin, and we asserted noObjects in the heap
> > > > and it worked like a charm.
> > > >
> > > >
> > > > I just need to find a way to deploy a maven repo somewhere.
> > > >
> > > > On Sat, Nov 5, 2022 at 11:13 AM Jiri Daněk <jd...@redhat.com> wrote:
> > > > >
> > > > > On Sat, Nov 5, 2022 at 4:00 PM Clebert Suconic <
> > > > clebert.suconic@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > There are of course tools that would been written before. I have
> > not
> > > > > > seen anything that would be as simple as I'm proposing (with the
> > > > > > exception of the .NET where you can get a list of objects).
> > > > > >
> > > > >
> > > > > Yes! It is weird that C#, Python, JavaScript, all have a simple way
> > to
> > > > list
> > > > > heap objects, only in Java it seems soo hard to do.
> > > > >
> > > > >
> > > > > > What I'm doing is based on the JBoss Profiler that I wrote many
> > years
> > > > > > ago (and I left it rotten and dead)... so this would revive the
> > simple
> > > > > > parts just for the JUnit Tests.
> > > > > >
> > > > >
> > > > > Yeah, I linked it in my previous mail. I was thinking that the code
> > looks
> > > > > somewhat similar, except it's in C and not C++...
> > > > >
> > > > >
> > > > > > You just get a simple list of Objects before and later.
> > > > > >
> > > > > >
> > > > > > Regarding the criticism to JMVTI.. I have it already written,
> > besides
> > > > > > any tooling around Java and Debugging will require C dev...
> > > > > >
> > > > > >
> > > > > > the project itself already exists.. and I think it's viable. I am
> > > > > > looking for options to host it, while you seem to be looking for
> > > > > > counter arguments for having it done?
> > > > > >
> > > > >
> > > > > I'm just writing up the results of my explorations. Initially I
> > hoped to
> > > > > learn what is possible. Best case would be I found something that's
> > > > already
> > > > > written and hosted, and that does not include native code (think all
> > > > those
> > > > > possible build/compatibility issues).... that did not really
> > happen...
> > > > >
> > > > > Some tests can be written using the weakref approach, but that has
> > > > limited
> > > > > use for only specific scenarios. The dumping and reading of hprof
> > file
> > > > > might work, but I did not find a good Java implementation (Shark is
> > in
> > > > > Kotlin). The jetbrains JVMTI library looks reasonable, on a first
> > sight,
> > > > > though.
> > > > >
> > > > >
> > > > > > if you give me an alternative in Java where i can do
> > > > > > jvmti.getallObject(clazz); and eventually list the references for
> > it..
> > > > > > perhaps I could just use that option instead. Such thing will be
> > > > > > written in C anyways.
> > > > > >
> > > > >
> > > > > https://github.com/JetBrains/debugger-memory-agent
> > > > >
> > > > > MemoryAgent ma = MemoryAgent.get();
> > > > > if (ma == null) { /* handle failure to load */ }
> > > > > ma.getAllReachableObjects(null, clazz);
> > > > > --
> > > > > Mit freundlichen Grüßen / Kind regards
> > > > > Jiri Daněk
> > > >
> > > >
> > > >
> > > > --
> > > > Clebert Suconic
> > > >
> >
--
Clebert Suconic
Re: DISCUSS: Memory Leak Test...
Posted by Arthur Naseef <ar...@amlinv.com>.
Yeah, Sontatype OSSRH is a little involved to get going, but not too bad.
I use it (for example
https://mvnrepository.com/artifact/com.artnaseef/correlation-id-utils).
On Mon, Nov 7, 2022 at 4:20 AM Robbie Gemmell <ro...@gmail.com>
wrote:
> You can release stuff to Maven Central via Sonatype OSSRH,
> https://central.sonatype.org/publish/publish-guide/
>
> For example by using your GitHub Pages details for the groupid
> coordinates i.e. io.github.your-id, among other options:
>
> https://central.sonatype.org/publish/requirements/coordinates/#supported-code-hosting-services-for-personal-groupid
>
> On Sun, 6 Nov 2022 at 10:10, Domenico Francesco Bruscino
> <br...@gmail.com> wrote:
> >
> > JitPack is free for open-source projects and it doesn't require
> > authentication to download artifacts, see https://jitpack.io/
> >
> > Another free alternative for open-source projects is GitHub Packages but
> it
> > requires authentication, see
> >
> https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry
> >
> >
> > On Sat, 5 Nov 2022 at 17:15, Clebert Suconic <cl...@gmail.com>
> > wrote:
> >
> > > getAllReachableObjects is somewhat easy to implement in C.
> > >
> > > I will try to pursue my own implementation.. and if we fail we have a
> > > failback on the jetbrains route.
> > >
> > >
> > > there are some reporting things I'm doing to show where are eventual
> leaks.
> > >
> > >
> > >
> > > I worked on a test with Justin, and we asserted noObjects in the heap
> > > and it worked like a charm.
> > >
> > >
> > > I just need to find a way to deploy a maven repo somewhere.
> > >
> > > On Sat, Nov 5, 2022 at 11:13 AM Jiri Daněk <jd...@redhat.com> wrote:
> > > >
> > > > On Sat, Nov 5, 2022 at 4:00 PM Clebert Suconic <
> > > clebert.suconic@gmail.com>
> > > > wrote:
> > > >
> > > > > There are of course tools that would been written before. I have
> not
> > > > > seen anything that would be as simple as I'm proposing (with the
> > > > > exception of the .NET where you can get a list of objects).
> > > > >
> > > >
> > > > Yes! It is weird that C#, Python, JavaScript, all have a simple way
> to
> > > list
> > > > heap objects, only in Java it seems soo hard to do.
> > > >
> > > >
> > > > > What I'm doing is based on the JBoss Profiler that I wrote many
> years
> > > > > ago (and I left it rotten and dead)... so this would revive the
> simple
> > > > > parts just for the JUnit Tests.
> > > > >
> > > >
> > > > Yeah, I linked it in my previous mail. I was thinking that the code
> looks
> > > > somewhat similar, except it's in C and not C++...
> > > >
> > > >
> > > > > You just get a simple list of Objects before and later.
> > > > >
> > > > >
> > > > > Regarding the criticism to JMVTI.. I have it already written,
> besides
> > > > > any tooling around Java and Debugging will require C dev...
> > > > >
> > > > >
> > > > > the project itself already exists.. and I think it's viable. I am
> > > > > looking for options to host it, while you seem to be looking for
> > > > > counter arguments for having it done?
> > > > >
> > > >
> > > > I'm just writing up the results of my explorations. Initially I
> hoped to
> > > > learn what is possible. Best case would be I found something that's
> > > already
> > > > written and hosted, and that does not include native code (think all
> > > those
> > > > possible build/compatibility issues).... that did not really
> happen...
> > > >
> > > > Some tests can be written using the weakref approach, but that has
> > > limited
> > > > use for only specific scenarios. The dumping and reading of hprof
> file
> > > > might work, but I did not find a good Java implementation (Shark is
> in
> > > > Kotlin). The jetbrains JVMTI library looks reasonable, on a first
> sight,
> > > > though.
> > > >
> > > >
> > > > > if you give me an alternative in Java where i can do
> > > > > jvmti.getallObject(clazz); and eventually list the references for
> it..
> > > > > perhaps I could just use that option instead. Such thing will be
> > > > > written in C anyways.
> > > > >
> > > >
> > > > https://github.com/JetBrains/debugger-memory-agent
> > > >
> > > > MemoryAgent ma = MemoryAgent.get();
> > > > if (ma == null) { /* handle failure to load */ }
> > > > ma.getAllReachableObjects(null, clazz);
> > > > --
> > > > Mit freundlichen Grüßen / Kind regards
> > > > Jiri Daněk
> > >
> > >
> > >
> > > --
> > > Clebert Suconic
> > >
>
Re: DISCUSS: Memory Leak Test...
Posted by Robbie Gemmell <ro...@gmail.com>.
You can release stuff to Maven Central via Sonatype OSSRH,
https://central.sonatype.org/publish/publish-guide/
For example by using your GitHub Pages details for the groupid
coordinates i.e. io.github.your-id, among other options:
https://central.sonatype.org/publish/requirements/coordinates/#supported-code-hosting-services-for-personal-groupid
On Sun, 6 Nov 2022 at 10:10, Domenico Francesco Bruscino
<br...@gmail.com> wrote:
>
> JitPack is free for open-source projects and it doesn't require
> authentication to download artifacts, see https://jitpack.io/
>
> Another free alternative for open-source projects is GitHub Packages but it
> requires authentication, see
> https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry
>
>
> On Sat, 5 Nov 2022 at 17:15, Clebert Suconic <cl...@gmail.com>
> wrote:
>
> > getAllReachableObjects is somewhat easy to implement in C.
> >
> > I will try to pursue my own implementation.. and if we fail we have a
> > failback on the jetbrains route.
> >
> >
> > there are some reporting things I'm doing to show where are eventual leaks.
> >
> >
> >
> > I worked on a test with Justin, and we asserted noObjects in the heap
> > and it worked like a charm.
> >
> >
> > I just need to find a way to deploy a maven repo somewhere.
> >
> > On Sat, Nov 5, 2022 at 11:13 AM Jiri Daněk <jd...@redhat.com> wrote:
> > >
> > > On Sat, Nov 5, 2022 at 4:00 PM Clebert Suconic <
> > clebert.suconic@gmail.com>
> > > wrote:
> > >
> > > > There are of course tools that would been written before. I have not
> > > > seen anything that would be as simple as I'm proposing (with the
> > > > exception of the .NET where you can get a list of objects).
> > > >
> > >
> > > Yes! It is weird that C#, Python, JavaScript, all have a simple way to
> > list
> > > heap objects, only in Java it seems soo hard to do.
> > >
> > >
> > > > What I'm doing is based on the JBoss Profiler that I wrote many years
> > > > ago (and I left it rotten and dead)... so this would revive the simple
> > > > parts just for the JUnit Tests.
> > > >
> > >
> > > Yeah, I linked it in my previous mail. I was thinking that the code looks
> > > somewhat similar, except it's in C and not C++...
> > >
> > >
> > > > You just get a simple list of Objects before and later.
> > > >
> > > >
> > > > Regarding the criticism to JMVTI.. I have it already written, besides
> > > > any tooling around Java and Debugging will require C dev...
> > > >
> > > >
> > > > the project itself already exists.. and I think it's viable. I am
> > > > looking for options to host it, while you seem to be looking for
> > > > counter arguments for having it done?
> > > >
> > >
> > > I'm just writing up the results of my explorations. Initially I hoped to
> > > learn what is possible. Best case would be I found something that's
> > already
> > > written and hosted, and that does not include native code (think all
> > those
> > > possible build/compatibility issues).... that did not really happen...
> > >
> > > Some tests can be written using the weakref approach, but that has
> > limited
> > > use for only specific scenarios. The dumping and reading of hprof file
> > > might work, but I did not find a good Java implementation (Shark is in
> > > Kotlin). The jetbrains JVMTI library looks reasonable, on a first sight,
> > > though.
> > >
> > >
> > > > if you give me an alternative in Java where i can do
> > > > jvmti.getallObject(clazz); and eventually list the references for it..
> > > > perhaps I could just use that option instead. Such thing will be
> > > > written in C anyways.
> > > >
> > >
> > > https://github.com/JetBrains/debugger-memory-agent
> > >
> > > MemoryAgent ma = MemoryAgent.get();
> > > if (ma == null) { /* handle failure to load */ }
> > > ma.getAllReachableObjects(null, clazz);
> > > --
> > > Mit freundlichen Grüßen / Kind regards
> > > Jiri Daněk
> >
> >
> >
> > --
> > Clebert Suconic
> >
Re: DISCUSS: Memory Leak Test...
Posted by Domenico Francesco Bruscino <br...@gmail.com>.
JitPack is free for open-source projects and it doesn't require
authentication to download artifacts, see https://jitpack.io/
Another free alternative for open-source projects is GitHub Packages but it
requires authentication, see
https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry
On Sat, 5 Nov 2022 at 17:15, Clebert Suconic <cl...@gmail.com>
wrote:
> getAllReachableObjects is somewhat easy to implement in C.
>
> I will try to pursue my own implementation.. and if we fail we have a
> failback on the jetbrains route.
>
>
> there are some reporting things I'm doing to show where are eventual leaks.
>
>
>
> I worked on a test with Justin, and we asserted noObjects in the heap
> and it worked like a charm.
>
>
> I just need to find a way to deploy a maven repo somewhere.
>
> On Sat, Nov 5, 2022 at 11:13 AM Jiri Daněk <jd...@redhat.com> wrote:
> >
> > On Sat, Nov 5, 2022 at 4:00 PM Clebert Suconic <
> clebert.suconic@gmail.com>
> > wrote:
> >
> > > There are of course tools that would been written before. I have not
> > > seen anything that would be as simple as I'm proposing (with the
> > > exception of the .NET where you can get a list of objects).
> > >
> >
> > Yes! It is weird that C#, Python, JavaScript, all have a simple way to
> list
> > heap objects, only in Java it seems soo hard to do.
> >
> >
> > > What I'm doing is based on the JBoss Profiler that I wrote many years
> > > ago (and I left it rotten and dead)... so this would revive the simple
> > > parts just for the JUnit Tests.
> > >
> >
> > Yeah, I linked it in my previous mail. I was thinking that the code looks
> > somewhat similar, except it's in C and not C++...
> >
> >
> > > You just get a simple list of Objects before and later.
> > >
> > >
> > > Regarding the criticism to JMVTI.. I have it already written, besides
> > > any tooling around Java and Debugging will require C dev...
> > >
> > >
> > > the project itself already exists.. and I think it's viable. I am
> > > looking for options to host it, while you seem to be looking for
> > > counter arguments for having it done?
> > >
> >
> > I'm just writing up the results of my explorations. Initially I hoped to
> > learn what is possible. Best case would be I found something that's
> already
> > written and hosted, and that does not include native code (think all
> those
> > possible build/compatibility issues).... that did not really happen...
> >
> > Some tests can be written using the weakref approach, but that has
> limited
> > use for only specific scenarios. The dumping and reading of hprof file
> > might work, but I did not find a good Java implementation (Shark is in
> > Kotlin). The jetbrains JVMTI library looks reasonable, on a first sight,
> > though.
> >
> >
> > > if you give me an alternative in Java where i can do
> > > jvmti.getallObject(clazz); and eventually list the references for it..
> > > perhaps I could just use that option instead. Such thing will be
> > > written in C anyways.
> > >
> >
> > https://github.com/JetBrains/debugger-memory-agent
> >
> > MemoryAgent ma = MemoryAgent.get();
> > if (ma == null) { /* handle failure to load */ }
> > ma.getAllReachableObjects(null, clazz);
> > --
> > Mit freundlichen Grüßen / Kind regards
> > Jiri Daněk
>
>
>
> --
> Clebert Suconic
>
Re: DISCUSS: Memory Leak Test...
Posted by Clebert Suconic <cl...@gmail.com>.
getAllReachableObjects is somewhat easy to implement in C.
I will try to pursue my own implementation.. and if we fail we have a
failback on the jetbrains route.
there are some reporting things I'm doing to show where are eventual leaks.
I worked on a test with Justin, and we asserted noObjects in the heap
and it worked like a charm.
I just need to find a way to deploy a maven repo somewhere.
On Sat, Nov 5, 2022 at 11:13 AM Jiri Daněk <jd...@redhat.com> wrote:
>
> On Sat, Nov 5, 2022 at 4:00 PM Clebert Suconic <cl...@gmail.com>
> wrote:
>
> > There are of course tools that would been written before. I have not
> > seen anything that would be as simple as I'm proposing (with the
> > exception of the .NET where you can get a list of objects).
> >
>
> Yes! It is weird that C#, Python, JavaScript, all have a simple way to list
> heap objects, only in Java it seems soo hard to do.
>
>
> > What I'm doing is based on the JBoss Profiler that I wrote many years
> > ago (and I left it rotten and dead)... so this would revive the simple
> > parts just for the JUnit Tests.
> >
>
> Yeah, I linked it in my previous mail. I was thinking that the code looks
> somewhat similar, except it's in C and not C++...
>
>
> > You just get a simple list of Objects before and later.
> >
> >
> > Regarding the criticism to JMVTI.. I have it already written, besides
> > any tooling around Java and Debugging will require C dev...
> >
> >
> > the project itself already exists.. and I think it's viable. I am
> > looking for options to host it, while you seem to be looking for
> > counter arguments for having it done?
> >
>
> I'm just writing up the results of my explorations. Initially I hoped to
> learn what is possible. Best case would be I found something that's already
> written and hosted, and that does not include native code (think all those
> possible build/compatibility issues).... that did not really happen...
>
> Some tests can be written using the weakref approach, but that has limited
> use for only specific scenarios. The dumping and reading of hprof file
> might work, but I did not find a good Java implementation (Shark is in
> Kotlin). The jetbrains JVMTI library looks reasonable, on a first sight,
> though.
>
>
> > if you give me an alternative in Java where i can do
> > jvmti.getallObject(clazz); and eventually list the references for it..
> > perhaps I could just use that option instead. Such thing will be
> > written in C anyways.
> >
>
> https://github.com/JetBrains/debugger-memory-agent
>
> MemoryAgent ma = MemoryAgent.get();
> if (ma == null) { /* handle failure to load */ }
> ma.getAllReachableObjects(null, clazz);
> --
> Mit freundlichen Grüßen / Kind regards
> Jiri Daněk
--
Clebert Suconic
Re: DISCUSS: Memory Leak Test...
Posted by Jiri Daněk <jd...@redhat.com>.
On Sat, Nov 5, 2022 at 4:00 PM Clebert Suconic <cl...@gmail.com>
wrote:
> There are of course tools that would been written before. I have not
> seen anything that would be as simple as I'm proposing (with the
> exception of the .NET where you can get a list of objects).
>
Yes! It is weird that C#, Python, JavaScript, all have a simple way to list
heap objects, only in Java it seems soo hard to do.
> What I'm doing is based on the JBoss Profiler that I wrote many years
> ago (and I left it rotten and dead)... so this would revive the simple
> parts just for the JUnit Tests.
>
Yeah, I linked it in my previous mail. I was thinking that the code looks
somewhat similar, except it's in C and not C++...
> You just get a simple list of Objects before and later.
>
>
> Regarding the criticism to JMVTI.. I have it already written, besides
> any tooling around Java and Debugging will require C dev...
>
>
> the project itself already exists.. and I think it's viable. I am
> looking for options to host it, while you seem to be looking for
> counter arguments for having it done?
>
I'm just writing up the results of my explorations. Initially I hoped to
learn what is possible. Best case would be I found something that's already
written and hosted, and that does not include native code (think all those
possible build/compatibility issues).... that did not really happen...
Some tests can be written using the weakref approach, but that has limited
use for only specific scenarios. The dumping and reading of hprof file
might work, but I did not find a good Java implementation (Shark is in
Kotlin). The jetbrains JVMTI library looks reasonable, on a first sight,
though.
> if you give me an alternative in Java where i can do
> jvmti.getallObject(clazz); and eventually list the references for it..
> perhaps I could just use that option instead. Such thing will be
> written in C anyways.
>
https://github.com/JetBrains/debugger-memory-agent
MemoryAgent ma = MemoryAgent.get();
if (ma == null) { /* handle failure to load */ }
ma.getAllReachableObjects(null, clazz);
--
Mit freundlichen Grüßen / Kind regards
Jiri Daněk
Re: DISCUSS: Memory Leak Test...
Posted by Clebert Suconic <cl...@gmail.com>.
There are of course tools that would been written before. I have not
seen anything that would be as simple as I'm proposing (with the
exception of the .NET where you can get a list of objects).
What I'm doing is based on the JBoss Profiler that I wrote many years
ago (and I left it rotten and dead)... so this would revive the simple
parts just for the JUnit Tests.
You just get a simple list of Objects before and later.
Regarding the criticism to JMVTI.. I have it already written, besides
any tooling around Java and Debugging will require C dev...
the project itself already exists.. and I think it's viable. I am
looking for options to host it, while you seem to be looking for
counter arguments for having it done?
if you give me an alternative in Java where i can do
jvmti.getallObject(clazz); and eventually list the references for it..
perhaps I could just use that option instead. Such thing will be
written in C anyways.
Re: DISCUSS: Memory Leak Test...
Posted by Jiri Daněk <jd...@redhat.com>.
On Fri, Nov 4, 2022 at 10:05 PM Clebert Suconic <cl...@gmail.com>
wrote:
> More importantly.. that goes without saying... If we make this happen,
> I am looking for contributors to share the project with me :)
>
> On Fri, Nov 4, 2022 at 4:51 PM Clebert Suconic
> <cl...@gmail.com> wrote:
> >
> > I have a pet project that I have kept on hold for many years... but I
> > see a need in Artemis... and perhaps I could use this as a push to
> > make it happen..
> >
> > https://github.com/clebertsuconic/d-sect
> >
> >
> > The project is a JVMTI C Agent, that I can use to fetch objects,
> > references and do some reports. It's a lot simpler to write tests to
> > avoid memory leaks with this.. and you can do things like this:
> >
> > @Test
> > void test() {
> >
> >
> > ... do your test thing.. and then.
> >
> >
> > JVMTIInterface jvmti = new JVMTIInterface();
> > Object[] objects = jvmti.getAllObjects("mypackage.MyClass");
> >
> > Assert.assertEquals("My class is leaking", objects.length);
> >
> > }
> >
> > There are reporting methods that will show what would eventually be
> > holding the references.
> >
> >
> > What would be a good way to consume this in ActiveMQ Artemis? We
> > should add a few tests like this but I'm not sure how we would consume
> > the library in there?
>
Isn't there any such library already? Looking for it, I indeed had trouble
finding anything for Java.
In .NET/C# world, there is dotMemory Unit,
https://www.nuget.org/packages/JetBrains.DotMemoryUnit/, (some custom EULA,
not opensource). There are some usage examples in an announcement blog,
https://blog.jetbrains.com/dotnet/2015/03/04/unit-testing-and-memory-profiling-can-they-be-combined/,
and in the docs
https://www.jetbrains.com/help/dotmemory-unit/Introduction.html.
dotMemory.Check(memory =>
Assert.That(memory.GetObjects(where =>
where.Type.Is<Goo>()).ObjectsCount,
Is.EqualTo(0)));
In JavaScript world, there seem to be some research efforts such as
http://plasma-umass.org/BLeak/,
https://cacm.acm.org/magazines/2020/11/248223-bleak/abstract, which
requires writing UI tests that navigate in a webapp in a loop, and compare
objects in heap snapshots after going through the loop several times. There
is also https://www.npmjs.com/package/memlab from Facebook, and looks like
many more.
For Android Java/Kotlin, there is https://square.github.io/leakcanary/,
(ASL-2.0). Written in Kotlin, not sure if it has a reasonable Java API.
They focus on Android, and the main mode of operation is to instrument the
application during e2e testing (and possibly also in production
deployment). It has knowledge of Android application structure to detect
disposed-off application screens that are stuck in memory for some reason.
https://square.github.io/leakcanary/fundamentals-how-leakcanary-works/.
There is some support for usage on the JVM (not Android)
https://square.github.io/leakcanary/recipes/#detecting-leaks-in-jvm-applications.
They have assertNoLeaks() test rule which apparently works only on android,
and they have a ticket wanting to port over this JVM implementation
https://github.com/cashapp/paparazzi/pull/431/files#diff-7ea5ca018df762ee2392a3ca3afe479a5e9212ab7c7bb36c582331deef6448edR20.
This seems to trigger heap dumps into a file on disk and then uses the
Shark library to analyze the dump and do assertions on object counts:
Class.forName("com.sun.management.HotSpotDiagnosticMXBean")
.getMethod("dumpHeap", String::class.java,
Boolean::class.javaPrimitiveType)
.invoke(mBean, fileName, LIVE)
}
And I also found
https://github.com/centic9/commons-test/blob/master/src/main/java/org/dstadler/commons/testing/MemoryLeakVerifier.java,
which works by keeping a list of weak references to the objects that are
supposed to be gc'd at the end of the test, and the final check consists of
verifying that all the weak references have been nulled. (This does not
look practical in case the objects are some internal objects not visible to
the test, of course.)
Some people on StackOverflow dislike the JVM Tool Interface API approach,
claiming it is slow, and of course it requires writing in C/C++,
https://stackoverflow.com/questions/67148994/jvm-getting-the-largest-objects-in-the-heap-programmatically.
There they suggest using OpenJDK-specific api in JMX to get object counts.
Looking for existing JVMTI agents, I found
https://github.com/JetBrains/debugger-memory-agent (ASL-2.0), which is
supposed to provide the Memory view feature in IntelliJ,
https://www.jetbrains.com/help/idea/analyze-objects-in-the-jvm-heap.html,
Looks like IntelliJ and Eclipse need this functionality for their debugger,
so there can hopefully be an opportunity to reuse a maintained
implementation. I was hopeful about
https://github.com/GoogleCloudPlatform/cloud-debug-java/tree/main/src/agent
but they don't appear to focus on inspecting the heap, strangely. There was
something called JBoss Profiler JVMTI 2,
https://anonsvn.jboss.org/repos/jbossprofiler/branches/JBossProfiler2/native/src/main/native/agent.cpp,
but it looks dead.
Exposed API for the debugger-memory-agent in previous paragraph is
https://github.com/JetBrains/debugger-memory-agent/blob/b36a6d807da89d330206067ccc772f1487ca8312/test_data/proxy/src/com/intellij/memory/agent/MemoryAgent.java#L65-L98
Btw, while searching for all the above, I found something they call
Lincheck, https://github.com/Kotlin/kotlinx-lincheck, and
https://blog.jetbrains.com/kotlin/2021/02/how-we-test-concurrent-primitives-in-kotlin-coroutines/.
It is able to run a test multiple times, changing thread interleaving, and
assert that the result is the same every time. It seems to be useful for
small tests only (individual classes, not entire programs). The blogs show
Kotlin, but the library should be just fine testing Java classes.
--
Mit freundlichen Grüßen / Kind regards
Jiri Daněk
Re: DISCUSS: Memory Leak Test...
Posted by Clebert Suconic <cl...@gmail.com>.
More importantly.. that goes without saying... If we make this happen,
I am looking for contributors to share the project with me :)
On Fri, Nov 4, 2022 at 4:51 PM Clebert Suconic
<cl...@gmail.com> wrote:
>
> I have a pet project that I have kept on hold for many years... but I
> see a need in Artemis... and perhaps I could use this as a push to
> make it happen..
>
> https://github.com/clebertsuconic/d-sect
>
>
> The project is a JVMTI C Agent, that I can use to fetch objects,
> references and do some reports. It's a lot simpler to write tests to
> avoid memory leaks with this.. and you can do things like this:
>
> @Test
> void test() {
>
>
> ... do your test thing.. and then.
>
>
> JVMTIInterface jvmti = new JVMTIInterface();
> Object[] objects = jvmti.getAllObjects("mypackage.MyClass");
>
> Assert.assertEquals("My class is leaking", objects.length);
>
> }
>
> There are reporting methods that will show what would eventually be
> holding the references.
>
>
> What would be a good way to consume this in ActiveMQ Artemis? We
> should add a few tests like this but I'm not sure how we would consume
> the library in there?
>
>
>
>
> --
> Clebert Suconic
--
Clebert Suconic