[GitHub] raphw edited a comment on issue #1666: Huge memory is used when using skywalking-agent.

[GitBox <gi...@apache.org>]

raphw edited a comment on issue #1666: Huge memory is used when using skywalking-agent.
URL: https://github.com/apache/incubator-skywalking/issues/1666#issuecomment-422414252
 
 
   I had a look at the dump and there is no memory leak in my understanding. However, the skywalking agent is [configured a bit too naively](https://github.com/apache/incubator-skywalking/blob/5e8410834de20254c2bdab2d86cdbfd79d600904/apm-sniffer/apm-agent/src/main/java/org/apache/skywalking/apm/agent/SkyWalkingAgent.java#L67) and will unlikely tackle huge loads. By using the default `BinaryLocator`, Byte Buddy needs to read class files from disc upon any class loading when the transformer is invoked. This will generate a huge load of I/O activity and therefore the observed traffic.
   
   I have several suggestions on how this can be improved:
   1. It is important to filter by "cheap" categories first, if possible. If you can exclude a namespace, Byte Buddy can discard a class for transformation without parsing the class. If you require complex properties, Byte Buddy might however need to parse multiple class files if this is necessary to process a matcher.
   2. It is also recommended to exclude classes that are known to be irrelevant to an agent, e.g. classes for reflection or on certain class loaders.
   3. It is important for performance and for reducing memory consumption, to cache type descriptions by defining a custom type strategy that applies some caching. This way I/O processing can be reduced by a large margin. Note that caching offers an actual opportunity for a memory leak, there needs to be some form of eviction.
   
   The "leak" that is observed is rather slow GC due to jar file streams being expensive to collect. As they use finallizers, they need two collections and stay around for much longer as they should.
   
   There are other possibilities to improve the use of Byte Buddy. There is not currently a good manual on this but I am planning to prepare a talk on this topic, also for explaining more on how to use Advice.
   
   For a good configuration of Byte Buddy that is also open source, have a look at [Stage Monitor](https://github.com/stagemonitor/stagemonitor/blob/6804a0fe506c3f7fd210f8c05332d9648fbf3fce/stagemonitor-core/src/main/java/org/stagemonitor/core/instrument/AgentAttacher.java#L230) which has tackled most challanges.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services