[GitHub] [qpid-proton] astitcher opened a new pull request #219: Memory usage tracking for proton-c

[GitBox <gi...@apache.org>]

astitcher opened a new pull request #219: Memory usage tracking for proton-c
URL: https://github.com/apache/qpid-proton/pull/219
 
 
   This code is preliminary code to allow tracking of memory use in proton-c by class of memory object being used.
   
   It tracks the various classes of memory allocated by proton-c - the number of allocations/deallocation & sizes of these for each individual memory class.
   
   It tracks separately the memory used in the object instances themselves - direct memory and memory allocated in variable buffers by the objects - indirect memory use. For some classes of memory the direct/indirect distinction is not that clear and this code does the best it can without undue overhead.
   
   This system is attached to the newly introduced logging system and memory usage stats will be dumped using the logger on program exit. Details appear in the DEBUG level and the summary in the INFO level. The application can request a memory dump by calling `pn_mem_dump_stats()`.
   
   Memory tracing can be made *more* active by adding "memory" to the PN_LOG environment variable. In this case it will dump memory stats when SIGUSR1 is sent to the process - this is only on Linux or similar as there is no equivalent on Windows.
   
   The code works well enough for basic tracking where the memory goes, however it has 2 ares that need further work before it can be safely merged to master:
   * A measure of the performance impact - this code intercepts the memory allocation/deallocation paths so is potentially used a lot. It uses a single table to hold its stats so this might be a caching bottleneck in a big multi threaded application.
   * There is currently no attempt to ensure that the stats update is thread safe - this could in theory lead to inconsistent updates of the stats. Making this correct either requires locking the operations, using atomic operations or keeping a stats table per thread. All of these options potentially have a (perhaps significant) performance impact.
   
   The related issue is PROTON-2144

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org