You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@trafficserver.apache.org by GitBox <gi...@apache.org> on 2020/07/17 20:21:20 UTC
[GitHub] [trafficserver] shinrich opened a new pull request #7014: Add memory_profile plugin
shinrich opened a new pull request #7014:
URL: https://github.com/apache/trafficserver/pull/7014
A simple plugin to perform jemalloc ctl operations on the traffic_server running with jemalloc. I found it really useful to control when I wanted to make profile dumps.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] shukitchan commented on pull request #7014: Add memory_profile plugin
Posted by GitBox <gi...@apache.org>.
shukitchan commented on pull request #7014:
URL: https://github.com/apache/trafficserver/pull/7014#issuecomment-660331336
this is a good idea. Will review this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #7014: Add memory_profile plugin
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #7014:
URL: https://github.com/apache/trafficserver/pull/7014#discussion_r457506212
##########
File path: doc/admin-guide/plugins/memory_profile.en.rst
##########
@@ -0,0 +1,106 @@
+Memory_profile Plugin
+*********************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+This plugin listens for plugin msgs and invokes jemalloc control
+operations.
+
+Installation
+============
+
+Add the following line to :file:`plugin.config`::
+
+ memory_profile.so
+
+In addition, |TS| must be able to read jemalloc configuration
+information either through the ``JEMALLOC_CONF`` environment variable
+or via the string sym linked to ``/etc/malloc.conf``.
+
+For example, if the string below is in ``JEMALLOC_CONF`` or in the sym link string, it
+enables profiling and indicates that the memory dump prefix is ``/tmp/jeprof``.::
+
+ prof:true,prof_prefix:/tmp/jeprof
+
+Details on configuration jemalloc options at `<http://jemalloc.net/jemalloc.3.html>`.
+Changes to the configuration in ``JEMALLOC_CONF`` or ``/etc/malloc.conf`` require a process
+restart to pick up.
+
+Plugin Messages
+===============
+
+The plugin responds to the following mesages sent via traffic_ctl.
+
+Message Action
+========== ===================================================================================
+activate Start jemalloc profiling. Useful if prof_active:false was in the configure string.
+
+deactivate Stop jemalloc profiling.
+
+dump If profiling is enabled and active, it will generate a profile dump file.
+
+stats Print jemalloc statistics in traffic.out
+
+The command below sends the stats message to the plugin causing the current statistics to be written to traffic.out::
+
+ traffic_ctl plugin msg memory_profile stats
+
+Example Usage
+=============
+
+If your run time configuration string is::
+
+ prof:true,prof_prefix:/tmp/jeprof:prof_active:false
+
+|TS| has started without profile sampling started. Perhaps you didn't want to profile the start up phase of |TS|. To start
+you need to send the activate message to the plugin::
+
+ traffic_ctl plugin msg memory_profile activate
+
+If your run time configuration string does not indicate that the profiling is not started (e.g. the prof_active field is missing or set to true), you do not
+need to send the activate message.
+
+After waiting sometime for |TS| to gather some memory allocation data, you can send the dump message::
+
+ traffic_ctl plugin msg memory_profile dump
+
+This will cause a file containing information about the current state of the |TS| memory allocation to be dumped in a file prefixed
+by the value of prof_prefix. In this example, it would be something like ``/tmp/jeprof.1234.0.m0.heap``, where 1234 is the process id
+and 0 is a running counter indicating how many dumps have been performed on this process. Each dump is independent of the others
+and records the current stat of allocations since the profiling was activated. The dump file can be processed by jeprof
+to get text output or graphs. Details of how to use jeprof are in the man pages or `<https://manpages.debian.org/unstable/libjemalloc-dev/jeprof.1.en.html>`.
+
+You may want to send the dump message periodically to analyze how the |TS| memory allocation changes over time. This periodic dump can also be achieved by setting the
+``lg_prof_interval`` option in the run time configuration string.
+
+If the profiling is taking a significating amount of processing time and affecting |TS| performance, send the deactivate message to turn off profiling.::
+
+ traffic_ctl plugin msg memory_profile deactivate
+
+Send the stats message to cause detailed jemalloc stats to be printed in traffic.out. These stats represent actvitiy since the start of the |TS| process.::
Review comment:
Probably omit the period after the colon.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #7014: Add memory_profile plugin
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #7014:
URL: https://github.com/apache/trafficserver/pull/7014#discussion_r457506212
##########
File path: doc/admin-guide/plugins/memory_profile.en.rst
##########
@@ -0,0 +1,106 @@
+Memory_profile Plugin
+*********************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+This plugin listens for plugin msgs and invokes jemalloc control
+operations.
+
+Installation
+============
+
+Add the following line to :file:`plugin.config`::
+
+ memory_profile.so
+
+In addition, |TS| must be able to read jemalloc configuration
+information either through the ``JEMALLOC_CONF`` environment variable
+or via the string sym linked to ``/etc/malloc.conf``.
+
+For example, if the string below is in ``JEMALLOC_CONF`` or in the sym link string, it
+enables profiling and indicates that the memory dump prefix is ``/tmp/jeprof``.::
+
+ prof:true,prof_prefix:/tmp/jeprof
+
+Details on configuration jemalloc options at `<http://jemalloc.net/jemalloc.3.html>`.
+Changes to the configuration in ``JEMALLOC_CONF`` or ``/etc/malloc.conf`` require a process
+restart to pick up.
+
+Plugin Messages
+===============
+
+The plugin responds to the following mesages sent via traffic_ctl.
+
+Message Action
+========== ===================================================================================
+activate Start jemalloc profiling. Useful if prof_active:false was in the configure string.
+
+deactivate Stop jemalloc profiling.
+
+dump If profiling is enabled and active, it will generate a profile dump file.
+
+stats Print jemalloc statistics in traffic.out
+
+The command below sends the stats message to the plugin causing the current statistics to be written to traffic.out::
+
+ traffic_ctl plugin msg memory_profile stats
+
+Example Usage
+=============
+
+If your run time configuration string is::
+
+ prof:true,prof_prefix:/tmp/jeprof:prof_active:false
+
+|TS| has started without profile sampling started. Perhaps you didn't want to profile the start up phase of |TS|. To start
+you need to send the activate message to the plugin::
+
+ traffic_ctl plugin msg memory_profile activate
+
+If your run time configuration string does not indicate that the profiling is not started (e.g. the prof_active field is missing or set to true), you do not
+need to send the activate message.
+
+After waiting sometime for |TS| to gather some memory allocation data, you can send the dump message::
+
+ traffic_ctl plugin msg memory_profile dump
+
+This will cause a file containing information about the current state of the |TS| memory allocation to be dumped in a file prefixed
+by the value of prof_prefix. In this example, it would be something like ``/tmp/jeprof.1234.0.m0.heap``, where 1234 is the process id
+and 0 is a running counter indicating how many dumps have been performed on this process. Each dump is independent of the others
+and records the current stat of allocations since the profiling was activated. The dump file can be processed by jeprof
+to get text output or graphs. Details of how to use jeprof are in the man pages or `<https://manpages.debian.org/unstable/libjemalloc-dev/jeprof.1.en.html>`.
+
+You may want to send the dump message periodically to analyze how the |TS| memory allocation changes over time. This periodic dump can also be achieved by setting the
+``lg_prof_interval`` option in the run time configuration string.
+
+If the profiling is taking a significating amount of processing time and affecting |TS| performance, send the deactivate message to turn off profiling.::
+
+ traffic_ctl plugin msg memory_profile deactivate
+
+Send the stats message to cause detailed jemalloc stats to be printed in traffic.out. These stats represent actvitiy since the start of the |TS| process.::
Review comment:
We probably don't want the period before the colon. Also on line 92.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] randall commented on pull request #7014: Add memory_profile plugin
Posted by GitBox <gi...@apache.org>.
randall commented on pull request #7014:
URL: https://github.com/apache/trafficserver/pull/7014#issuecomment-661208473
[approve ci autest]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] zwoop commented on pull request #7014: Add memory_profile plugin
Posted by GitBox <gi...@apache.org>.
zwoop commented on pull request #7014:
URL: https://github.com/apache/trafficserver/pull/7014#issuecomment-668331317
Cherry-picked to v9.0.x branch.
Trying to pull in other PRs to 9.0.x breaks with this one, so wth lets just pull it in.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] shinrich commented on a change in pull request #7014: Add memory_profile plugin
Posted by GitBox <gi...@apache.org>.
shinrich commented on a change in pull request #7014:
URL: https://github.com/apache/trafficserver/pull/7014#discussion_r457426511
##########
File path: doc/admin-guide/plugins/memory_profile.en.rst
##########
@@ -0,0 +1,68 @@
+Memory_profile Plugin
+*********************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+This plugin listens for plugin msgs and invokes jemalloc control
+operations.
+
+Installation
+============
+
+Add the following line to :file:`plugin.config`::
+
+ memory_profile.so
+
+In addition, |TS| must be able to read jemalloc configuration
+information either through the ``JEMALLOC_CONF`` environment variable
+or via the string sym linked to ``/etc/malloc.conf``.
Review comment:
No, you will need to restart the process when changing that data.
##########
File path: doc/admin-guide/plugins/memory_profile.en.rst
##########
@@ -0,0 +1,68 @@
+Memory_profile Plugin
+*********************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+This plugin listens for plugin msgs and invokes jemalloc control
+operations.
+
+Installation
+============
+
+Add the following line to :file:`plugin.config`::
+
+ memory_profile.so
+
+In addition, |TS| must be able to read jemalloc configuration
+information either through the ``JEMALLOC_CONF`` environment variable
+or via the string sym linked to ``/etc/malloc.conf``.
+
+For example, if the string below is in ``JEMALLOC_CONF`` or in the sym link string, it
+enables profiling and indicates that the memory dump prefix is ``/tmp/jeprof``.::
+
+ prof:true,prof_prefix:/tmp/jeprof
+
+Details on configuration jemalloc options at `<http://jemalloc.net/jemalloc.3.html>`.
+
+Plugin Messages
+===============
+
+The plugin responds to the following mesages sent via traffic_ctl.
+
+Message Action
+========== ===================================================================================
+activate Start jemalloc profiling. Useful if prof_active:false was in the configure string.
+
+deactivate Stop jemalloc profiling.
+
+dump If profiling is enabled and active, it will generate a profile dump file.
+
+stats Print jemalloc statistics in traffic.out
+
+The command below sends the stats message to the plugin causing the current statistics to be written to traffic.out::
+
+ traffic_ctl plugin msg memory_profile stats
+
Review comment:
Good idea.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] shinrich merged pull request #7014: Add memory_profile plugin
Posted by GitBox <gi...@apache.org>.
shinrich merged pull request #7014:
URL: https://github.com/apache/trafficserver/pull/7014
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] shukitchan commented on a change in pull request #7014: Add memory_profile plugin
Posted by GitBox <gi...@apache.org>.
shukitchan commented on a change in pull request #7014:
URL: https://github.com/apache/trafficserver/pull/7014#discussion_r457428584
##########
File path: doc/admin-guide/plugins/memory_profile.en.rst
##########
@@ -0,0 +1,68 @@
+Memory_profile Plugin
+*********************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+This plugin listens for plugin msgs and invokes jemalloc control
+operations.
+
+Installation
+============
+
+Add the following line to :file:`plugin.config`::
+
+ memory_profile.so
+
+In addition, |TS| must be able to read jemalloc configuration
+information either through the ``JEMALLOC_CONF`` environment variable
+or via the string sym linked to ``/etc/malloc.conf``.
Review comment:
Let's mention that then .. that an ATS restart is needed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on a change in pull request #7014: Add memory_profile plugin
Posted by GitBox <gi...@apache.org>.
bneradt commented on a change in pull request #7014:
URL: https://github.com/apache/trafficserver/pull/7014#discussion_r456692386
##########
File path: doc/admin-guide/plugins/memory_profile.en.rst
##########
@@ -0,0 +1,68 @@
+Memory_profile Plugin
+*********************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+This plugin listens for plugin msgs and invokes jemalloc control
+operations.
+
+Installation
+============
+
+Add the following line to :file:`plugin.config`::
+
+ memory_profile.so
+
+In addition, |TS| must be able to read jemalloc configuration
+information either through the ``JEMALLOC_CONF`` environment variable
+or via the string sym linked to ``/etc/malloc.conf``.
+
+For example, if the string below is in ``JEMALLOC_CONF`` or in the sym link string, it
+enables profiling and indicates that the memory dump prefix is ``/tmp/jeprof``.::
+
+ prof:true,prof_prefix:/tmp/jeprof
+
+Details on configuration jemalloc options at `<http://jemalloc.net/jemalloc.3.html>`.
+
+Plugin Messages
+===============
+
+The plugin responds to the following mesages sent via traffic_ctl.
+
+Message Action
+========== ===================================================================================
+activate Start jemalloc profiling. Useful if prof_active:false was in the configure string.
+
+deactivate Stop jemalloc profiling.
+
+dump If profiling is enabled and active, it will generate a profile dump file.
+
+stats Print jemalloc statistics in traffic.out
+
+The command below sends the stats message to the plugin causing the current statistics to be written to traffic.out::
+
+ traffic_ctl plugin msg memory_profile stats
+
Review comment:
Looks great. I suggest expanding the examples section a bit to demonstrate how these commands are used. Maybe something like the following?
```
To capture stats for a session, first start profiling::
traffic_ctl plugin msg memory_profile activate
With profiling now enabled, perform the tasks for which you wish to collect jemalloc data. Once this is done, dump stats for that session via::
traffic_ctl plugin msg memory_profile dump
You can repeat this dump command to continue to get reports for this session. Once you are done with your session, stop profiling via::
traffic_ctl plugin msg memory_profile deactivate
The command below sends the stats message to the plugin causing the current jemalloc statistics to be written to traffic.out::
traffic_ctl plugin msg memory_profile stats
Note that unlike the previous set of commands, this latter command prints aggregate data which applies since process start and is not directly tied to an active or previous session.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org