You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by Dk Jack <dn...@gmail.com> on 2019/03/04 23:50:37 UTC
memory leak
Hi,
I am trying to chase a memory leak issue. I am using ATS version 6.2.3 and
I have a custom plugin written using the c++ API (atscppapi) running on the
same server. I enabled memory dump in my configuration and sent traffic
into the ATS with and without my plugin for an hour. Later, I extracted the
individual memory stats and plotted the stats for each object type against
time. The link shown below contains side-by-side plots for each ATS object
type without and with my plugin (the right side figures are with my plugin
enabled).
https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
Without my plugin, the memory consumption increases for various objects in
the beginning and then tapers off. However, with my plugin enabled, the
memory consumption is higher and more erratic. I am not holding on to the
request in my plugin. My plugin attaches TransactionPlugin to some
transactions (sending back 4xx response to client). For most others, it
lets it pass through. My plugin, doesn't allocate much memory. The memory
it allocates is sort of semi-static (manages a few maps that are updated
periodically). I don't anticipate my plugin's memory consumption to
increase with the transactions.
Since the presence of my plugin is the affecting the memory stats of
internal ATS objects, I am suspecting this could result of me not using the
API correctly. Would appreciate if anyone can venture a guess as to what
could be happening. Thank for help.
Dk.
Re: memory leak
Posted by Pushkar Pradhan <pp...@verizonmedia.com.INVALID>.
No I don't think so. It's a gcc feature so it should pull in what's
required.
On Mon, Mar 4, 2019 at 6:08 PM Dk Jack <dn...@gmail.com> wrote:
> btw, do I need to link it with any additional libraries to use lsan?
> Thanks.
>
>
> On Mon, Mar 4, 2019 at 6:05 PM Dk Jack <dn...@gmail.com> wrote:
>
> > Thanks Pushkar, I'll try that...
> >
> > On Mon, Mar 4, 2019 at 4:44 PM Pushkar Pradhan
> > <pp...@verizonmedia.com.invalid> wrote:
> >
> >> Perhaps compiling with LSAN will help: --enable-lsan enable
> >> stand-alone Leak Sanitizer
> >>
> >>
> >> On Mon, Mar 4, 2019 at 4:42 PM Bryan Call <bc...@apache.org> wrote:
> >>
> >> > Looking at the graphs it looks like your headers and http1client
> >> sessions
> >> > are leaking. Have you looked at this example to on how to send back
> >> error
> >> > messages using the C++ API:
> >> >
> >>
> https://github.com/apache/trafficserver/blob/master/example/cppapi/customresponse/CustomResponse.cc
> >> >
> >> > -Bryan
> >> >
> >> > > On Mar 4, 2019, at 3:50 PM, Dk Jack <dn...@gmail.com> wrote:
> >> > >
> >> > > Hi,
> >> > > I am trying to chase a memory leak issue. I am using ATS version
> 6.2.3
> >> > and
> >> > > I have a custom plugin written using the c++ API (atscppapi) running
> >> on
> >> > the
> >> > > same server. I enabled memory dump in my configuration and sent
> >> traffic
> >> > > into the ATS with and without my plugin for an hour. Later, I
> >> extracted
> >> > the
> >> > > individual memory stats and plotted the stats for each object type
> >> > against
> >> > > time. The link shown below contains side-by-side plots for each ATS
> >> > object
> >> > > type without and with my plugin (the right side figures are with my
> >> > plugin
> >> > > enabled).
> >> > >
> >> > >
> >> https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
> >> > >
> >> > > Without my plugin, the memory consumption increases for various
> >> objects
> >> > in
> >> > > the beginning and then tapers off. However, with my plugin enabled,
> >> the
> >> > > memory consumption is higher and more erratic. I am not holding on
> to
> >> the
> >> > > request in my plugin. My plugin attaches TransactionPlugin to some
> >> > > transactions (sending back 4xx response to client). For most others,
> >> it
> >> > > lets it pass through. My plugin, doesn't allocate much memory. The
> >> memory
> >> > > it allocates is sort of semi-static (manages a few maps that are
> >> updated
> >> > > periodically). I don't anticipate my plugin's memory consumption to
> >> > > increase with the transactions.
> >> > >
> >> > > Since the presence of my plugin is the affecting the memory stats of
> >> > > internal ATS objects, I am suspecting this could result of me not
> >> using
> >> > the
> >> > > API correctly. Would appreciate if anyone can venture a guess as to
> >> what
> >> > > could be happening. Thank for help.
> >> > >
> >> > > Dk.
> >> >
> >> >
> >>
> >
>
Re: memory leak
Posted by Dk Jack <dn...@gmail.com>.
btw, do I need to link it with any additional libraries to use lsan? Thanks.
On Mon, Mar 4, 2019 at 6:05 PM Dk Jack <dn...@gmail.com> wrote:
> Thanks Pushkar, I'll try that...
>
> On Mon, Mar 4, 2019 at 4:44 PM Pushkar Pradhan
> <pp...@verizonmedia.com.invalid> wrote:
>
>> Perhaps compiling with LSAN will help: --enable-lsan enable
>> stand-alone Leak Sanitizer
>>
>>
>> On Mon, Mar 4, 2019 at 4:42 PM Bryan Call <bc...@apache.org> wrote:
>>
>> > Looking at the graphs it looks like your headers and http1client
>> sessions
>> > are leaking. Have you looked at this example to on how to send back
>> error
>> > messages using the C++ API:
>> >
>> https://github.com/apache/trafficserver/blob/master/example/cppapi/customresponse/CustomResponse.cc
>> >
>> > -Bryan
>> >
>> > > On Mar 4, 2019, at 3:50 PM, Dk Jack <dn...@gmail.com> wrote:
>> > >
>> > > Hi,
>> > > I am trying to chase a memory leak issue. I am using ATS version 6.2.3
>> > and
>> > > I have a custom plugin written using the c++ API (atscppapi) running
>> on
>> > the
>> > > same server. I enabled memory dump in my configuration and sent
>> traffic
>> > > into the ATS with and without my plugin for an hour. Later, I
>> extracted
>> > the
>> > > individual memory stats and plotted the stats for each object type
>> > against
>> > > time. The link shown below contains side-by-side plots for each ATS
>> > object
>> > > type without and with my plugin (the right side figures are with my
>> > plugin
>> > > enabled).
>> > >
>> > >
>> https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
>> > >
>> > > Without my plugin, the memory consumption increases for various
>> objects
>> > in
>> > > the beginning and then tapers off. However, with my plugin enabled,
>> the
>> > > memory consumption is higher and more erratic. I am not holding on to
>> the
>> > > request in my plugin. My plugin attaches TransactionPlugin to some
>> > > transactions (sending back 4xx response to client). For most others,
>> it
>> > > lets it pass through. My plugin, doesn't allocate much memory. The
>> memory
>> > > it allocates is sort of semi-static (manages a few maps that are
>> updated
>> > > periodically). I don't anticipate my plugin's memory consumption to
>> > > increase with the transactions.
>> > >
>> > > Since the presence of my plugin is the affecting the memory stats of
>> > > internal ATS objects, I am suspecting this could result of me not
>> using
>> > the
>> > > API correctly. Would appreciate if anyone can venture a guess as to
>> what
>> > > could be happening. Thank for help.
>> > >
>> > > Dk.
>> >
>> >
>>
>
Re: memory leak
Posted by Dk Jack <dn...@gmail.com>.
Could someone let me know the atscppapi's AsyncHttpFetch and AsyncReceiver
classes can only used in the context of a Transaction or can they be used
like I was using independent of the Transaction. Most of the examples seem
to be using it in the context of a transaction. For my plugin, I'd like it
to be independent of the Transaction since I am using Async to update my
plugin maps. Thanks.
On Thu, Mar 7, 2019 at 6:39 PM Dk Jack <dn...@gmail.com> wrote:
> I narrowed down my problem to the async fetcher class which is derived
> from AsyncReceiver class.
> In my plugin, I maintain a few maps which get periodically updated. In my
> plugin, I create a thread
> to constantly (every 30s) poll an external server. When the thread wakes
> up, it uses the async fetcher
> class to retrieve info from an external server to update my maps. The
> following two lines seem
> to be causing the issue:
>
> AsyncHttpFetch* fetcher = new AsyncHttpFetch(mUrl,
> AsyncHttpFetch::STREAMING_DISABLED);
> Async::execute<AsyncHttpFetch>(this, fetcher, mMutex);
>
> If I comment them out, there's no leak. My async fetcher class follows a
> similar pattern as some
> of the atscppapi examples. To narrow down the issue, I've created a small
> plugin which models
> my plugin and also creates the issue I am seeing in my plugin. I've
> attached my test-plugin files.
> I would appreciate it if someone can take a look at my implementation and
> let me know I am
> using the API incorrectly.
>
> Thank you.
>
> On Tue, Mar 5, 2019 at 10:52 AM Dk Jack <dn...@gmail.com> wrote:
>
>> Thanks Leif,
>> I'd like to understand a bit more before introducing another variable.
>> The thing that puzzles me the most is why the system objects are behaving
>> erratically when I am not modifying them or holding on to them..
>>
>> On Mon, Mar 4, 2019 at 7:11 PM Leif Hedstrom <zw...@apache.org> wrote:
>>
>>> Before going down this rathole, I’d give v7.1.6 or v8.0.2 a spin.
>>> Remember, v6.x is an unsupported version, there is no more development nor
>>> any more releases.
>>>
>>> Cheers,
>>>
>>> — Leif
>>>
>>> > On Mar 4, 2019, at 20:00, Dk Jack <dn...@gmail.com> wrote:
>>> >
>>> > I am using atscppapi's for my plugin. I do something like this...
>>> >
>>> > Headers& headers = transaction.getClientRequest().getHeaders();
>>> >
>>> >
>>> >
>>> > On Mon, Mar 4, 2019 at 6:09 PM Pushkar Pradhan
>>> > <pp...@verizonmedia.com.invalid> wrote:
>>> >
>>> >> I am also trying to learn the TS APIs. I think you are supposed to
>>> call the
>>> >> TS...Release if you get a reference.
>>> >> E.g.
>>> >> if (TS_SUCCESS == TSHttpTxnClientReqGet(txnp, &buffer, &hdr_loc)) {
>>> >> TSDebug(PLUGIN_NAME, "Found client request");
>>> >> txn_info += R"(,"client-request":)" + collect_headers(buffer,
>>> >> hdr_loc, TSHttpTxnClientReqBodyBytesGet(txnp));
>>> >> TSHandleMLocRelease(buffer, TS_NULL_MLOC, hdr_loc);
>>> >> buffer = nullptr;
>>> >> }
>>> >>
>>> >>
>>> >>> On Mon, Mar 4, 2019 at 6:06 PM Dk Jack <dn...@gmail.com> wrote:
>>> >>>
>>> >>> Thanks Pushkar, I'll try that...
>>> >>>
>>> >>> On Mon, Mar 4, 2019 at 4:44 PM Pushkar Pradhan
>>> >>> <pp...@verizonmedia.com.invalid> wrote:
>>> >>>
>>> >>>> Perhaps compiling with LSAN will help: --enable-lsan
>>> enable
>>> >>>> stand-alone Leak Sanitizer
>>> >>>>
>>> >>>>
>>> >>>>> On Mon, Mar 4, 2019 at 4:42 PM Bryan Call <bc...@apache.org>
>>> wrote:
>>> >>>>>
>>> >>>>> Looking at the graphs it looks like your headers and http1client
>>> >>> sessions
>>> >>>>> are leaking. Have you looked at this example to on how to send
>>> back
>>> >>>> error
>>> >>>>> messages using the C++ API:
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>
>>> https://github.com/apache/trafficserver/blob/master/example/cppapi/customresponse/CustomResponse.cc
>>> >>>>>
>>> >>>>> -Bryan
>>> >>>>>
>>> >>>>>> On Mar 4, 2019, at 3:50 PM, Dk Jack <dn...@gmail.com> wrote:
>>> >>>>>>
>>> >>>>>> Hi,
>>> >>>>>> I am trying to chase a memory leak issue. I am using ATS version
>>> >>> 6.2.3
>>> >>>>> and
>>> >>>>>> I have a custom plugin written using the c++ API (atscppapi)
>>> >> running
>>> >>> on
>>> >>>>> the
>>> >>>>>> same server. I enabled memory dump in my configuration and sent
>>> >>> traffic
>>> >>>>>> into the ATS with and without my plugin for an hour. Later, I
>>> >>> extracted
>>> >>>>> the
>>> >>>>>> individual memory stats and plotted the stats for each object type
>>> >>>>> against
>>> >>>>>> time. The link shown below contains side-by-side plots for each
>>> ATS
>>> >>>>> object
>>> >>>>>> type without and with my plugin (the right side figures are with
>>> my
>>> >>>>> plugin
>>> >>>>>> enabled).
>>> >>>>>>
>>> >>>>>>
>>> >>>
>>> https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
>>> >>>>>>
>>> >>>>>> Without my plugin, the memory consumption increases for various
>>> >>> objects
>>> >>>>> in
>>> >>>>>> the beginning and then tapers off. However, with my plugin
>>> enabled,
>>> >>> the
>>> >>>>>> memory consumption is higher and more erratic. I am not holding on
>>> >> to
>>> >>>> the
>>> >>>>>> request in my plugin. My plugin attaches TransactionPlugin to some
>>> >>>>>> transactions (sending back 4xx response to client). For most
>>> >> others,
>>> >>> it
>>> >>>>>> lets it pass through. My plugin, doesn't allocate much memory. The
>>> >>>> memory
>>> >>>>>> it allocates is sort of semi-static (manages a few maps that are
>>> >>>> updated
>>> >>>>>> periodically). I don't anticipate my plugin's memory consumption
>>> to
>>> >>>>>> increase with the transactions.
>>> >>>>>>
>>> >>>>>> Since the presence of my plugin is the affecting the memory stats
>>> >> of
>>> >>>>>> internal ATS objects, I am suspecting this could result of me not
>>> >>> using
>>> >>>>> the
>>> >>>>>> API correctly. Would appreciate if anyone can venture a guess as
>>> to
>>> >>>> what
>>> >>>>>> could be happening. Thank for help.
>>> >>>>>>
>>> >>>>>> Dk.
>>> >>>>>
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>
>>>
>>>
Re: memory leak
Posted by Dk Jack <dn...@gmail.com>.
I narrowed down my problem to the async fetcher class which is derived from
AsyncReceiver class.
In my plugin, I maintain a few maps which get periodically updated. In my
plugin, I create a thread
to constantly (every 30s) poll an external server. When the thread wakes
up, it uses the async fetcher
class to retrieve info from an external server to update my maps. The
following two lines seem
to be causing the issue:
AsyncHttpFetch* fetcher = new AsyncHttpFetch(mUrl,
AsyncHttpFetch::STREAMING_DISABLED);
Async::execute<AsyncHttpFetch>(this, fetcher, mMutex);
If I comment them out, there's no leak. My async fetcher class follows a
similar pattern as some
of the atscppapi examples. To narrow down the issue, I've created a small
plugin which models
my plugin and also creates the issue I am seeing in my plugin. I've
attached my test-plugin files.
I would appreciate it if someone can take a look at my implementation and
let me know I am
using the API incorrectly.
Thank you.
On Tue, Mar 5, 2019 at 10:52 AM Dk Jack <dn...@gmail.com> wrote:
> Thanks Leif,
> I'd like to understand a bit more before introducing another variable. The
> thing that puzzles me the most is why the system objects are behaving
> erratically when I am not modifying them or holding on to them..
>
> On Mon, Mar 4, 2019 at 7:11 PM Leif Hedstrom <zw...@apache.org> wrote:
>
>> Before going down this rathole, I’d give v7.1.6 or v8.0.2 a spin.
>> Remember, v6.x is an unsupported version, there is no more development nor
>> any more releases.
>>
>> Cheers,
>>
>> — Leif
>>
>> > On Mar 4, 2019, at 20:00, Dk Jack <dn...@gmail.com> wrote:
>> >
>> > I am using atscppapi's for my plugin. I do something like this...
>> >
>> > Headers& headers = transaction.getClientRequest().getHeaders();
>> >
>> >
>> >
>> > On Mon, Mar 4, 2019 at 6:09 PM Pushkar Pradhan
>> > <pp...@verizonmedia.com.invalid> wrote:
>> >
>> >> I am also trying to learn the TS APIs. I think you are supposed to
>> call the
>> >> TS...Release if you get a reference.
>> >> E.g.
>> >> if (TS_SUCCESS == TSHttpTxnClientReqGet(txnp, &buffer, &hdr_loc)) {
>> >> TSDebug(PLUGIN_NAME, "Found client request");
>> >> txn_info += R"(,"client-request":)" + collect_headers(buffer,
>> >> hdr_loc, TSHttpTxnClientReqBodyBytesGet(txnp));
>> >> TSHandleMLocRelease(buffer, TS_NULL_MLOC, hdr_loc);
>> >> buffer = nullptr;
>> >> }
>> >>
>> >>
>> >>> On Mon, Mar 4, 2019 at 6:06 PM Dk Jack <dn...@gmail.com> wrote:
>> >>>
>> >>> Thanks Pushkar, I'll try that...
>> >>>
>> >>> On Mon, Mar 4, 2019 at 4:44 PM Pushkar Pradhan
>> >>> <pp...@verizonmedia.com.invalid> wrote:
>> >>>
>> >>>> Perhaps compiling with LSAN will help: --enable-lsan
>> enable
>> >>>> stand-alone Leak Sanitizer
>> >>>>
>> >>>>
>> >>>>> On Mon, Mar 4, 2019 at 4:42 PM Bryan Call <bc...@apache.org> wrote:
>> >>>>>
>> >>>>> Looking at the graphs it looks like your headers and http1client
>> >>> sessions
>> >>>>> are leaking. Have you looked at this example to on how to send back
>> >>>> error
>> >>>>> messages using the C++ API:
>> >>>>>
>> >>>>
>> >>>
>> >>
>> https://github.com/apache/trafficserver/blob/master/example/cppapi/customresponse/CustomResponse.cc
>> >>>>>
>> >>>>> -Bryan
>> >>>>>
>> >>>>>> On Mar 4, 2019, at 3:50 PM, Dk Jack <dn...@gmail.com> wrote:
>> >>>>>>
>> >>>>>> Hi,
>> >>>>>> I am trying to chase a memory leak issue. I am using ATS version
>> >>> 6.2.3
>> >>>>> and
>> >>>>>> I have a custom plugin written using the c++ API (atscppapi)
>> >> running
>> >>> on
>> >>>>> the
>> >>>>>> same server. I enabled memory dump in my configuration and sent
>> >>> traffic
>> >>>>>> into the ATS with and without my plugin for an hour. Later, I
>> >>> extracted
>> >>>>> the
>> >>>>>> individual memory stats and plotted the stats for each object type
>> >>>>> against
>> >>>>>> time. The link shown below contains side-by-side plots for each ATS
>> >>>>> object
>> >>>>>> type without and with my plugin (the right side figures are with my
>> >>>>> plugin
>> >>>>>> enabled).
>> >>>>>>
>> >>>>>>
>> >>>
>> https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
>> >>>>>>
>> >>>>>> Without my plugin, the memory consumption increases for various
>> >>> objects
>> >>>>> in
>> >>>>>> the beginning and then tapers off. However, with my plugin enabled,
>> >>> the
>> >>>>>> memory consumption is higher and more erratic. I am not holding on
>> >> to
>> >>>> the
>> >>>>>> request in my plugin. My plugin attaches TransactionPlugin to some
>> >>>>>> transactions (sending back 4xx response to client). For most
>> >> others,
>> >>> it
>> >>>>>> lets it pass through. My plugin, doesn't allocate much memory. The
>> >>>> memory
>> >>>>>> it allocates is sort of semi-static (manages a few maps that are
>> >>>> updated
>> >>>>>> periodically). I don't anticipate my plugin's memory consumption to
>> >>>>>> increase with the transactions.
>> >>>>>>
>> >>>>>> Since the presence of my plugin is the affecting the memory stats
>> >> of
>> >>>>>> internal ATS objects, I am suspecting this could result of me not
>> >>> using
>> >>>>> the
>> >>>>>> API correctly. Would appreciate if anyone can venture a guess as to
>> >>>> what
>> >>>>>> could be happening. Thank for help.
>> >>>>>>
>> >>>>>> Dk.
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>>
>>
Re: memory leak
Posted by Dk Jack <dn...@gmail.com>.
Thanks Leif,
I'd like to understand a bit more before introducing another variable. The
thing that puzzles me the most is why the system objects are behaving
erratically when I am not modifying them or holding on to them..
On Mon, Mar 4, 2019 at 7:11 PM Leif Hedstrom <zw...@apache.org> wrote:
> Before going down this rathole, I’d give v7.1.6 or v8.0.2 a spin.
> Remember, v6.x is an unsupported version, there is no more development nor
> any more releases.
>
> Cheers,
>
> — Leif
>
> > On Mar 4, 2019, at 20:00, Dk Jack <dn...@gmail.com> wrote:
> >
> > I am using atscppapi's for my plugin. I do something like this...
> >
> > Headers& headers = transaction.getClientRequest().getHeaders();
> >
> >
> >
> > On Mon, Mar 4, 2019 at 6:09 PM Pushkar Pradhan
> > <pp...@verizonmedia.com.invalid> wrote:
> >
> >> I am also trying to learn the TS APIs. I think you are supposed to call
> the
> >> TS...Release if you get a reference.
> >> E.g.
> >> if (TS_SUCCESS == TSHttpTxnClientReqGet(txnp, &buffer, &hdr_loc)) {
> >> TSDebug(PLUGIN_NAME, "Found client request");
> >> txn_info += R"(,"client-request":)" + collect_headers(buffer,
> >> hdr_loc, TSHttpTxnClientReqBodyBytesGet(txnp));
> >> TSHandleMLocRelease(buffer, TS_NULL_MLOC, hdr_loc);
> >> buffer = nullptr;
> >> }
> >>
> >>
> >>> On Mon, Mar 4, 2019 at 6:06 PM Dk Jack <dn...@gmail.com> wrote:
> >>>
> >>> Thanks Pushkar, I'll try that...
> >>>
> >>> On Mon, Mar 4, 2019 at 4:44 PM Pushkar Pradhan
> >>> <pp...@verizonmedia.com.invalid> wrote:
> >>>
> >>>> Perhaps compiling with LSAN will help: --enable-lsan enable
> >>>> stand-alone Leak Sanitizer
> >>>>
> >>>>
> >>>>> On Mon, Mar 4, 2019 at 4:42 PM Bryan Call <bc...@apache.org> wrote:
> >>>>>
> >>>>> Looking at the graphs it looks like your headers and http1client
> >>> sessions
> >>>>> are leaking. Have you looked at this example to on how to send back
> >>>> error
> >>>>> messages using the C++ API:
> >>>>>
> >>>>
> >>>
> >>
> https://github.com/apache/trafficserver/blob/master/example/cppapi/customresponse/CustomResponse.cc
> >>>>>
> >>>>> -Bryan
> >>>>>
> >>>>>> On Mar 4, 2019, at 3:50 PM, Dk Jack <dn...@gmail.com> wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>> I am trying to chase a memory leak issue. I am using ATS version
> >>> 6.2.3
> >>>>> and
> >>>>>> I have a custom plugin written using the c++ API (atscppapi)
> >> running
> >>> on
> >>>>> the
> >>>>>> same server. I enabled memory dump in my configuration and sent
> >>> traffic
> >>>>>> into the ATS with and without my plugin for an hour. Later, I
> >>> extracted
> >>>>> the
> >>>>>> individual memory stats and plotted the stats for each object type
> >>>>> against
> >>>>>> time. The link shown below contains side-by-side plots for each ATS
> >>>>> object
> >>>>>> type without and with my plugin (the right side figures are with my
> >>>>> plugin
> >>>>>> enabled).
> >>>>>>
> >>>>>>
> >>> https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
> >>>>>>
> >>>>>> Without my plugin, the memory consumption increases for various
> >>> objects
> >>>>> in
> >>>>>> the beginning and then tapers off. However, with my plugin enabled,
> >>> the
> >>>>>> memory consumption is higher and more erratic. I am not holding on
> >> to
> >>>> the
> >>>>>> request in my plugin. My plugin attaches TransactionPlugin to some
> >>>>>> transactions (sending back 4xx response to client). For most
> >> others,
> >>> it
> >>>>>> lets it pass through. My plugin, doesn't allocate much memory. The
> >>>> memory
> >>>>>> it allocates is sort of semi-static (manages a few maps that are
> >>>> updated
> >>>>>> periodically). I don't anticipate my plugin's memory consumption to
> >>>>>> increase with the transactions.
> >>>>>>
> >>>>>> Since the presence of my plugin is the affecting the memory stats
> >> of
> >>>>>> internal ATS objects, I am suspecting this could result of me not
> >>> using
> >>>>> the
> >>>>>> API correctly. Would appreciate if anyone can venture a guess as to
> >>>> what
> >>>>>> could be happening. Thank for help.
> >>>>>>
> >>>>>> Dk.
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>
Re: memory leak
Posted by Leif Hedstrom <zw...@apache.org>.
Before going down this rathole, I’d give v7.1.6 or v8.0.2 a spin. Remember, v6.x is an unsupported version, there is no more development nor any more releases.
Cheers,
— Leif
> On Mar 4, 2019, at 20:00, Dk Jack <dn...@gmail.com> wrote:
>
> I am using atscppapi's for my plugin. I do something like this...
>
> Headers& headers = transaction.getClientRequest().getHeaders();
>
>
>
> On Mon, Mar 4, 2019 at 6:09 PM Pushkar Pradhan
> <pp...@verizonmedia.com.invalid> wrote:
>
>> I am also trying to learn the TS APIs. I think you are supposed to call the
>> TS...Release if you get a reference.
>> E.g.
>> if (TS_SUCCESS == TSHttpTxnClientReqGet(txnp, &buffer, &hdr_loc)) {
>> TSDebug(PLUGIN_NAME, "Found client request");
>> txn_info += R"(,"client-request":)" + collect_headers(buffer,
>> hdr_loc, TSHttpTxnClientReqBodyBytesGet(txnp));
>> TSHandleMLocRelease(buffer, TS_NULL_MLOC, hdr_loc);
>> buffer = nullptr;
>> }
>>
>>
>>> On Mon, Mar 4, 2019 at 6:06 PM Dk Jack <dn...@gmail.com> wrote:
>>>
>>> Thanks Pushkar, I'll try that...
>>>
>>> On Mon, Mar 4, 2019 at 4:44 PM Pushkar Pradhan
>>> <pp...@verizonmedia.com.invalid> wrote:
>>>
>>>> Perhaps compiling with LSAN will help: --enable-lsan enable
>>>> stand-alone Leak Sanitizer
>>>>
>>>>
>>>>> On Mon, Mar 4, 2019 at 4:42 PM Bryan Call <bc...@apache.org> wrote:
>>>>>
>>>>> Looking at the graphs it looks like your headers and http1client
>>> sessions
>>>>> are leaking. Have you looked at this example to on how to send back
>>>> error
>>>>> messages using the C++ API:
>>>>>
>>>>
>>>
>> https://github.com/apache/trafficserver/blob/master/example/cppapi/customresponse/CustomResponse.cc
>>>>>
>>>>> -Bryan
>>>>>
>>>>>> On Mar 4, 2019, at 3:50 PM, Dk Jack <dn...@gmail.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>> I am trying to chase a memory leak issue. I am using ATS version
>>> 6.2.3
>>>>> and
>>>>>> I have a custom plugin written using the c++ API (atscppapi)
>> running
>>> on
>>>>> the
>>>>>> same server. I enabled memory dump in my configuration and sent
>>> traffic
>>>>>> into the ATS with and without my plugin for an hour. Later, I
>>> extracted
>>>>> the
>>>>>> individual memory stats and plotted the stats for each object type
>>>>> against
>>>>>> time. The link shown below contains side-by-side plots for each ATS
>>>>> object
>>>>>> type without and with my plugin (the right side figures are with my
>>>>> plugin
>>>>>> enabled).
>>>>>>
>>>>>>
>>> https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
>>>>>>
>>>>>> Without my plugin, the memory consumption increases for various
>>> objects
>>>>> in
>>>>>> the beginning and then tapers off. However, with my plugin enabled,
>>> the
>>>>>> memory consumption is higher and more erratic. I am not holding on
>> to
>>>> the
>>>>>> request in my plugin. My plugin attaches TransactionPlugin to some
>>>>>> transactions (sending back 4xx response to client). For most
>> others,
>>> it
>>>>>> lets it pass through. My plugin, doesn't allocate much memory. The
>>>> memory
>>>>>> it allocates is sort of semi-static (manages a few maps that are
>>>> updated
>>>>>> periodically). I don't anticipate my plugin's memory consumption to
>>>>>> increase with the transactions.
>>>>>>
>>>>>> Since the presence of my plugin is the affecting the memory stats
>> of
>>>>>> internal ATS objects, I am suspecting this could result of me not
>>> using
>>>>> the
>>>>>> API correctly. Would appreciate if anyone can venture a guess as to
>>>> what
>>>>>> could be happening. Thank for help.
>>>>>>
>>>>>> Dk.
>>>>>
>>>>>
>>>>
>>>
>>
Re: memory leak
Posted by Dk Jack <dn...@gmail.com>.
I am using atscppapi's for my plugin. I do something like this...
Headers& headers = transaction.getClientRequest().getHeaders();
On Mon, Mar 4, 2019 at 6:09 PM Pushkar Pradhan
<pp...@verizonmedia.com.invalid> wrote:
> I am also trying to learn the TS APIs. I think you are supposed to call the
> TS...Release if you get a reference.
> E.g.
> if (TS_SUCCESS == TSHttpTxnClientReqGet(txnp, &buffer, &hdr_loc)) {
> TSDebug(PLUGIN_NAME, "Found client request");
> txn_info += R"(,"client-request":)" + collect_headers(buffer,
> hdr_loc, TSHttpTxnClientReqBodyBytesGet(txnp));
> TSHandleMLocRelease(buffer, TS_NULL_MLOC, hdr_loc);
> buffer = nullptr;
> }
>
>
> On Mon, Mar 4, 2019 at 6:06 PM Dk Jack <dn...@gmail.com> wrote:
>
> > Thanks Pushkar, I'll try that...
> >
> > On Mon, Mar 4, 2019 at 4:44 PM Pushkar Pradhan
> > <pp...@verizonmedia.com.invalid> wrote:
> >
> > > Perhaps compiling with LSAN will help: --enable-lsan enable
> > > stand-alone Leak Sanitizer
> > >
> > >
> > > On Mon, Mar 4, 2019 at 4:42 PM Bryan Call <bc...@apache.org> wrote:
> > >
> > > > Looking at the graphs it looks like your headers and http1client
> > sessions
> > > > are leaking. Have you looked at this example to on how to send back
> > > error
> > > > messages using the C++ API:
> > > >
> > >
> >
> https://github.com/apache/trafficserver/blob/master/example/cppapi/customresponse/CustomResponse.cc
> > > >
> > > > -Bryan
> > > >
> > > > > On Mar 4, 2019, at 3:50 PM, Dk Jack <dn...@gmail.com> wrote:
> > > > >
> > > > > Hi,
> > > > > I am trying to chase a memory leak issue. I am using ATS version
> > 6.2.3
> > > > and
> > > > > I have a custom plugin written using the c++ API (atscppapi)
> running
> > on
> > > > the
> > > > > same server. I enabled memory dump in my configuration and sent
> > traffic
> > > > > into the ATS with and without my plugin for an hour. Later, I
> > extracted
> > > > the
> > > > > individual memory stats and plotted the stats for each object type
> > > > against
> > > > > time. The link shown below contains side-by-side plots for each ATS
> > > > object
> > > > > type without and with my plugin (the right side figures are with my
> > > > plugin
> > > > > enabled).
> > > > >
> > > > >
> > https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
> > > > >
> > > > > Without my plugin, the memory consumption increases for various
> > objects
> > > > in
> > > > > the beginning and then tapers off. However, with my plugin enabled,
> > the
> > > > > memory consumption is higher and more erratic. I am not holding on
> to
> > > the
> > > > > request in my plugin. My plugin attaches TransactionPlugin to some
> > > > > transactions (sending back 4xx response to client). For most
> others,
> > it
> > > > > lets it pass through. My plugin, doesn't allocate much memory. The
> > > memory
> > > > > it allocates is sort of semi-static (manages a few maps that are
> > > updated
> > > > > periodically). I don't anticipate my plugin's memory consumption to
> > > > > increase with the transactions.
> > > > >
> > > > > Since the presence of my plugin is the affecting the memory stats
> of
> > > > > internal ATS objects, I am suspecting this could result of me not
> > using
> > > > the
> > > > > API correctly. Would appreciate if anyone can venture a guess as to
> > > what
> > > > > could be happening. Thank for help.
> > > > >
> > > > > Dk.
> > > >
> > > >
> > >
> >
>
Re: memory leak
Posted by Pushkar Pradhan <pp...@verizonmedia.com.INVALID>.
I am also trying to learn the TS APIs. I think you are supposed to call the
TS...Release if you get a reference.
E.g.
if (TS_SUCCESS == TSHttpTxnClientReqGet(txnp, &buffer, &hdr_loc)) {
TSDebug(PLUGIN_NAME, "Found client request");
txn_info += R"(,"client-request":)" + collect_headers(buffer,
hdr_loc, TSHttpTxnClientReqBodyBytesGet(txnp));
TSHandleMLocRelease(buffer, TS_NULL_MLOC, hdr_loc);
buffer = nullptr;
}
On Mon, Mar 4, 2019 at 6:06 PM Dk Jack <dn...@gmail.com> wrote:
> Thanks Pushkar, I'll try that...
>
> On Mon, Mar 4, 2019 at 4:44 PM Pushkar Pradhan
> <pp...@verizonmedia.com.invalid> wrote:
>
> > Perhaps compiling with LSAN will help: --enable-lsan enable
> > stand-alone Leak Sanitizer
> >
> >
> > On Mon, Mar 4, 2019 at 4:42 PM Bryan Call <bc...@apache.org> wrote:
> >
> > > Looking at the graphs it looks like your headers and http1client
> sessions
> > > are leaking. Have you looked at this example to on how to send back
> > error
> > > messages using the C++ API:
> > >
> >
> https://github.com/apache/trafficserver/blob/master/example/cppapi/customresponse/CustomResponse.cc
> > >
> > > -Bryan
> > >
> > > > On Mar 4, 2019, at 3:50 PM, Dk Jack <dn...@gmail.com> wrote:
> > > >
> > > > Hi,
> > > > I am trying to chase a memory leak issue. I am using ATS version
> 6.2.3
> > > and
> > > > I have a custom plugin written using the c++ API (atscppapi) running
> on
> > > the
> > > > same server. I enabled memory dump in my configuration and sent
> traffic
> > > > into the ATS with and without my plugin for an hour. Later, I
> extracted
> > > the
> > > > individual memory stats and plotted the stats for each object type
> > > against
> > > > time. The link shown below contains side-by-side plots for each ATS
> > > object
> > > > type without and with my plugin (the right side figures are with my
> > > plugin
> > > > enabled).
> > > >
> > > >
> https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
> > > >
> > > > Without my plugin, the memory consumption increases for various
> objects
> > > in
> > > > the beginning and then tapers off. However, with my plugin enabled,
> the
> > > > memory consumption is higher and more erratic. I am not holding on to
> > the
> > > > request in my plugin. My plugin attaches TransactionPlugin to some
> > > > transactions (sending back 4xx response to client). For most others,
> it
> > > > lets it pass through. My plugin, doesn't allocate much memory. The
> > memory
> > > > it allocates is sort of semi-static (manages a few maps that are
> > updated
> > > > periodically). I don't anticipate my plugin's memory consumption to
> > > > increase with the transactions.
> > > >
> > > > Since the presence of my plugin is the affecting the memory stats of
> > > > internal ATS objects, I am suspecting this could result of me not
> using
> > > the
> > > > API correctly. Would appreciate if anyone can venture a guess as to
> > what
> > > > could be happening. Thank for help.
> > > >
> > > > Dk.
> > >
> > >
> >
>
Re: memory leak
Posted by Dk Jack <dn...@gmail.com>.
Thanks Pushkar, I'll try that...
On Mon, Mar 4, 2019 at 4:44 PM Pushkar Pradhan
<pp...@verizonmedia.com.invalid> wrote:
> Perhaps compiling with LSAN will help: --enable-lsan enable
> stand-alone Leak Sanitizer
>
>
> On Mon, Mar 4, 2019 at 4:42 PM Bryan Call <bc...@apache.org> wrote:
>
> > Looking at the graphs it looks like your headers and http1client sessions
> > are leaking. Have you looked at this example to on how to send back
> error
> > messages using the C++ API:
> >
> https://github.com/apache/trafficserver/blob/master/example/cppapi/customresponse/CustomResponse.cc
> >
> > -Bryan
> >
> > > On Mar 4, 2019, at 3:50 PM, Dk Jack <dn...@gmail.com> wrote:
> > >
> > > Hi,
> > > I am trying to chase a memory leak issue. I am using ATS version 6.2.3
> > and
> > > I have a custom plugin written using the c++ API (atscppapi) running on
> > the
> > > same server. I enabled memory dump in my configuration and sent traffic
> > > into the ATS with and without my plugin for an hour. Later, I extracted
> > the
> > > individual memory stats and plotted the stats for each object type
> > against
> > > time. The link shown below contains side-by-side plots for each ATS
> > object
> > > type without and with my plugin (the right side figures are with my
> > plugin
> > > enabled).
> > >
> > > https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
> > >
> > > Without my plugin, the memory consumption increases for various objects
> > in
> > > the beginning and then tapers off. However, with my plugin enabled, the
> > > memory consumption is higher and more erratic. I am not holding on to
> the
> > > request in my plugin. My plugin attaches TransactionPlugin to some
> > > transactions (sending back 4xx response to client). For most others, it
> > > lets it pass through. My plugin, doesn't allocate much memory. The
> memory
> > > it allocates is sort of semi-static (manages a few maps that are
> updated
> > > periodically). I don't anticipate my plugin's memory consumption to
> > > increase with the transactions.
> > >
> > > Since the presence of my plugin is the affecting the memory stats of
> > > internal ATS objects, I am suspecting this could result of me not using
> > the
> > > API correctly. Would appreciate if anyone can venture a guess as to
> what
> > > could be happening. Thank for help.
> > >
> > > Dk.
> >
> >
>
Re: memory leak
Posted by Pushkar Pradhan <pp...@verizonmedia.com.INVALID>.
Perhaps compiling with LSAN will help: --enable-lsan enable
stand-alone Leak Sanitizer
On Mon, Mar 4, 2019 at 4:42 PM Bryan Call <bc...@apache.org> wrote:
> Looking at the graphs it looks like your headers and http1client sessions
> are leaking. Have you looked at this example to on how to send back error
> messages using the C++ API:
> https://github.com/apache/trafficserver/blob/master/example/cppapi/customresponse/CustomResponse.cc
>
> -Bryan
>
> > On Mar 4, 2019, at 3:50 PM, Dk Jack <dn...@gmail.com> wrote:
> >
> > Hi,
> > I am trying to chase a memory leak issue. I am using ATS version 6.2.3
> and
> > I have a custom plugin written using the c++ API (atscppapi) running on
> the
> > same server. I enabled memory dump in my configuration and sent traffic
> > into the ATS with and without my plugin for an hour. Later, I extracted
> the
> > individual memory stats and plotted the stats for each object type
> against
> > time. The link shown below contains side-by-side plots for each ATS
> object
> > type without and with my plugin (the right side figures are with my
> plugin
> > enabled).
> >
> > https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
> >
> > Without my plugin, the memory consumption increases for various objects
> in
> > the beginning and then tapers off. However, with my plugin enabled, the
> > memory consumption is higher and more erratic. I am not holding on to the
> > request in my plugin. My plugin attaches TransactionPlugin to some
> > transactions (sending back 4xx response to client). For most others, it
> > lets it pass through. My plugin, doesn't allocate much memory. The memory
> > it allocates is sort of semi-static (manages a few maps that are updated
> > periodically). I don't anticipate my plugin's memory consumption to
> > increase with the transactions.
> >
> > Since the presence of my plugin is the affecting the memory stats of
> > internal ATS objects, I am suspecting this could result of me not using
> the
> > API correctly. Would appreciate if anyone can venture a guess as to what
> > could be happening. Thank for help.
> >
> > Dk.
>
>
Re: memory leak
Posted by Dk Jack <dn...@gmail.com>.
I saw that too. However, I don't do anything besides getting a reference to
headers and iterating over them.
http1client, I don't even know how it's involved since I am mostly dealing
with XXXPlugin apis in c++ library.
On Mon, Mar 4, 2019 at 4:42 PM Bryan Call <bc...@apache.org> wrote:
> Looking at the graphs it looks like your headers and http1client sessions
> are leaking. Have you looked at this example to on how to send back error
> messages using the C++ API:
> https://github.com/apache/trafficserver/blob/master/example/cppapi/customresponse/CustomResponse.cc
>
> -Bryan
>
> > On Mar 4, 2019, at 3:50 PM, Dk Jack <dn...@gmail.com> wrote:
> >
> > Hi,
> > I am trying to chase a memory leak issue. I am using ATS version 6.2.3
> and
> > I have a custom plugin written using the c++ API (atscppapi) running on
> the
> > same server. I enabled memory dump in my configuration and sent traffic
> > into the ATS with and without my plugin for an hour. Later, I extracted
> the
> > individual memory stats and plotted the stats for each object type
> against
> > time. The link shown below contains side-by-side plots for each ATS
> object
> > type without and with my plugin (the right side figures are with my
> plugin
> > enabled).
> >
> > https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
> >
> > Without my plugin, the memory consumption increases for various objects
> in
> > the beginning and then tapers off. However, with my plugin enabled, the
> > memory consumption is higher and more erratic. I am not holding on to the
> > request in my plugin. My plugin attaches TransactionPlugin to some
> > transactions (sending back 4xx response to client). For most others, it
> > lets it pass through. My plugin, doesn't allocate much memory. The memory
> > it allocates is sort of semi-static (manages a few maps that are updated
> > periodically). I don't anticipate my plugin's memory consumption to
> > increase with the transactions.
> >
> > Since the presence of my plugin is the affecting the memory stats of
> > internal ATS objects, I am suspecting this could result of me not using
> the
> > API correctly. Would appreciate if anyone can venture a guess as to what
> > could be happening. Thank for help.
> >
> > Dk.
>
>
Re: memory leak
Posted by Bryan Call <bc...@apache.org>.
Looking at the graphs it looks like your headers and http1client sessions are leaking. Have you looked at this example to on how to send back error messages using the C++ API: https://github.com/apache/trafficserver/blob/master/example/cppapi/customresponse/CustomResponse.cc
-Bryan
> On Mar 4, 2019, at 3:50 PM, Dk Jack <dn...@gmail.com> wrote:
>
> Hi,
> I am trying to chase a memory leak issue. I am using ATS version 6.2.3 and
> I have a custom plugin written using the c++ API (atscppapi) running on the
> same server. I enabled memory dump in my configuration and sent traffic
> into the ATS with and without my plugin for an hour. Later, I extracted the
> individual memory stats and plotted the stats for each object type against
> time. The link shown below contains side-by-side plots for each ATS object
> type without and with my plugin (the right side figures are with my plugin
> enabled).
>
> https://www.dropbox.com/s/rfszd6fnjydm8s7/ats_mem_stats_comparison.pdf
>
> Without my plugin, the memory consumption increases for various objects in
> the beginning and then tapers off. However, with my plugin enabled, the
> memory consumption is higher and more erratic. I am not holding on to the
> request in my plugin. My plugin attaches TransactionPlugin to some
> transactions (sending back 4xx response to client). For most others, it
> lets it pass through. My plugin, doesn't allocate much memory. The memory
> it allocates is sort of semi-static (manages a few maps that are updated
> periodically). I don't anticipate my plugin's memory consumption to
> increase with the transactions.
>
> Since the presence of my plugin is the affecting the memory stats of
> internal ATS objects, I am suspecting this could result of me not using the
> API correctly. Would appreciate if anyone can venture a guess as to what
> could be happening. Thank for help.
>
> Dk.