You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by Acácio Centeno <ac...@azion.com> on 2014/08/18 15:24:02 UTC
How to debug a TSHttpTxn handler that is becoming invalid
I've written a remap plugin but from time to time it's segfaulting. Looking
at the generated Stack Trace, it's dying on the following function:
int
TSHttpTxnDebugGet(TSHttpTxn txnp)
{
* sdk_assert(sdk_sanity_check_*
*txn(txnp) == TS_SUCCESS);* return ((HttpSM *)txnp)->debug_on;
}
And it's dying on the assert, that is, the transaction handler is no longer
valid. This seems strange given that this handler had just been passed to
the plugin when TsRemapDoRemap was called:
/usr/bin/traffic_server - STACK TRACE:
/usr/lib64/trafficserver/libtsutil.so.5(ink_fatal_die+0x0)[0x2b0fd7c999b2]
/usr/lib64/trafficserver/libtsutil.so.5(_Z12ink_get_randv+0x0)[0x2b0fd7c9866c]
/usr/bin/traffic_server(_TSAssert+0x0)[0x4ff548]
*/usr/bin/traffic_server(TSHttpTxnDebugGet+0x30)[0x50c254]*
/usr/lib64/trafficserver/plugins/request_filter.so(_ZN20RequestFilterContext9matchListERSt6vectorIP20RequestFilterElementSaIS2_EE+0x59c)[0x2b0fe5f97b42]
/usr/lib64/trafficserver/plugins/request_filter.so(_ZN20RequestFilterContext14matchAllowListEv+0x23)[0x2b0fe5f97c7d]
*/usr/lib64/trafficserver/plugins/request_filter.so(TSRemapDoRemap+0x1ec)[0x2b0fe5fa0ad7]*
/usr/bin/traffic_server(_ZN12RemapPlugins10run_pluginEP17remap_plugin_info+0x13d)[0x661157]
... <further stack elements removed for readability> ...
Attaching the process to GDB, I've seen that:
(gdb) f 7
#7 0x000000000050c254 in TSHttpTxnDebugGet (txnp=0x2b5797e0dcb0) at
InkAPI.cc:5736
5736 sdk_assert(sdk_sanity_check_txn(txnp) == TS_SUCCESS);
(gdb) p txnp
$2 = (TSHttpTxn) 0x2b5797e0dcb0
(gdb) p /x ((HttpSM *) txnp)->magic
$3 = 0xdeadfeed
That is, indeed this handler is no longer valid, as the transaction is dead.
Would there be some condition under which the transaction handler was
supposed to get invalid while a remap plugin is active? Say, if the client
happens to disconnect exactly after the plugin was activated?
If it may happen, would there be a way for me to assert the handler is
valid prior to calling a Core function that would otherwise raise a
segfault? The reason I would like to do this check is that I could make
only this request fail, instead of making all other possibly thousands of
requests that are being processed along with this one fail, which is what
is going to happen should the system abort.
If that was not supposed to happen, any ideas on how I could investigate
further the reason why the transaction handler became invalid? Obviously
the plugin is not mangling with the object, as it's an opaque pointer on
the plugin's perspective, so it must have been something that happened on
the Core side, I just cant figure out how to pin point where the handler
became invalid, as this point where it's dying seems to be the first call
to the Core after the plugin was activated.
We're using version 5.0.1.
Best Regards,
Acácio Centeno
Software Engineering
Azion Technologies
Porto Alegre, Brasil +55 51 3012 3005 | +55 51 8118 9947
Miami, USA +1 305 704 8816
Quaisquer informações contidas neste e-mail e anexos podem ser
confidenciais e privilegiadas, protegidas por sigilo legal. Qualquer forma
de utilização deste documento depende de autorização do emissor, sujeito as
penalidades cabíveis.
Any information in this e-mail and attachments may be confidential and
privileged, protected by legal confidentiality. The use of this document
require authorization by the issuer, subject to penalties.
Re: How to debug a TSHttpTxn handler that is becoming invalid
Posted by Sudheer Vinukonda <su...@yahoo-inc.com.INVALID>.
You may want to add handling for TS_HTTP_TXN_CLOSE_HOOK.
For example:
https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;a=blob;f=plugin
s/experimental/stale_while_revalidate/swr.cpp;hb=69bd5473
Thanks,
Sudheer
On 8/18/14, 6:24 AM, "Acácio Centeno" <ac...@azion.com> wrote:
>I've written a remap plugin but from time to time it's segfaulting.
>Looking
>at the generated Stack Trace, it's dying on the following function:
>
>int
>TSHttpTxnDebugGet(TSHttpTxn txnp)
>{
>* sdk_assert(sdk_sanity_check_*
>*txn(txnp) == TS_SUCCESS);* return ((HttpSM *)txnp)->debug_on;
>}
>
>And it's dying on the assert, that is, the transaction handler is no
>longer
>valid. This seems strange given that this handler had just been passed to
>the plugin when TsRemapDoRemap was called:
>
>/usr/bin/traffic_server - STACK TRACE:
>/usr/lib64/trafficserver/libtsutil.so.5(ink_fatal_die+0x0)[0x2b0fd7c999b2]
>/usr/lib64/trafficserver/libtsutil.so.5(_Z12ink_get_randv+0x0)[0x2b0fd7c98
>66c]
>/usr/bin/traffic_server(_TSAssert+0x0)[0x4ff548]
>
>*/usr/bin/traffic_server(TSHttpTxnDebugGet+0x30)[0x50c254]*
>/usr/lib64/trafficserver/plugins/request_filter.so(_ZN20RequestFilterConte
>xt9matchListERSt6vectorIP20RequestFilterElementSaIS2_EE+0x59c)[0x2b0fe5f97
>b42]
>/usr/lib64/trafficserver/plugins/request_filter.so(_ZN20RequestFilterConte
>xt14matchAllowListEv+0x23)[0x2b0fe5f97c7d]
>
>*/usr/lib64/trafficserver/plugins/request_filter.so(TSRemapDoRemap+0x1ec)[
>0x2b0fe5fa0ad7]*
>/usr/bin/traffic_server(_ZN12RemapPlugins10run_pluginEP17remap_plugin_info
>+0x13d)[0x661157]
>... <further stack elements removed for readability> ...
>
>Attaching the process to GDB, I've seen that:
>
>(gdb) f 7
>#7 0x000000000050c254 in TSHttpTxnDebugGet (txnp=0x2b5797e0dcb0) at
>InkAPI.cc:5736
>5736 sdk_assert(sdk_sanity_check_txn(txnp) == TS_SUCCESS);
>(gdb) p txnp
>$2 = (TSHttpTxn) 0x2b5797e0dcb0
>(gdb) p /x ((HttpSM *) txnp)->magic
>$3 = 0xdeadfeed
>
>That is, indeed this handler is no longer valid, as the transaction is
>dead.
>
>Would there be some condition under which the transaction handler was
>supposed to get invalid while a remap plugin is active? Say, if the client
>happens to disconnect exactly after the plugin was activated?
>
>If it may happen, would there be a way for me to assert the handler is
>valid prior to calling a Core function that would otherwise raise a
>segfault? The reason I would like to do this check is that I could make
>only this request fail, instead of making all other possibly thousands of
>requests that are being processed along with this one fail, which is what
>is going to happen should the system abort.
>
>If that was not supposed to happen, any ideas on how I could investigate
>further the reason why the transaction handler became invalid? Obviously
>the plugin is not mangling with the object, as it's an opaque pointer on
>the plugin's perspective, so it must have been something that happened on
>the Core side, I just cant figure out how to pin point where the handler
>became invalid, as this point where it's dying seems to be the first call
>to the Core after the plugin was activated.
>
>We're using version 5.0.1.
>
>Best Regards,
>Acácio Centeno
>Software Engineering
>Azion Technologies
>Porto Alegre, Brasil +55 51 3012 3005 | +55 51 8118 9947
>Miami, USA +1 305 704 8816
>
>Quaisquer informações contidas neste e-mail e anexos podem ser
>confidenciais e privilegiadas, protegidas por sigilo legal. Qualquer forma
>de utilização deste documento depende de autorização do emissor, sujeito
>as
>penalidades cabíveis.
>
>Any information in this e-mail and attachments may be confidential and
>privileged, protected by legal confidentiality. The use of this document
>require authorization by the issuer, subject to penalties.
Re: How to debug a TSHttpTxn handler that is becoming invalid
Posted by Acácio Centeno <ac...@azion.com>.
The naming is just coincidence, it's not a transformation plugin, but just
a simple plugin that denies/allows requests based on the presence/absence
of specific headers. It's a remap plugin because we need different
configurations for different mappings, otherwise it could have been a
simple header manipulation plugin.
However, I think it may be the case that the TSRemapDoRemap function was
activated twice, the second time with an invalid transaction handler, and I
will add some code to either prove or disprove this theory. I'll let you
guys know in either case.
Thanks.
Acácio Centeno
Software Engineering
Azion Technologies
Porto Alegre, Brasil +55 51 3012 3005 | +55 51 8118 9947
Miami, USA +1 305 704 8816
Quaisquer informações contidas neste e-mail e anexos podem ser
confidenciais e privilegiadas, protegidas por sigilo legal. Qualquer forma
de utilização deste documento depende de autorização do emissor, sujeito as
penalidades cabíveis.
Any information in this e-mail and attachments may be confidential and
privileged, protected by legal confidentiality. The use of this document
require authorization by the issuer, subject to penalties.
On Mon, Aug 18, 2014 at 11:43 AM, Nick Kew <ni...@apache.org> wrote:
> On Mon, 18 Aug 2014 10:24:02 -0300
> Acácio Centeno <ac...@azion.com> wrote:
>
> > [chop]
>
> Your plugin is called request_filter and appears to involve methods of
> a RequestFilter object. Is this a red herring, or does this crash in fact
> originate in a filter?
>
> The reason for the question is that this looks like a close cousin to two
> crashes we've encountered recently in filter functions. Maybe the fix
> suggested in the thread at
> http://comments.gmane.org/gmane.comp.apache.trafficserver.devel/1920
> might be worth a try?
>
>
> --
> Nick Kew
>
Re: How to debug a TSHttpTxn handler that is becoming invalid
Posted by Nick Kew <ni...@apache.org>.
On Mon, 18 Aug 2014 10:24:02 -0300
Acácio Centeno <ac...@azion.com> wrote:
> [chop]
Your plugin is called request_filter and appears to involve methods of
a RequestFilter object. Is this a red herring, or does this crash in fact
originate in a filter?
The reason for the question is that this looks like a close cousin to two
crashes we've encountered recently in filter functions. Maybe the fix
suggested in the thread at
http://comments.gmane.org/gmane.comp.apache.trafficserver.devel/1920
might be worth a try?
--
Nick Kew