[jira] [Closed] (TS-3970) Core in PluginVC

["Susan Hinrichs (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TS-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Susan Hinrichs closed TS-3970.
------------------------------
    Resolution: Invalid

Turns out the problem was in the plugin.  Their clean up code called TSIOBufferDestroy() before TSIOBufferReaderFree() on a reader for that buffer.  

This revealed a use after free error that was found quickly with ASAN.  With this error, it was possible to have the Buffer reallocated and effectively have readers on the newly reallocated buffer cleared randomly.

> Core in PluginVC
> ----------------
>
>                 Key: TS-3970
>                 URL: https://issues.apache.org/jira/browse/TS-3970
>             Project: Traffic Server
>          Issue Type: Bug
>            Reporter: Susan Hinrichs
>            Assignee: Susan Hinrichs
>              Labels: crash
>             Fix For: 6.1.0
>
>         Attachments: ts-3970.diff
>
>
> One of our plugins moving from 5.0.1 to 5.3.x (plus 6.0 backports) started seeing the following stack trace with high frequency.
> {code}
> Program terminated with signal 11, Segmentation fault.
> #0 0x000000000054c232 in PluginVC::process_read_side (this=0x2b9ace2f3850, other_side_call=true) at PluginVC.cc:638
> in PluginVC.cc
> #0 0x000000000054c232 in PluginVC::process_read_side (this=0x2b9ace2f3850, other_side_call=true) at PluginVC.cc:638
> #1 0x000000000054be2a in PluginVC::process_write_side (this=0x2b9ace2f3a40, other_side_call=false) at PluginVC.cc:555
> #2 0x000000000054acdb in PluginVC::main_handler (this=0x2b9ace2f3a40, event=1, data=0x2b9b1e32e930) at PluginVC.cc:208
> #3 0x0000000000510c84 in Continuation::handleEvent (this=0x2b9ace2f3a40, event=1, data=0x2b9b1e32e930) at ../iocore/eventsystem/I_Continuation.h:145
> #4 0x000000000079a2a6 in EThread::process_event (this=0x2b9ab63d9010, e=0x2b9b1e32e930, calling_code=1) at UnixEThread.cc:128
> #5 0x000000000079a474 in EThread::execute (this=0x2b9ab63d9010) at UnixEThread.cc:179
> #6 0x0000000000799851 in spawn_thread_internal (a=0x2fea360) at Thread.cc:85
> #7 0x00002b9ab457e9d1 in start_thread () from /lib64/libpthread.so.0
> #8 0x00000030d38e88fd in clone () from /lib64/libc.so.6
> {code}
> The output buffer fetched by PluginVC::process_read_side was NULL.
> I think they reason this appears in 5.3 is due to the fix for TS-3522.  Before that change only one do_io_read was made very early to set up the read from server.  This bug fix delays the real read to later and pulls mbuf out of server_buffer_reader. In some cases for this plugin, the mbuf is NULL by the time we get there.
> I fixed the core by using server_session->read_buffer in the do_io_read instead of server_buffer_reader->mbuf.  This seems to fix the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)