[jira] [Comment Edited] (TS-2497) Failed post results in tunnel buffers being returned to freelist prematurely

["Brian Geffon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TS-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13984585#comment-13984585 ] 

Brian Geffon edited comment on TS-2497 at 4/29/14 5:58 PM:
-----------------------------------------------------------

[~shaunmcginnity], something is a little unclear to me, the only difference between the two is:

Apr 29 11:05:29 - INFO - free_MIOBuffer 0x5001620
Apr 29 11:05:29 - INFO - free_MIOBuffer 0x5001620 DONE

Which happens after add_producer, can you provide a back trace or something for those two frees? It's not clear where they are coming from, also, are you sure this behavior didn't exist before? Can you also provide output from -T ".*" debug tag? 


was (Author: briang):
[~shaunmcginnity], something is a little unclear to me, the only difference between the two is:

Apr 29 11:05:29 - INFO - free_MIOBuffer 0x5001620
Apr 29 11:05:29 - INFO - free_MIOBuffer 0x5001620 DONE

Which happens after add_producer, can you provide a back trace or something for those two frees? It's not clear where they are coming from, also, are you sure this behavior didn't exist before?

> Failed post results in tunnel buffers being returned to freelist prematurely
> ----------------------------------------------------------------------------
>
>                 Key: TS-2497
>                 URL: https://issues.apache.org/jira/browse/TS-2497
>             Project: Traffic Server
>          Issue Type: Bug
>          Components: Core
>            Reporter: Brian Geffon
>            Assignee: Brian Geffon
>             Fix For: 4.2.0
>
>
> When a post fails to an origin server either the server died or the server returned a response without reading all of the post data, in either case, TS will destroy buffers too early. This normally does not result in a crash because the MIOBuffers are returned to the freelist and only with sufficient load will the race happen causing a crash. Additionally, even if a crash doesn't happen you might have data corruption across post requests from the buffers being used after being returned to the freelist.
> Thanks to Thomas Jackson for help reproducing and resolving this bug.
> An example stack trace, while we've seen other crashes in write_avail too.
> #0  0x00000000004eff14 in IOBufferBlock::read_avail (this=0x0) at ../iocore/eventsystem/I_IOBuffer.h:362
> #1  0x000000000050d151 in MIOBuffer::append_block_internal (this=0x2aab38001130, b=0x2aab0c037200) at ../iocore/eventsystem/P_IOBuffer.h:946
> #2  0x000000000050d39b in MIOBuffer::append_block (this=0x2aab38001130, asize_index=15) at ../iocore/eventsystem/P_IOBuffer.h:986
> #3  0x000000000050d49b in MIOBuffer::add_block (this=0x2aab38001130) at ../iocore/eventsystem/P_IOBuffer.h:994
> #4  0x000000000055cee2 in MIOBuffer::check_add_block (this=0x2aab38001130) at ../iocore/eventsystem/P_IOBuffer.h:1002
> #5  0x000000000055d115 in MIOBuffer::write_avail (this=0x2aab38001130) at ../iocore/eventsystem/P_IOBuffer.h:1048
> #6  0x00000000006c18f3 in read_from_net (nh=0x2aaafca0d208, vc=0x2aab1c009140, thread=0x2aaafca0a010) at UnixNetVConnection.cc:234
> #7  0x00000000006c37bf in UnixNetVConnection::net_read_io (this=0x2aab1c009140, nh=0x2aaafca0d208, lthread=0x2aaafca0a010) at UnixNetVConnection.cc:816
> #8  0x00000000006be392 in NetHandler::mainNetEvent (this=0x2aaafca0d208, event=5, e=0x271d8e0) at UnixNet.cc:380
> #9  0x00000000004f05c4 in Continuation::handleEvent (this=0x2aaafca0d208, event=5, data=0x271d8e0) at ../iocore/eventsystem/I_Continuation.h:146
> #10 0x00000000006e361e in EThread::process_event (this=0x2aaafca0a010, e=0x271d8e0, calling_code=5) at UnixEThread.cc:142
> #11 0x00000000006e3b13 in EThread::execute (this=0x2aaafca0a010) at UnixEThread.cc:264
> #12 0x00000000006e290b in spawn_thread_internal (a=0x2716400) at Thread.cc:88
> #13 0x0000003372c077e1 in start_thread () from /lib64/libpthread.so.0
> #14 0x00000033728e68ed in clone () from /lib64/libc.so.6



--
This message was sent by Atlassian JIRA
(v6.2#6252)