[jira] [Commented] (TS-702) FATAL: MIME.cc:1250: failed assert `j < block_count`

["Yakov Markovitch (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TS-702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13021570#comment-13021570 ] 

Yakov Markovitch commented on TS-702:
-------------------------------------

This bug is in the algorithm of adjusting pointers inside MIME field blocks after copying MIME headers. mime_hdr_copy_onto, which is directly called from TSMimeHdrCopy and indirectly from TSMimeHdrClone, copies MIME headers as raw memory blocks and then calls mime_hdr_field_block_list_adjust to adjust fields' internal pointers to duplicates (MIMEField::m_next_dup). 

The algorithm of mime_hdr_field_block_list_adjust is both complicated and incorrect, since it implicitly assumes relative offsets between original and copied blocks do not change. Worse is, they most often actually do not change, accidentally: the source blocks allocated continuously, as well as target blocks. The bug is very insidious and hard to reproduce, to manifest itself it needs _all_ the following:

 1. There must be more than one block allocated for a header, i.e. the header must contain more than MIME_FIELD_BLOCK_SLOTS (==16) fields.
 2. There must be duplicated fields in the header, and at least some of those fields must be allocated not in the first block.
 3. Relative offsets between target heap blocks must not match offsets between source blocks.

I've attached a patch that fixes the bug. The patch was tested in production environment.

> FATAL: MIME.cc:1250: failed assert `j < block_count` 
> -----------------------------------------------------
>
>                 Key: TS-702
>                 URL: https://issues.apache.org/jira/browse/TS-702
>             Project: Traffic Server
>          Issue Type: Bug
>          Components: MIME
>    Affects Versions: 2.0.1
>         Environment: Sun Blade X6240 (64G Ram), Running Linux Debian Lenny, Kernel  2.6.26-2-amd64 
>            Reporter: Ricky Chan
>            Priority: Critical
>         Attachments: trafficserver.2.1.7.too.many.mimefields.patch
>
>
> I have 20 servers in a CDN farm which I brought live recently, I have noticed with in a day 5 servers had this error reported in traffic.out.  Essentially aborting due to assertion failure.  The server restarts (from traffic_cop).
> This platform has not had any load go through it yet, it's running around 5MB/s a second with around 25 connection per second.  So doing much.
> I was going to migrate more traffic onto it, but holding off due to this assertion issue.
> Looking at the code, we have:
> for (j=0; j < block_count; j++) {
>  ... with a condition which breaks out ..
> }
> ink_release_assert(j < block_count) 
> So for this assert to be hit the entire list must be gone through without triggering the break clause, i.e. j == block_count
> I don't know this code well, is this a real bug or should the assert be there (or j <= block_count)?

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira