[jira] [Commented] (TS-1080) Assert under heavy load with logging enabled

["Zhao Yongming (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TS-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221534#comment-13221534 ] 

Zhao Yongming commented on TS-1080:
-----------------------------------

hmm, in this case we have fill up the accepting queue with the 512*4 buffers before the flush thread flushed the flush queue, if we don't increase FLUSH_ARRAY_SIZE, we should have 2 options:
1, drop the buffer
  this may be a good solution for me, as it is better than crashing.
2, speed up the flush thread
  well, we will run into another complex direction, we can increase flush thread? or how can we flush at a higher speed while the IO is limited?
                
> Assert under heavy load with logging enabled
> --------------------------------------------
>
>                 Key: TS-1080
>                 URL: https://issues.apache.org/jira/browse/TS-1080
>             Project: Traffic Server
>          Issue Type: Bug
>          Components: Logging
>            Reporter: Leif Hedstrom
>            Priority: Critical
>             Fix For: 3.1.4
>
>
> Given enough load (in the 100,000 QPS or more), we run out of some sort of buffer space, with an assert of
> {code}
> #0  0x00002ba719d50285 in __GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> #1  0x00002ba719d51b9b in __GI_abort () at abort.c:91
> #2  0x00000000006b561a in ink_die_die_die (retval=<optimized out>) at ink_error.cc:43
> #3  ink_fatal_va(int, const char *, typedef __va_list_tag __va_list_tag *) (return_code=<optimized out>, message_format=<optimized out>, ap=0x7fff7275a7d8) at ink_error.cc:65
> #4  0x00000000006b56a7 in ink_fatal (return_code=<optimized out>, message_format=<optimized out>) at ink_error.cc:73
> #5  0x00000000006b4970 in _ink_assert (a=0x6fd380 "_num_flush_buffers[_open_flush_array] < FLUSH_ARRAY_SIZE", f=<optimized out>, l=96) at ink_assert.cc:44
> #6  0x00000000005a8b34 in add_to_flush_queue (buffer=0x2ba7443ca970, this=0x22fb918) at LogObject.h:96
> #7  LogObject::_checkout_write (this=0x22fb880, write_offset=0x7fff7275add8, bytes_needed=152) at LogObject.cc:455
> #8  0x00000000005a8fd3 in LogObject::log (this=0x22fb880, lad=0x7fff7275b030, text_entry=0x0) at LogObject.cc:580
> #9  0x000000000058e956 in log (lad=0x7fff7275b030, this=<optimized out>) at LogObject.h:475
> #10 Log::access (lad=0x7fff7275b030) at Log.cc:1086
> {code}
> Increasing FLUSH_ARRAY_SIZE alleviates the problem, but really, we shouldn't end up in this situation at all.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira