[GitHub] [incubator-doris] caiconghui edited a comment on pull request #3703: [Load] Add more metric to trace the time cost in stream load and make brpc_num_threads configurable

[GitBox <gi...@apache.org>]

caiconghui edited a comment on pull request #3703:
URL: https://github.com/apache/incubator-doris/pull/3703#issuecomment-637308718


   > @caiconghui Hi, If you only change the webserver_num_workers from 5 to 48, how many times stream load performance will improve?
   @kangkaisen 
   if every stream load perform well, this does not make a apparent improvement. But there always are some long tail task in our online environment, once a task is executed slowly, it will block all the http request on the same thread . So change the thread from 5 to 48 can reduce the extra time cost for waiting slow task to complete, because we use rpc call and furture get function in reactor model thread.(The real improvement for us is that http request for metric never timeout now when we change the thread num from 5 to 42 in our online environment)
   It is hard for us to write non blocking code in libevent callback function now.
   Similar problems can see following:
   https://stackoverflow.com/questions/42689219/does-calling-a-blocking-function-inside-libev-event-callback-function-blocks-who


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org