Log level adjustments

[Martin Henke <ma...@web.de>]

Hi,

we have showed in recent dev calls that the amount of logs the 
controllers and invoker write out to the log files does harm the 
performance under high load severely.

To reduce that burden we introduced the new metric infrastructure based 
on Kamon
 which is already delivered (but switched off by default).



Additionally we now pushed PR

https://github.com/apache/incubator-openwhisk/pull/2960

which reassigns the level of the remaining log messages. 



Both changes reduce the amount of messages logged at INFO level (which 
is the default) to about a 5th of the original size. Our tests show that 
the performance under load improves by at least a factor of two.



In the PR we aimed for better error logging and to still ensure 
maintainability of the system.

We would like to emphasize that with that change it is more important to 
have a monitoring infrastructure in place (as a starter the document 
‘metrics.md’ in the repo explains how to test the Kamon and statsd based 
metrics in your local environment).



Please leave your feedback as comment to the PR or as answer to this 
message.



Kind regards,

Vadim and Martin