You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2019/08/09 02:16:17 UTC
[GitHub] [incubator-mxnet] juliusshufan edited a comment on issue #15757:
[Discussion] Unified performance tests and dashboard
juliusshufan edited a comment on issue #15757: [Discussion] Unified performance tests and dashboard
URL: https://github.com/apache/incubator-mxnet/issues/15757#issuecomment-519751336
Update some benchmark and accuracy test from Intel side.
Currently, we track the performance, accuracy and convergence of the MXNet github repo nightly, covering different models and MXNet Op. The kernel level performance is also measured with MKLDNN upgrade. The performance measurement on Xeon platform, covering the "top-bin" and "main-stream" SKUs. The scripts involve the internals and also levergate the public MXNet examples.
The performance report normally compared and presented by,
1. day-to-day comparison, if the performance fluctuation exceeds a preset threshold (model level normally 10%, accuracy is 0 gap), it will raise an suspicious regression;
2. Long-term trends tracking, The recent 30-day performance are presented as a curve;
3. The most recent nightly performance data will be the default criteria for the internal CI test and comparison target.
Detailed HW spec we used for performance tracking in below table, we using CentOS 7.5 and metal machine using below HW spec.
| Socket | Physical Core | HT | Turbo | RAM | RAM Slot | Memory Bandwidth
-- | -- | -- | -- | -- | -- | -- | --
SKX-8180 | 2 | 28 | On | On | DDR4 2666 | 2*6 | 255GB/s
SKX-6148 | 2 | 20 | On | On | DDR4 2666 | 2*6 | 255GB/s
CLX-8280 | 2 | 28 | On | On | DDR4 2933 | 2*6 | 281GB/s
CLX-8260 | 2 | 24 | On | On | DDR4 2933 | 2*6 | 281GB/s
CLX-6248 | 2 | 20 | On | On | DDR4 2666 | 2*6 | 255GB/s
To reflect the real production scenario, the SW configurations we used for performance tracking,
| target | socket/core/instance configuration
-- | -- | --
inference | throughput | 1 instance 1 socket
throughput | 2 instance 2 socket
latency | 1 instance 1 socket
latency | 1 instance low core
latency | multi-instance low core
training | throughput | 2 socket 1 instance
Other details we may discuss offline. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services