[GitHub] ThomasDelteil commented on a change in pull request #10955: [MXNET-422] Distributed training tutorial

[GitBox <gi...@apache.org>]

ThomasDelteil commented on a change in pull request #10955: [MXNET-422] Distributed training tutorial
URL: https://github.com/apache/incubator-mxnet/pull/10955#discussion_r188728690
 
 

 ##########
 File path: example/distributed_training/README.md
 ##########
 @@ -0,0 +1,231 @@
+# Distributed Training using Gluon
+
+Deep learning models are usually trained using GPUs because GPUs can do a lot more computations in parallel that CPUs. But even with the modern GPUs, it could take several days to train big models. Training can be done faster by using multiple GPUs like described in [this](https://gluon.mxnet.io/chapter07_distributed-learning/multiple-gpus-gluon.html) tutorial. However only a certain number of GPUs can be attached to one host (typically 8 or 16). To make the training even faster, we can use multiple GPUs attached to multiple hosts.
+
+In this tutorial, we will show how to train a model faster using multi-host distributed training.
+
+![Multiple GPUs connected to multiple hosts](https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/example/distributed_training/distributed_training.png)
+
+We will use data parallelism to distribute the training which involves splitting the training data across GPUs attached to multiple hosts. Since the hosts are working with different subset of the training data in parallel, the training completes lot faster.
 
 Review comment:
   a lot faster*

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services