[jira] [Commented] (SINGA-98) Add Support for AlexNet ImageNet Classification Model

["wangwei (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SINGA-98?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14984412#comment-14984412 ] 

wangwei commented on SINGA-98:
------------------------------

You may need to do some calculations at first for the memory cost of the whole alexnet, including data and grad blob.
Alex used two groups because his GPU had limited memory. The two groups were running on different GPUs.
If our current GPU has enough memory, then we can simulate the two groups training in a single GPU. 
Otherwise, we need to use two workers, one group per worker (one worker per GPU).

For CPU training, memory should not be a problem. But we need to use multiple CPUs to accelerate the training. 
Otherwise it would take weeks to months to finish the training.

You can paste the memory cost of each layer in the ticket description, e.g., batchsize, memory for data blob (in MB), memory for gradient blob (in MB) .

> Add Support for AlexNet ImageNet Classification Model
> -----------------------------------------------------
>
>                 Key: SINGA-98
>                 URL: https://issues.apache.org/jira/browse/SINGA-98
>             Project: Singa
>          Issue Type: New Feature
>            Reporter: Wang Ji
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> This ticket is for ImageNet Classification model a.k.a AlexNet.
> This model is a replication of the CNN model described [AlexNet](http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks) publication.
> This model is used to classify high resolution imagines from ImageNet dataset.
> To implement this model, convolutional layer is needed to add a feature that support for partitioning the input and output channels into groups to stimulate the net structure described in the original publication. Input layer needs features to support data augmentation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)