[jira] [Commented] (SYSTEMML-540) Deep Learning

["Mike Dusenberry (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SYSTEMML-540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375967#comment-15375967 ] 

Mike Dusenberry commented on SYSTEMML-540:
------------------------------------------

[~niketanpansare] What are your thoughts on the proposed plan I added to the description?  I basically tried to condense all of the discussions we've had down to a short timeline.  It would be nice to include some timing to it as well.

> Deep Learning
> -------------
>
>                 Key: SYSTEMML-540
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-540
>             Project: SystemML
>          Issue Type: Epic
>            Reporter: Mike Dusenberry
>            Assignee: Mike Dusenberry
>
> This epic covers the addition of deep learning to SystemML, including:
> * Core DML layer abstractions for deep (convolutional, recurrent) neural nets, with simple forward/backward API: affine, convolution (start with 2D), max-pooling, non-linearities (relu, sigmoid, softmax), dropout, loss functions.
> * Modularized DML optimizers: (mini-batch, stochastic) gradient descent (w/ momentum, etc.).
> * Additional DML language support as necessary (tensors, built-in functions such as convolution, function pointers, list structures, etc.).
> * Integration with other deep learning frameworks (Caffe, Torch, Theano, TensoFlow, etc.) via automatic DML code generation.
> * etc.
> ---
> *Plan*:
> \[*DONE*\] Phase 1:  *MVPs*
> * Create mathematically correct DML deep learning library for running basic feed-forward and convolutional neural nets on a singlenode.
> * Create mathematically correct built-in operators for convolution and max pooling for singlenode operation.
> \[*CURRENT*\] Phase 2:  *Singlenode*
> * Improve performance of DML deep learning library in singlenode operation.
> * Expand DML deep learning library to include additional commonly-used layers, such as RNNs and LSTMs, as well as additional optimizers.
> * Improve built-in operators for convolution and max pooling to be highly performant in singlenode operation.
> * Implement performant GPU acceleration for built-in operators (and end-to-end deep learning algorithms) in singlenode operation.
> * Add general engine improvements to improve bottlenecks, such as left-indexing within DML-bodied functions.
> * Add end-to-end deep learning algorithm examples, such as a "LeNet" convolutional neural net.
> Phase 3: *Distributed*
> * Expand deep learning support to include *distributed operations* with large models.  This includes improvements to the DML deep learning library, the built-in operators, the GPU acceleration, and general engine improvements.
> Phase 4: *APIs/Wrappers*
> * Explore integration with Caffe, creating a SystemML interpreter for Caffe model definitions.
> * Explore integration with Keras, creating a SystemML backend for Keras.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)