Re: long short term memory in singe

[Wang Wei <wa...@gmail.com>]

Welcome to join us!

Please follow the readme file
https://github.com/apache/incubator-singa/tree/dev for installation and a
CNN example.
We will add more documentation next week and prepare the release by the end
of this month.

Thanks.

regards,


On Tue, Jul 19, 2016 at 2:02 PM, Shayan Shams <ss...@lsu.edu> wrote:

> Thats a good news that you are working on lstm and gru implementation and
> im more than glad to help in developing that. I have been following your
> development on bersion 1 and i really like it. Very good job. Just to be
> clear in version one you will have both GRU and also Lstm im asking because
> in the current version you just have the GRU
> Thanks alot
> And i start working on version1 asap
>
> Shayan Shams
>
> On Jul 19, 2016, at 00:58, Wang Wei <wa...@gmail.com> wrote:
>
> Yes. We will support CPU and GPU for CNN and RNN.
> The v0.3 supports CPU and GPU for CNN and GRU.
> We are working on v1.0 which would be much easier to use and to customize.
> The GPU part of RNN (including LSTM/GRU/TANH) would be ready next week.
> I am busy on the v1.0 development, and may not have time to work on LSTM
> for v0.3 until the end of this month.
> Sorry about that..
>
> I would encourage you to try v1.0 and help us to improve it as you are
> familiar with LSTM and GRU.
>
> regards,
> Wei
>
>
> On Tue, Jul 19, 2016 at 1:44 PM, Shayan Shams <ss...@lsu.edu> wrote:
>
>>
>> Hi Wei
>> Thanks for reply i am using lstm on the top of cnn for some medical image
>> data I do have gpu but i prefer to have both gpu and cpu code and as far as
>> i know you guys have gru but not lstm implemented and another reason i have
>> changed the cnn codes and im running them on cpu so it would be great if i
>> can run both of them on cpu
>> Shayan Shams
>>
>> On Jul 19, 2016, at 00:22, Wang Wei <wa...@gmail.com> wrote:
>>
>> Hi Shanyan,
>>
>> May I know your purpose or application of running lstm?
>> We are working on RNN (LSTM/TANH/GRU) using cuDNN
>> https://github.com/apache/incubator-singa/pull/203
>>
>> If you have GPU, I think it would be more efficient to use cuDNN v5 for
>> RNN applications.
>>
>> regards,
>> Wei
>>
>> On Mon, Jul 18, 2016 at 12:53 PM, Shayan Shams <ss...@lsu.edu> wrote:
>>
>>> Dear all,
>>>
>>> I am writing to ask for you consultation about a problem that I have
>>> been faced with. I tried to use the current GRU implementation and change
>>> it a little bit to create  LSTM on singa, I have created the following
>>> files in singe
>>>
>>> "lstm.cc"
>>>
>>> lstm section in "job.proto"
>>>
>>> lstm connection layer in "neuralnet.h"
>>>
>>> lstm layer in "neuron_layer.h"
>>>
>>> and also I created one function called mem to return mem_ in "layer.h"
>>>
>>> I have registered my layer in "driver.cc"
>>>
>>> I have attached all the files.
>>>
>>> I was able to recompile the singa but when I tried to run the char-rnn
>>> example for which the conf file is there with just changing the gru layer
>>> to lstm,
>>>
>>> it is compiled and running without any error but after some steps of
>>> training the gradvec[1] is getting so big  that I get nan for lost. I
>>> tried different parameter and if I make the learning rate small it works
>>> but it doesnt converge.
>>>
>>> I tried to debug it but i couldnt.
>>>
>>>
>>> so would you mind please take a look at it and guide me in correct
>>> direction, I know its a lot to ask but any help is very much appreciated.
>>>
>>> I sincerely appreciate your kindness and consideration.
>>>
>>> Yours Sincerely,
>>>
>>> Shayan Shams
>>>
>>
>>
>