[jira] [Created] (SINGA-502) Avoid moving data between host and gpu devices in some operations

["Chris Yeung (Jira)" <ji...@apache.org>]

Chris Yeung created SINGA-502:
---------------------------------

             Summary: Avoid moving data between host and gpu devices in some operations
                 Key: SINGA-502
                 URL: https://issues.apache.org/jira/browse/SINGA-502
             Project: Singa
          Issue Type: Improvement
          Components: Core
            Reporter: Chris Yeung


Some functions move data between GPU and host memory, which should be fixed for many reasons such as efficiency and asynchronization (and buffering operation in the future). For example:

The softmax_cross_entropy move to data to host and then back to gpu, so the whole function needed to be changed:

class SoftMaxCrossEntropy(Operation):   

def __init__(self, t):       
  super(SoftMaxCrossEntropy, self).__init__()       
  self.t = t.data

def forward(self, x):       
  self.p = singa.SoftMax(x)       
  loss = CTensor((1,), self.p.device())       
  ret = singa.CrossEntropyFwd(self.p, self.t)       
  loss.SetFloatValue(singa.SumAsFloat(ret) / x.shape()[0])       
  return loss

Here the SumAsFloat return a c++ float value,  and this value is read back to gpu in the SetFloatValue function. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)