[GitHub] feevos commented on issue #11119: Feature request: please speed up asnumpy()

[GitBox <gi...@apache.org>]

feevos commented on issue #11119: Feature request: please speed up asnumpy()
URL: https://github.com/apache/incubator-mxnet/issues/11119#issuecomment-393761934
 
 
   Hi @reminisce  thank you very much for your reply. I couldn't wait for it to finish (after ~ 15min) I killed it and used asnumpy() in much smaller patches of images (6,256x256) that allows the code to run. 
   
   Some additional information, I am using version ```mxnet_cu90-1.2.0b20180514```.  
   
   This is the source code: I am reading an image that has shape (5,2944,2944) and I am doing inference in patches of it each of size (5,256x256).  I make predictions that I store in a matrix ```tpreds``` of size (6,2944,2944) (Number of Classes, img_size_x, img_size_y). 
   
   ```Python
   # here mynet4 is a ConvNet defined elsewhere in the code, works well, well tested 
   ctx = mx.cpu()
   
   # I need to make this a function, since I'll be applying it in many many different problems. 
   
   # Window size 
   img_size = 2816
   F = 256
   stride = F//4
   
   # I will use a padded image, with reflect padding in the sides equal to F/2
   ijsteps = (F//4 + img_size + F//4 - F) // stride + 1 # How many steps for a complete window (F) with stride F/4 to cover the whole padded row of the image  
   tpreds = nd.zeros(shape = [NClasses,img_size+F//2,img_size+F//2],ctx=ctx) # Corresponds to padded image 
   constants = nd.zeros_like(tpreds,ctx=ctx)
   
   normalize = ISPRSNormal() # Necessary for normalizing img
   # For a single image: 
   img_use = np.pad(imgs[0],pad_width=((0,0),(F//4,F//4),(F//4,F//4)),mode='reflect')
   for i in range(ijsteps):
       print (i)
       for j in range(ijsteps):
           img_patch = nd.array(normalize(img_use[:,i*stride:i*stride+F,j*stride:j*stride+F]),ctx=ctx)
           img_patch = nd.expand_dims(img_patch,axis=0)
           with autograd.predict_mode():
               preds = mynet4(img_patch)
                           
           preds = preds[0]
                           
           # This is the CMA formula - cummulative moving average 
           tpreds[:,i*stride:i*stride+F,j*stride:j*stride+F] = constants[:,i*stride:i*stride+F,j*stride:j*stride+F]*tpreds[:,i*stride:i*stride+F,j*stride:j*stride+F] + preds
           tpreds[:,i*stride:i*stride+F,j*stride:j*stride+F] /= (constants[:,i*stride:i*stride+F,j*stride:j*stride+F]+1)        
           constants[:,i*stride:i*stride+F,j*stride:j*stride+F] += 1
   
   ```
   Now I translate the matrix that has the average prediction probabilities to a 2D array (to speed up things)
   ```Python
   tpreds_use = nd.argmax(tpreds,axis=0) # This is lightning fast, produces a matrix of ~3000x3000
   ```
   and here I am (was) trying to make preds_use into a numpy array
   ```Python
   preds_use = tpreds_use.asnumpy() # This was taking forever :( 
   ```
   
   Thank you very much for your help. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services