[GitHub] adrianloy opened a new issue #9034: Understanding the correlation layer

[GitBox <gi...@apache.org>]

adrianloy opened a new issue #9034: Understanding the correlation layer
URL: https://github.com/apache/incubator-mxnet/issues/9034
 
 
   Hey everyone,
   
   I do not understand the impact of the stride parameter for the mxnet.sym.correlation layer. I have this simple minimal example:
   
   ```
   import numpy as np
   import mxnet as mx
   
   # Network
   data1 = mx.symbol.Variable('data1')
   #data2 = mx.symbol.Variable('data2')
   cor = mx.sym.Correlation(data1=data1, data2=data1, kernel_size=28, stride1=1, stride2=1, pad_size=0, max_displacement=0)
   loss = mx.sym.MakeLoss(cor, normalization='batch')
   group = mx.symbol.Group([mx.sym.BlockGrad(cor), loss])
   
   # Data
   datashape = (1, 1, 28, 28)  # like mnist
   data = np.random.rand(1, 1, 28, 28)
   
   #Bind, execute, get computed correlation
   executor = group.simple_bind(ctx=mx.cpu(), data1=datashape)
   outs = executor.forward(is_train=True, data1=data)
   cor = executor.outputs[0]
   grad1 = executor.outputs[1]
   print(cor)
   ```
   
   It computes the correlation between 2 identical feature maps with a kernel of the same size as the feature maps. **I do not understand why the output of the correlation layer has shape 1x1x2x2**. If I set the strides to anything bigger than 1, the output is a 1x1x1x1 matrix, that makes more sense to me. As the Kernel has the size of the feature maps, the output should always be a 1x1x1x1 matrix, independent from the value of the strides parameters! 
   
   The documentation notes
   
   > 
   > Given a maximum displacement :math:`d`, for each location :math:`x_{1}` it computes correlations :math:`c(x_{1}, x_{2})` only in a neighborhood of size :math:`D:=2d+1`,
   >     by limiting the range of :math:`x_{2}`. We use strides :math:`s_{1}, s_{2}`, to quantize :math:`x_{1}` globally and to quantize :math:`x_{2}` within the neighborhood
   >     centered around :math:`x_{1}`.
   > 
   >     The final output is defined by the following expression:
   > 
   >     .. math::
   >       out[n, q, i, j] = c(x_{i, j}, x_{q})
   > 
   >     where :math:`i` and :math:`j` enumerate spatial locations in :math:`f_{1}`, and :math:`q` denotes the :math:`q^{th}` neighborhood of :math:`x_{i,j}`.
   
   Unfortunately I do not really understand from this the impact of the strides parameter, as s_1 and s_2 do not appear in the equation.
   
   Furthermore, if you just run the code a couple of times you will notice that the correlation matrix sometimes contains "Infinity" values. I do not understand mathematically or conceptually how the correlation can be infinity. 
   
   Any help is much appreciated.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services