[GitHub] charleshm opened a new issue #9266: something wrong with SparseEmbedding?

[GitBox <gi...@apache.org>]

charleshm opened a new issue #9266: something wrong with SparseEmbedding?
URL: https://github.com/apache/incubator-mxnet/issues/9266
 
 
   I wrote a embedding version of fm, but got bad results. I checked my code several times,  didn't find any bug. anybody help check it ? 
   
   `    """ builds factorization machine network with proper formulation:
       y = w_0 \sum(x_i w_i) + 0.5(\sum\sum<v_i,v_j>x_ix_j - \sum<v_iv_i>x_i^2)
       F : field_size
       K : factor_size
       """
       x = mx.symbol.Variable("data") # None * F
       # ?????: D * K
       v = mx.symbol.Variable("v", shape=(num_features, factor_size), stype='row_sparse',
                              init=init_config['v'], lr_mult=lr_mult_config['v'],
                              wd_mult=wd_mult_config['v'])
       # ????: D * 1
       w = mx.symbol.var('w', shape=(num_features, 1), stype='row_sparse',
                         init=init_config['w'], lr_mult=lr_mult_config['w'],
                         wd_mult=wd_mult_config['w'])
       # ??bias
       w0 = mx.symbol.var('w0', shape=(1,), init=init_config['w0'],
                          lr_mult=lr_mult_config['w0'], wd_mult=wd_mult_config['w0'])
       
       # wx + b
       y_first_order = mx.symbol.contrib.SparseEmbedding(data=x, weight=w, \
                       input_dim=num_features, output_dim=1)  # None * F * 1
       y_first_order = mx.symbol.broadcast_add(y_first_order.sum(axis=1), w0) # None * F
   
       # squared terms for subtracting self interactions
       embedding = mx.symbol.contrib.SparseEmbedding(data=x, weight=v, \
                       input_dim=num_features, output_dim=factor_size)  # None * F * K
       # ???
       squared_sum_part = mx.symbol.sum(embedding.square(), axis=1)    # None * k
       # ???
       sum_squared_part = mx.symbol.square(data=embedding.sum(axis=1)) # None * K
       y_second_order   = 0.5* (sum_squared_part - squared_sum_part)   # None * K
   
       # putting everything together
       output = mx.symbol.Concat(y_first_order, y_second_order, dim=1)
       model  = output.sum(axis=1, keepdims=True)
   
       y = mx.symbol.Variable("softmax_label")
       model = mx.symbol.LogisticRegressionOutput(data=model, label=y)`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services