[GitHub] GSanchis commented on issue #8669: module.Module and CSVIter

[GitBox <gi...@apache.org>]

GSanchis commented on issue #8669: module.Module and CSVIter
URL: https://github.com/apache/incubator-mxnet/issues/8669#issuecomment-358917324
 
 
   After quite some time stumbling into this problem, apparently at random, I believe I have found out what the problem was, after reading [this](https://github.com/apache/incubator-mxnet/blob/master/example/recommenders/demo1-MF.ipynb) example, which imports [this](https://github.com/apache/incubator-mxnet/blob/master/example/recommenders/movielens_data.py) module:
   
   ```
   def max_id(fname):
       mu = 0
       mi = 0
       for line in open(fname):
           tks = line.strip().split('\t')
           if len(tks) != 4:
               continue
           mu = max(mu, int(tks[0]))
           mi = max(mi, int(tks[1]))
       return mu + 1, mi + 1
   ```
   then:
   ```
   max_user, max_item = max_id('./ml-100k/u.data')
   ```
   Finally:
   ```
       user = mx.symbol.Embedding(data = user, input_dim = max_user, output_dim = k) 
       item = mx.symbol.Embedding(data = item, input_dim = max_item, output_dim = k)
   ```
   
   So if the maximum user id in the data is e.g. 10, then the `input_dim` for the Embedding should be 11. Is that correct? I would say this is not very intuitive, especially given that the [documentation](https://mxnet.incubator.apache.org/api/python/symbol.html#mxnet.symbol.Embedding) for the Embedding operator is focused on the word embedding case. In my case, a "vocabulary" of {1..10} seems to require `input_dim=11`. Am I correct? Is it also the case for the sparse embeddings? If so, perhaps the documentation could be updated so that future users don't stumble into this. I am willing to help with some text, to the extent I am able to.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services