[GitHub] GSanchis commented on issue #8669: module.Module and CSVIter

[GitBox <gi...@apache.org>]

GSanchis commented on issue #8669: module.Module and CSVIter
URL: https://github.com/apache/incubator-mxnet/issues/8669#issuecomment-350022613
 
 
   Sorry, Eric. I had to move to another task, so I didn't have time to look into that example until now.
   
   I just did.
   
   The problem with this example is that the training data is loaded fully into memory, so if the amount of data is very large (in my case a couple of GBs) I run into high memory requirements just because of loading the data. That's why I was trying to use CSVIter, which I was expecting to iterate through the CSV file (and not load it into memory directly). But if there's any other solution that does not load all the data into memory, that would work too.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services