[GitHub] squidszyd commented on issue #12141: C_API for C++ iterator only supports one data and one label for one sample

[GitBox <gi...@apache.org>]

squidszyd commented on issue #12141: C_API for C++ iterator only supports one data and one label for one sample
URL: https://github.com/apache/incubator-mxnet/issues/12141#issuecomment-442360279
 
 
   Hi @leleamol,
   Sorry for the late reply.
   Basically in my use case, I want to implement a C++ data loader and use it with Python interface:
   ```
   // I have my C++ data parser here ...
   class MyDataParser {
     // parse data from MXNet's record file using InputSplit class 
   };
   
   // and my DataLoader
   class MyDataLoader : public IIterator<DataInst> {
     // ...  
     // Init function inherited from IIterator class
     virtual void Init(const std::vector<std::pair<std::string, std::string>>& kwargs) {
       param_.InitAllowUnknown(kwargs);
       parser_.Init(kwargs); // initialize MyDataParser
       iter_.set_max_capacity(4);
       // Init() takes two arguments, the first is a function object describes how single data instance are parsed
       // the second argument is a function object describes how to restart
       iter_.Init(
         [this](std::vector<MyDataInstVector<DType>>** dptr) {
           if(*dptr == nullptr) {
             *dptr = new std::vector<MyDataInstVector<DType>>();
           }
           return parser_.ParseNext(*dptr);
       },
       [this]() {parser_.BeforeFirst(); });
     }
   };
   ```
   You should have noticed that I've defined my own class for a collection of data instances - `MyDataInstVector`. The reason is that the provided type `DataInst` that comes with original MxNet does not meet my needs. It can only store one label for one data.
   
   The above actually has nothing to do with MxNet's C_API. 
   But the point is that if I want to use Python Interface to use my custom data loader, then I have to use C_API, which does not support multi-labels use case.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services