You are viewing a plain text version of this content. The canonical link for it is here.

Posted to discuss-archive@mxnet.apache.org by gammaSquared via MXNet Forum <mx...@discoursemail.com.INVALID> on 2020/07/09 06:30:42 UTC

[MXNet Forum] [Gluon] Custom dataset similar to Python generator?


Hi!

I am coming from Keras/Tensorflow and would like to move to mxnet. So far it looks awesome.

I am however wondering if there is any possibility to set up a custom dataset where the data is generated similar to a Python generator? I am working on time series data and reading it all into memory and storing it in a 2D array is not an option. In Tensorflow this is possible: https://www.tensorflow.org/guide/data#consuming_python_generators

Somewhere I read that using a custom dataset and dataloader this should be no problem? The API will not read all the data into memory at once.

Finally, I found this post https://guertl.me/post/183427090170/custom-dataset-and-dataloader-in-gluon on how to create custom datasets. Could you recommend me any other tutorials?

This is my first post so please let me know if I have done something wrong. Thanks!





---
[Visit Topic](https://discuss.mxnet.io/t/custom-dataset-similar-to-python-generator/6395/1) or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.mxnet.io/email/unsubscribe/3ba23fb9196ed6dfb720843488b2c6c759ddae9927a744e4e839d4e01a02a134).

[MXNet Forum] [Gluon] Custom dataset similar to Python generator?

Posted by gammaSquared via MXNet Forum <mx...@discoursemail.com.INVALID>.


Thanks! I'll check it out. :slight_smile:





---
[Visit Topic](https://discuss.mxnet.io/t/custom-dataset-similar-to-python-generator/6395/3) or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.mxnet.io/email/unsubscribe/bccce463b4a9b74e546afcaa47a660341ec0fd8d65dacec7a752ab5ecd26e546).

[MXNet Forum] [Gluon] Custom dataset similar to Python generator?

Posted by Foivos Diakogiannis via MXNet Forum <mx...@discoursemail.com.INVALID>.


If you search in the forum you'll find several discussions about custom datasets. This may also help: 

https://www.linkedin.com/pulse/semantic-segmentation-tutorial-mxnetgluon-part-i-foivos-diakogiannis/





---
[Visit Topic](https://discuss.mxnet.io/t/custom-dataset-similar-to-python-generator/6395/2) or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.mxnet.io/email/unsubscribe/3e7ebe737a356c37d8989e130af35fcddea254560a5888265c872b6819d6bc41).

[MXNet Forum] [Gluon] Custom dataset similar to Python generator?

Posted by seema singh via MXNet Forum <mx...@discoursemail.com.INVALID>.


Nice to read about the Custom dataset similar to Python generator. 
If I already know what a generator function is, and want to learn about how to use it, to create a custom data generator given link by you can proceed. However, if I don’t know what a generator function is, or want a refresher, quickly go through here: https://www.cetpainfotech.com/technology/python-training





---
[Visit Topic](https://discuss.mxnet.io/t/custom-dataset-similar-to-python-generator/6395/4) or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.mxnet.io/email/unsubscribe/2810083e9d0c0c9b79b1fd591324f22f86a4394f1be7f439077f948b5cfa49b9).