[GitHub] reminisce opened a new pull request #10139: [REQUEST FOR REVIEW | DO NOT MERGE] [MXNET-109] Logging APIs for Visualizing MXNet Data in TensorBoard

[GitBox <gi...@apache.org>]

reminisce opened a new pull request #10139: [REQUEST FOR REVIEW | DO NOT MERGE] [MXNET-109] Logging APIs for Visualizing MXNet Data in TensorBoard
URL: https://github.com/apache/incubator-mxnet/pull/10139
 
 
   ## Description ##
   This PR implements Python APIs of logging MXNet data for visualization in [TensorBoard](https://www.tensorflow.org/programmers_guide/summaries_and_tensorboard).
   - Design proposal: https://cwiki.apache.org/confluence/display/MXNET/Logging+MXNet+Data+for+Visualization+in+TensorBoard
   - Jira link: https://issues.apache.org/jira/browse/MXNET-109
   
   ### Why choose TensorBoard as the frontend for data visualization? ###
   - TensorBoard has a nice GUI for data visualization and it's production environment friendly.
   - Installing TensorBoard is as simple as typing `pip install tensorboard` in a terminal.
   
   ### How are MXNet logging APIs implemented? ###
   - The major logging components are ported from their Python and C++ implementations in [TensorFlow](https://github.com/tensorflow/tensorflow) as documented in the code.
   - The algorithm of encoding data written to event files are borrowed from https://github.com/TeamHG-Memex/tensorboard_logger.
   - We adopted the user-level logging API design from https://github.com/lanpa/tensorboard-pytorch.
   - This work is inspired by @zihaolucky 's pioneering efforts on [dmlc/tensorboard](https://github.com/dmlc/tensorboard). Special thanks to him for many technical discussions during the process of development.
   
   ### How to use MXNet logging APIs? ###
   Please find the details in the following document: https://github.com/reminisce/mxnet/blob/tensorboard_logging/docs/api/python/contrib/summary.md
   
   ### Is there an example demonstrating logging data for visualization in a training job? ###
   Yes, please see here for logging data when training an MNIST model using the Gluon interfaces.
   https://github.com/reminisce/mxnet/blob/tensorboard_logging/example/visualization/tensorboard/mnist.py. Search for `sw` (short for `SummaryWriter`) in the code to see how logging APIs are used.
   
   ## References ##
   1. https://github.com/dmlc/tensorboard
   2. https://github.com/TeamHG-Memex/tensorboard_logger
   3. https://github.com/lanpa/tensorboard-pytorch
   4. https://github.com/tensorflow/tensorflow
   
   Please help to review and feel free to @ more reviewers. Thanks!
   @piiswrong @szha @eric-haibin-lin @zhreshold @astonzhang @zihaolucky @cjolivier01 @marcoabreu @anirudh2290 @rahul003 @haojin2 
   
   
   ## Checklist ##
   ### Essentials ###
   - [ ] Passed code style checking (`make lint`)
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
   - [x] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments are documented. 
   - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
   - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services