You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2018/11/16 01:27:21 UTC
[GitHub] yzhliu closed pull request #13201: [MXNET-1187] Added Java SSD
Inference Tutorial for website
yzhliu closed pull request #13201: [MXNET-1187] Added Java SSD Inference Tutorial for website
URL: https://github.com/apache/incubator-mxnet/pull/13201
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/tutorials/index.md b/docs/tutorials/index.md
index e5e26772064..7ed34fc9d18 100644
--- a/docs/tutorials/index.md
+++ b/docs/tutorials/index.md
@@ -158,6 +158,7 @@ Select API:
## Java Tutorials
* Getting Started
* [Developer Environment Setup on IntelliJ IDE](/tutorials/java/mxnet_java_on_intellij.html)
+* [Multi Object Detection using pre-trained Single Shot Detector (SSD) Model](/tutorials/java/ssd_inference.html)
* [MXNet-Java Examples](https://github.com/apache/incubator-mxnet/tree/master/scala-package/examples/src/main/java/org/apache/mxnetexamples)
<hr>
diff --git a/docs/tutorials/java/ssd_inference.md b/docs/tutorials/java/ssd_inference.md
new file mode 100644
index 00000000000..6bcaaa2504a
--- /dev/null
+++ b/docs/tutorials/java/ssd_inference.md
@@ -0,0 +1,186 @@
+# Multi Object Detection using pre-trained SSD Model via Java Inference APIs
+
+This tutorial shows how to use MXNet Java Inference APIs to run inference on a pre-trained Single Shot Detector (SSD) Model.
+
+The SSD model is trained on the Pascal VOC 2012 dataset. The network is a SSD model built on Resnet50 as the base network to extract image features. The model is trained to detect the following entities (classes): ['aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus', 'car', 'cat', 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike', 'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor']. For more details about the model, you can refer to the [MXNet SSD example](https://github.com/apache/incubator-mxnet/tree/master/example/ssd).
+
+## Prerequisites
+
+To complete this tutorial, you need the following:
+* [MXNet Java Setup on IntelliJ IDEA](/java/mxnet_java_on_intellij.html) (Optional)
+* [wget](https://www.gnu.org/software/wget/) To download model artifacts
+* SSD Model artifacts
+ * Use the following script to get the SSD Model files :
+```bash
+data_path=/tmp/resnet50_ssd
+mkdir -p "$data_path"
+wget https://s3.amazonaws.com/model-server/models/resnet50_ssd/resnet50_ssd_model-symbol.json -P $data_path
+wget https://s3.amazonaws.com/model-server/models/resnet50_ssd/resnet50_ssd_model-0000.params -P $data_path
+wget https://s3.amazonaws.com/model-server/models/resnet50_ssd/synset.txt -P $data_path
+```
+* Test images : A few sample images to run inference on.
+ * Use the following script to download sample images :
+```bash
+image_path=/tmp/resnet50_ssd/images
+mkdir -p "$image_path"
+cd $image_path
+wget https://cloud.githubusercontent.com/assets/3307514/20012567/cbb60336-a27d-11e6-93ff-cbc3f09f5c9e.jpg -O dog.jpg
+wget https://cloud.githubusercontent.com/assets/3307514/20012563/cbb41382-a27d-11e6-92a9-18dab4fd1ad3.jpg -O person.jpg
+```
+
+Alternately, you can get the entire SSD Model artifacts + images in one single script from the MXNet Repository by running [get_ssd_data.sh script](https://github.com/apache/incubator-mxnet/blob/master/scala-package/examples/scripts/infer/objectdetector/get_ssd_data.sh)
+
+## Time to code!
+1\. Following the [MXNet Java Setup on IntelliJ IDEA](/java/mxnet_java_on_intellij.html) tutorial, in the same project `JavaMXNet`, create a new empty class called : `ObjectDetectionTutorial.java`.
+
+2\. In the `main` function of `ObjectDetectionTutorial.java` define the downloaded model path and the image data paths. This is the same path where we downloaded the model artifacts and images in a previous step.
+
+```java
+String modelPathPrefix = "/tmp/resnet50_ssd/resnet50_ssd_model";
+String inputImagePath = "/tmp/resnet50_ssd/images/dog.jpg";
+```
+
+3\. We can run the inference code in this example on either CPU or GPU (if you have a GPU backed machine) by choosing the appropriate context.
+
+```java
+
+List<Context> context = getContext();
+...
+
+private static List<Context> getContext() {
+List<Context> ctx = new ArrayList<>();
+ctx.add(Context.cpu()); // Choosing CPU Context here
+
+return ctx;
+}
+```
+
+4\. To provide an input to the model, define the input shape to the model and the Input Data Descriptor (DataDesc) as shown below :
+
+```java
+Shape inputShape = new Shape(new int[] {1, 3, 512, 512});
+List<DataDesc> inputDescriptors = new ArrayList<DataDesc>();
+inputDescriptors.add(new DataDesc("data", inputShape, DType.Float32(), "NCHW"));
+```
+
+The input shape can be interpreted as follows : The input has a batch size of 1, with 3 RGB channels in the image, and the height and width of the image is 512 each.
+
+5\. To run an actual inference on the given image, add the following lines to the `ObjectDetectionTutorial.java` class :
+
+```java
+BufferedImage img = ObjectDetector.loadImageFromFile(inputImagePath);
+ObjectDetector objDet = new ObjectDetector(modelPathPrefix, inputDescriptors, context, 0);
+List<List<ObjectDetectorOutput>> output = objDet.imageObjectDetect(img, 3); // Top 3 objects detected will be returned
+```
+
+6\. Let's piece all of the above steps together by showing the final contents of the `ObjectDetectionTutorial.java`.
+
+```java
+package mxnet;
+
+import org.apache.mxnet.infer.javaapi.ObjectDetector;
+import org.apache.mxnet.infer.javaapi.ObjectDetectorOutput;
+import org.apache.mxnet.javaapi.Context;
+import org.apache.mxnet.javaapi.DType;
+import org.apache.mxnet.javaapi.DataDesc;
+import org.apache.mxnet.javaapi.Shape;
+
+import java.awt.image.BufferedImage;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+
+public class ObjectDetectionTutorial {
+
+ public static void main(String[] args) {
+
+ String modelPathPrefix = "/tmp/resnet50_ssd/resnet50_ssd_model";
+
+ String inputImagePath = "/tmp/resnet50_ssd/images/dog.jpg";
+
+ List<Context> context = getContext();
+
+ Shape inputShape = new Shape(new int[] {1, 3, 512, 512});
+
+ List<DataDesc> inputDescriptors = new ArrayList<DataDesc>();
+ inputDescriptors.add(new DataDesc("data", inputShape, DType.Float32(), "NCHW"));
+
+ BufferedImage img = ObjectDetector.loadImageFromFile(inputImagePath);
+ ObjectDetector objDet = new ObjectDetector(modelPathPrefix, inputDescriptors, context, 0);
+ List<List<ObjectDetectorOutput>> output = objDet.imageObjectDetect(img, 3);
+
+ printOutput(output, inputShape);
+ }
+
+
+ private static List<Context> getContext() {
+ List<Context> ctx = new ArrayList<>();
+ ctx.add(Context.cpu());
+
+ return ctx;
+ }
+
+ private static void printOutput(List<List<ObjectDetectorOutput>> output, Shape inputShape) {
+
+ StringBuilder outputStr = new StringBuilder();
+
+ int width = inputShape.get(3);
+ int height = inputShape.get(2);
+
+ for (List<ObjectDetectorOutput> ele : output) {
+ for (ObjectDetectorOutput i : ele) {
+ outputStr.append("Class: " + i.getClassName() + "\n");
+ outputStr.append("Probabilties: " + i.getProbability() + "\n");
+
+ List<Float> coord = Arrays.asList(i.getXMin() * width,
+ i.getXMax() * height, i.getYMin() * width, i.getYMax() * height);
+ StringBuilder sb = new StringBuilder();
+ for (float c: coord) {
+ sb.append(", ").append(c);
+ }
+ outputStr.append("Coord:" + sb.substring(2)+ "\n");
+ }
+ }
+ System.out.println(outputStr);
+
+ }
+}
+```
+
+7\. To compile and run this code, change directories to this project's root folder, then run the following:
+```bash
+mvn clean install dependency:copy-dependencies
+```
+
+The build generates a new jar file in the `target` folder called `javaMXNet-1.0-SNAPSHOT.jar`.
+
+To run the ObjectDetectionTutorial.java use the following command from the project's root folder.
+```bash
+java -cp target/javaMXNet-1.0-SNAPSHOT.jar:target/dependency/* mxnet.ObjectDetectionTutorial
+```
+
+You should see a similar output being generated for the dog image that we used:
+```bash
+Class: car
+Probabilties: 0.99847263
+Coord:312.21335, 72.02908, 456.01443, 150.66176
+Class: bicycle
+Probabilties: 0.9047381
+Coord:155.9581, 149.96365, 383.83694, 418.94516
+Class: dog
+Probabilties: 0.82268167
+Coord:83.82356, 179.14001, 206.63783, 476.78754
+```
+
+
+
+The results returned by the inference call translate into the regions in the image where the model detected objects.
+
+
+
+## Next Steps
+For more information about MXNet Java resources, see the following:
+
+* [Java Inference API](/api/java/infer.html)
+* [Java Inference Examples](https://github.com/apache/incubator-mxnet/tree/java-api/scala-package/examples/src/main/java/org/apache/mxnetexamples/infer/)
+* [MXNet Tutorials Index](/tutorials/index.html)
diff --git a/tests/tutorials/test_sanity_tutorials.py b/tests/tutorials/test_sanity_tutorials.py
index dc5fbf5d83a..9e5c38abc97 100644
--- a/tests/tutorials/test_sanity_tutorials.py
+++ b/tests/tutorials/test_sanity_tutorials.py
@@ -57,7 +57,8 @@
'vision/index.md',
'tensorrt/index.md',
'tensorrt/inference_with_trt.md',
- 'java/mxnet_java_on_intellij.md']
+ 'java/mxnet_java_on_intellij.md',
+ 'java/ssd_inference.md']
whitelist_set = set(whitelist)
def test_tutorial_downloadable():
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services