You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/14 23:44:59 UTC
[GitHub] [beam] yeandy commented on a diff in pull request #21871: Modify README for 3 pytorch examples
yeandy commented on code in PR #21871:
URL: https://github.com/apache/beam/pull/21871#discussion_r897405701
##########
sdks/python/apache_beam/examples/inference/README.md:
##########
@@ -48,77 +58,181 @@ i.e. "See the
for details."
-->
-### Datasets and models for RunInference
-
-The RunInference example pipelines read example data from `gs://apache-beam-ml/`.
-You can view the data
-[here](https://console.cloud.google.com/storage/browser/apache-beam-ml). You can
-also list the example data using the
-[gsutil tool](https://cloud.google.com/storage/docs/gsutil#gettingstarted).
-
-```
-gsutil ls gs://apache-beam-ml
-```
-
---
-## Image classification with ImageNet dataset
+## Image classification
[`pytorch_image_classification.py`](./pytorch_image_classification.py) contains
-an implementation for a RunInference pipeline that peforms image classification
-on the [ImageNet dataset](https://www.image-net.org/) using the MobileNetV2
+an implementation for a RunInference pipeline thatpeforms image classification using the mobilenet_v2
architecture.
The pipeline reads the images, performs basic preprocessing, passes them to the
PyTorch implementation of RunInference, and then writes the predictions
-to a text file in GCS.
+to a text file.
### Dataset and model for image classification
-The image classification pipeline uses the following data:
-<!---
-TODO: Add once benchmark test is released
-- `gs://apache-beam-ml/testing/inputs/imagenet_validation_inputs.txt`:
- text file containing the GCS paths of the images of all 5000 imagenet validation data
- - gs://apache-beam-ml/datasets/imagenet/raw-data/validation/ILSVRC2012_val_00000001.JPEG
- - ...
- - gs://apache-beam-ml/datasets/imagenet/raw-data/validation/ILSVRC2012_val_00050000.JPEG
--->
-- `gs://apache-beam-ml/testing/inputs/it_imagenet_validation_inputs.txt`:
- Text file containing the GCS paths of the images of a subset of ImageNet
- validation data. The following example command shows how to view contents of
- the file:
-
- ```
- $ gsutil cat gs://apache-beam-ml/testing/inputs/it_imagenet_validation_inputs.txt
- gs://apache-beam-ml/datasets/imagenet/raw-data/validation/ILSVRC2012_val_00000001.JPEG
- ...
- gs://apache-beam-ml/datasets/imagenet/raw-data/validation/ILSVRC2012_val_00000015.JPEG
- ```
-
-- `gs://apache-beam-ml/datasets/imagenet/raw-data/validation/ILSVRC2012_val_*.JPEG`:
- JPEG images for the entire validation dataset.
-
-- `gs://apache-beam-ml/models/torchvision.models.mobilenet_v2.pth`: Path to
- the location of the saved `state_dict` of the pretrained `mobilenet_v2` model
- from the `torchvision.models` subdirectory.
+You will need to create or download images, and place them into this directory called `IMAGES_DIR`. One popular dataset is from [ImageNet](https://www.image-net.org/). Please follow their instructions to download the images.
Review Comment:
Fixed
##########
sdks/python/apache_beam/examples/inference/README.md:
##########
@@ -48,77 +58,181 @@ i.e. "See the
for details."
-->
-### Datasets and models for RunInference
-
-The RunInference example pipelines read example data from `gs://apache-beam-ml/`.
-You can view the data
-[here](https://console.cloud.google.com/storage/browser/apache-beam-ml). You can
-also list the example data using the
-[gsutil tool](https://cloud.google.com/storage/docs/gsutil#gettingstarted).
-
-```
-gsutil ls gs://apache-beam-ml
-```
-
---
-## Image classification with ImageNet dataset
+## Image classification
[`pytorch_image_classification.py`](./pytorch_image_classification.py) contains
-an implementation for a RunInference pipeline that peforms image classification
-on the [ImageNet dataset](https://www.image-net.org/) using the MobileNetV2
+an implementation for a RunInference pipeline thatpeforms image classification using the mobilenet_v2
Review Comment:
Fixed
##########
sdks/python/apache_beam/examples/inference/README.md:
##########
@@ -48,77 +58,181 @@ i.e. "See the
for details."
-->
-### Datasets and models for RunInference
-
-The RunInference example pipelines read example data from `gs://apache-beam-ml/`.
-You can view the data
-[here](https://console.cloud.google.com/storage/browser/apache-beam-ml). You can
-also list the example data using the
-[gsutil tool](https://cloud.google.com/storage/docs/gsutil#gettingstarted).
-
-```
-gsutil ls gs://apache-beam-ml
-```
-
---
-## Image classification with ImageNet dataset
+## Image classification
[`pytorch_image_classification.py`](./pytorch_image_classification.py) contains
-an implementation for a RunInference pipeline that peforms image classification
-on the [ImageNet dataset](https://www.image-net.org/) using the MobileNetV2
+an implementation for a RunInference pipeline thatpeforms image classification using the mobilenet_v2
architecture.
The pipeline reads the images, performs basic preprocessing, passes them to the
PyTorch implementation of RunInference, and then writes the predictions
-to a text file in GCS.
+to a text file.
### Dataset and model for image classification
-The image classification pipeline uses the following data:
-<!---
-TODO: Add once benchmark test is released
-- `gs://apache-beam-ml/testing/inputs/imagenet_validation_inputs.txt`:
- text file containing the GCS paths of the images of all 5000 imagenet validation data
- - gs://apache-beam-ml/datasets/imagenet/raw-data/validation/ILSVRC2012_val_00000001.JPEG
- - ...
- - gs://apache-beam-ml/datasets/imagenet/raw-data/validation/ILSVRC2012_val_00050000.JPEG
--->
-- `gs://apache-beam-ml/testing/inputs/it_imagenet_validation_inputs.txt`:
- Text file containing the GCS paths of the images of a subset of ImageNet
- validation data. The following example command shows how to view contents of
- the file:
-
- ```
- $ gsutil cat gs://apache-beam-ml/testing/inputs/it_imagenet_validation_inputs.txt
- gs://apache-beam-ml/datasets/imagenet/raw-data/validation/ILSVRC2012_val_00000001.JPEG
- ...
- gs://apache-beam-ml/datasets/imagenet/raw-data/validation/ILSVRC2012_val_00000015.JPEG
- ```
-
-- `gs://apache-beam-ml/datasets/imagenet/raw-data/validation/ILSVRC2012_val_*.JPEG`:
- JPEG images for the entire validation dataset.
-
-- `gs://apache-beam-ml/models/torchvision.models.mobilenet_v2.pth`: Path to
- the location of the saved `state_dict` of the pretrained `mobilenet_v2` model
- from the `torchvision.models` subdirectory.
+You will need to create or download images, and place them into this directory called `IMAGES_DIR`. One popular dataset is from [ImageNet](https://www.image-net.org/). Please follow their instructions to download the images.
+- **Required**: A path to a file called `IMAGE_FILE_NAMES` that contains the absolute paths of each of the images in `IMAGES_DIR` on which you want to run image segmentation. For example:
+```
+/absolute/path/to/image1.jpg
+/absolute/path/to/image2.jpg
+```
+- **Required**: A path to a file called `MODEL_STATE_DICT` that contains the saved
+parameters of the maskrcnn_resnet50_fpn model. You will need to download the [mobilenet_v2](https://pytorch.org/vision/stable/_modules/torchvision/models/mobilenetv2.html)
+model from Pytorch's repository of pretrained models. Make sure you have installed `torchvision` too.
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org