You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/10 17:30:40 UTC

[GitHub] [beam] robertwb opened a new pull request, #21795: Merge ModelLoader and InferenceRunner into same class.

robertwb opened a new pull request, #21795:
URL: https://github.com/apache/beam/pull/21795

   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
    - [ ] Mention the appropriate issue in your description (for example: "addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment "fixes #<ISSUE NUMBER>" instead.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
   
   To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on a diff in pull request #21795: Merge ModelLoader and InferenceRunner into same class.

Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on code in PR #21795:
URL: https://github.com/apache/beam/pull/21795#discussion_r894763475


##########
sdks/python/apache_beam/ml/inference/base.py:
##########
@@ -64,8 +64,12 @@ def _to_microseconds(time_ns: int) -> int:
   return int(time_ns / _NANOSECOND_TO_MICROSECOND)
 
 
-class InferenceRunner(Generic[ExampleT, PredictionT, ModelT]):
-  """Implements running inferences for a framework."""
+class ModelHandler(Generic[ExampleT, PredictionT, ModelT]):
+  """Has the ability to load an ML model."""

Review Comment:
   Maybe update this docstring?



##########
sdks/python/apache_beam/ml/inference/base.py:
##########
@@ -64,8 +64,12 @@ def _to_microseconds(time_ns: int) -> int:
   return int(time_ns / _NANOSECOND_TO_MICROSECOND)
 
 
-class InferenceRunner(Generic[ExampleT, PredictionT, ModelT]):
-  """Implements running inferences for a framework."""
+class ModelHandler(Generic[ExampleT, PredictionT, ModelT]):
+  """Has the ability to load an ML model."""

Review Comment:
   Similarly for the implementations, consider if the the docstrings need to be merged.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] robertwb commented on pull request #21795: Merge ModelLoader and InferenceRunner into same class.

Posted by GitBox <gi...@apache.org>.
robertwb commented on PR #21795:
URL: https://github.com/apache/beam/pull/21795#issuecomment-1152585577

   R: @yeandy , @ryanthompson591 or @TheNeuralBit 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] robertwb commented on pull request #21795: Merge ModelLoader and InferenceRunner into same class.

Posted by GitBox <gi...@apache.org>.
robertwb commented on PR #21795:
URL: https://github.com/apache/beam/pull/21795#issuecomment-1152618689

   Run PythonLint PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] Abacn commented on pull request #21795: Merge ModelLoader and InferenceRunner into same class.

Posted by GitBox <gi...@apache.org>.
Abacn commented on PR #21795:
URL: https://github.com/apache/beam/pull/21795#issuecomment-1152667883

   Seems Unit tests are brocken:
   ```
   self = <apache_beam.ml.inference.sklearn_inference_test.SkLearnRunInferenceTest testMethod=test_infer_invalid_data_type>
       def test_infer_invalid_data_type(self):
         with self.assertRaises(ValueError):
           unexpected_input_type = [[1, 2, 3, 4], [5, 6, 7, 8]]
   >       inference_runner = SklearnModelLoader(model_uri=unused)
   E       NameError: name 'SklearnModelLoader' is not defined
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] robertwb commented on a diff in pull request #21795: Merge ModelLoader and InferenceRunner into same class.

Posted by GitBox <gi...@apache.org>.
robertwb commented on code in PR #21795:
URL: https://github.com/apache/beam/pull/21795#discussion_r894776553


##########
sdks/python/apache_beam/ml/inference/base.py:
##########
@@ -64,8 +64,12 @@ def _to_microseconds(time_ns: int) -> int:
   return int(time_ns / _NANOSECOND_TO_MICROSECOND)
 
 
-class InferenceRunner(Generic[ExampleT, PredictionT, ModelT]):
-  """Implements running inferences for a framework."""
+class ModelHandler(Generic[ExampleT, PredictionT, ModelT]):
+  """Has the ability to load an ML model."""

Review Comment:
   Yep. Done.



##########
sdks/python/apache_beam/ml/inference/sklearn_inference_test.py:
##########
@@ -71,7 +70,7 @@ def tearDown(self):
 
   def test_predict_output(self):
     fake_model = FakeModel()
-    inference_runner = SklearnInferenceRunner()
+    inference_runner = SklearnModelHandler('unused')

Review Comment:
   Done.



##########
sdks/python/apache_beam/ml/inference/base.py:
##########
@@ -64,8 +64,12 @@ def _to_microseconds(time_ns: int) -> int:
   return int(time_ns / _NANOSECOND_TO_MICROSECOND)
 
 
-class InferenceRunner(Generic[ExampleT, PredictionT, ModelT]):
-  """Implements running inferences for a framework."""
+class ModelHandler(Generic[ExampleT, PredictionT, ModelT]):
+  """Has the ability to load an ML model."""

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] yeandy commented on a diff in pull request #21795: Merge ModelLoader and InferenceRunner into same class.

Posted by GitBox <gi...@apache.org>.
yeandy commented on code in PR #21795:
URL: https://github.com/apache/beam/pull/21795#discussion_r894767368


##########
sdks/python/apache_beam/ml/inference/sklearn_inference.py:
##########
@@ -41,36 +40,20 @@ class ModelFileType(enum.Enum):
   JOBLIB = 2
 
 
-class SklearnInferenceRunner(InferenceRunner[numpy.ndarray,
-                                             PredictionResult,
-                                             BaseEstimator]):
-  def run_inference(
-      self, batch: List[numpy.ndarray], model: BaseEstimator,
-      **kwargs) -> Iterable[PredictionResult]:
-    # vectorize data for better performance
-    vectorized_batch = numpy.stack(batch, axis=0)
-    predictions = model.predict(vectorized_batch)
-    return [PredictionResult(x, y) for x, y in zip(batch, predictions)]
-
-  def get_num_bytes(self, batch: List[numpy.ndarray]) -> int:
-    """Returns the number of bytes of data for a batch."""
-    return sum(sys.getsizeof(element) for element in batch)
-
-
-class SklearnModelLoader(ModelLoader[numpy.ndarray,
-                                     PredictionResult,
-                                     BaseEstimator]):
-  """ Implementation of the ModelLoader interface for scikit learn.
+class SklearnModelHandler(ModelHandler[numpy.ndarray,
+                                       PredictionResult,
+                                       BaseEstimator]):
+  """ Implementation of the ModelHandler interface for scikit learn.

Review Comment:
   nit
   ```suggestion
     """ Implementation of the ModelHandler interface for scikit-learn.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] ryanthompson591 commented on a diff in pull request #21795: Merge ModelLoader and InferenceRunner into same class.

Posted by GitBox <gi...@apache.org>.
ryanthompson591 commented on code in PR #21795:
URL: https://github.com/apache/beam/pull/21795#discussion_r894767654


##########
sdks/python/apache_beam/ml/inference/sklearn_inference_test.py:
##########
@@ -71,7 +70,7 @@ def tearDown(self):
 
   def test_predict_output(self):
     fake_model = FakeModel()
-    inference_runner = SklearnInferenceRunner()
+    inference_runner = SklearnModelHandler('unused')

Review Comment:
   SklearnModelHandler(uri='unused')?



##########
sdks/python/apache_beam/ml/inference/base.py:
##########
@@ -64,8 +64,12 @@ def _to_microseconds(time_ns: int) -> int:
   return int(time_ns / _NANOSECOND_TO_MICROSECOND)
 
 
-class InferenceRunner(Generic[ExampleT, PredictionT, ModelT]):
-  """Implements running inferences for a framework."""
+class ModelHandler(Generic[ExampleT, PredictionT, ModelT]):
+  """Has the ability to load an ML model."""

Review Comment:
   Has the abiltity to load ML models and run inferences on that model.



##########
sdks/python/apache_beam/ml/inference/sklearn_inference.py:
##########
@@ -26,8 +26,7 @@
 

Review Comment:
   Might have to merge this file since I just merged a change. Should be straight forward.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] robertwb merged pull request #21795: Merge ModelLoader and InferenceRunner into same class.

Posted by GitBox <gi...@apache.org>.
robertwb merged PR #21795:
URL: https://github.com/apache/beam/pull/21795


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] commented on pull request #21795: Merge ModelLoader and InferenceRunner into same class.

Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #21795:
URL: https://github.com/apache/beam/pull/21795#issuecomment-1152613045

   # [Codecov](https://codecov.io/gh/apache/beam/pull/21795?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#21795](https://codecov.io/gh/apache/beam/pull/21795?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3eaccaf) into [master](https://codecov.io/gh/apache/beam/commit/0de98210f4531fbfd88265bc02052b27bd299602?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0de9821) will **increase** coverage by `0.03%`.
   > The diff coverage is `51.42%`.
   
   > :exclamation: Current head 3eaccaf differs from pull request most recent head 963117b. Consider uploading reports for the commit 963117b to get more accurate results
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #21795      +/-   ##
   ==========================================
   + Coverage   74.01%   74.05%   +0.03%     
   ==========================================
     Files         698      698              
     Lines       92224    92330     +106     
   ==========================================
   + Hits        68263    68377     +114     
   + Misses      22710    22702       -8     
     Partials     1251     1251              
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | python | `83.64% <51.42%> (+0.03%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/21795?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...thon/apache\_beam/ml/inference/pytorch\_inference.py](https://codecov.io/gh/apache/beam/pull/21795/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvaW5mZXJlbmNlL3B5dG9yY2hfaW5mZXJlbmNlLnB5) | `0.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/ml/inference/base.py](https://codecov.io/gh/apache/beam/pull/21795/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvaW5mZXJlbmNlL2Jhc2UucHk=) | `94.30% <88.88%> (+0.60%)` | :arrow_up: |
   | [sdks/python/apache\_beam/ml/inference/api.py](https://codecov.io/gh/apache/beam/pull/21795/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvaW5mZXJlbmNlL2FwaS5weQ==) | `90.00% <100.00%> (ø)` | |
   | [...thon/apache\_beam/ml/inference/sklearn\_inference.py](https://codecov.io/gh/apache/beam/pull/21795/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvaW5mZXJlbmNlL3NrbGVhcm5faW5mZXJlbmNlLnB5) | `91.66% <100.00%> (-0.84%)` | :arrow_down: |
   | [...eam/transforms/py\_dataflow\_distribution\_counter.py](https://codecov.io/gh/apache/beam/pull/21795/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9weV9kYXRhZmxvd19kaXN0cmlidXRpb25fY291bnRlci5weQ==) | `91.42% <0.00%> (-4.87%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/counters.py](https://codecov.io/gh/apache/beam/pull/21795/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvY291bnRlcnMucHk=) | `85.39% <0.00%> (-1.36%)` | :arrow_down: |
   | [...eam/runners/portability/fn\_api\_runner/execution.py](https://codecov.io/gh/apache/beam/pull/21795/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL2V4ZWN1dGlvbi5weQ==) | `92.44% <0.00%> (-0.65%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/21795/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.42% <0.00%> (-0.25%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/21795/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.94% <0.00%> (-0.16%)` | :arrow_down: |
   | [sdks/python/apache\_beam/transforms/window.py](https://codecov.io/gh/apache/beam/pull/21795/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy93aW5kb3cucHk=) | `87.20% <0.00%> (+0.15%)` | :arrow_up: |
   | ... and [10 more](https://codecov.io/gh/apache/beam/pull/21795/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/21795?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/21795?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [0de9821...963117b](https://codecov.io/gh/apache/beam/pull/21795?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org