You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/07/12 15:57:12 UTC
[GitHub] [beam] yeandy opened a new issue, #22240: [Bug]: Pytorch RunInference PredictionResult is a Dict
yeandy opened a new issue, #22240:
URL: https://github.com/apache/beam/issues/22240
### What happened?
Some model inferences produce Dict[str, Tensor], but RunInference currently expects List[Any]. A workaround is to create a wrapper.
The PredictionResult challenge is a known quirk due to the current design/implementation of RunInference. We have [a sample](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/pytorch_language_modeling.py#L49) that does something similar by creating a wrapper class. We should support both tensors and dictionary of tensors. Currently, it only works for the former, and that is baked into the logic. We should figure out how to natively support results that are a dict type.
### Issue Priority
Priority: 2
### Issue Component
Component: sdk-py-core
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm closed issue #22240: [Bug]: Pytorch RunInference PredictionResult is a Dict
Posted by GitBox <gi...@apache.org>.
damccorm closed issue #22240: [Bug]: Pytorch RunInference PredictionResult is a Dict
URL: https://github.com/apache/beam/issues/22240
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on issue #22240: [Bug]: Pytorch RunInference PredictionResult is a Dict
Posted by GitBox <gi...@apache.org>.
damccorm commented on issue #22240:
URL: https://github.com/apache/beam/issues/22240#issuecomment-1234606248
#22979 got created as an accidental duplicate. I'm closing that one, but copying over the description since I think it has some extra additional context:
Originally documented [here](https://beam.apache.org/documentation/sdks/python-machine-learning/#incorrect-inferences-in-the-predictionresult-object) in our RunInference Troubleshooting section:
For some models, the PredictionResults output might not include the correct predictions in the inferences field. This issue occurs when you use a model whose inferences return a dictionary that maps keys to predictions and other metadata. An example return type is `Dict[str, Tensor]`.
The RunInference API currently expects outputs to be an `Iterable[Any]`. Example return types are `Iterable[Tensor]` or `Iterable[Dict[str, Tensor]]`. When RunInference zips the inputs with the predictions, the predictions iterate over the dictionary keys instead of the batch elements. The result is that the key name is preserved but the prediction tensors are discarded.
To work with the current RunInference implementation, you can create a wrapper class that overrides the model(input) call. In PyTorch, for example, your wrapper would override the forward() function and return an output with the appropriate format of `List[Dict[str, torch.Tensor]]`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on issue #22240: [Bug]: Pytorch RunInference PredictionResult is a Dict
Posted by GitBox <gi...@apache.org>.
damccorm commented on issue #22240:
URL: https://github.com/apache/beam/issues/22240#issuecomment-1242398377
#23152 will close this once its merged (after the 2.43 release)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] yeandy commented on issue #22240: [Bug]: Pytorch RunInference PredictionResult is a Dict
Posted by GitBox <gi...@apache.org>.
yeandy commented on issue #22240:
URL: https://github.com/apache/beam/issues/22240#issuecomment-1181955865
.take-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] yeandy commented on issue #22240: [Bug]: Pytorch RunInference PredictionResult is a Dict
Posted by GitBox <gi...@apache.org>.
yeandy commented on issue #22240:
URL: https://github.com/apache/beam/issues/22240#issuecomment-1185757084
Subtask of https://github.com/apache/beam/issues/22117
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm closed issue #22240: [Bug]: Pytorch RunInference PredictionResult is a Dict
Posted by GitBox <gi...@apache.org>.
damccorm closed issue #22240: [Bug]: Pytorch RunInference PredictionResult is a Dict
URL: https://github.com/apache/beam/issues/22240
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] yeandy commented on issue #22240: [Bug]: Pytorch RunInference PredictionResult is a Dict
Posted by GitBox <gi...@apache.org>.
yeandy commented on issue #22240:
URL: https://github.com/apache/beam/issues/22240#issuecomment-1241908772
Thanks @damccorm! We should also remove (or at least clarify) the section on this issue from our [Troubleshooting guide](https://github.com/apache/beam/blob/master/website/www/site/content/en/documentation/sdks/python-machine-learning.md#incorrect-inferences-in-the-predictionresult-object). I had intended to make this comment in the PR yesterday, but forgot, and just remembered. 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on issue #22240: [Bug]: Pytorch RunInference PredictionResult is a Dict
Posted by GitBox <gi...@apache.org>.
damccorm commented on issue #22240:
URL: https://github.com/apache/beam/issues/22240#issuecomment-1241957948
That's a good callout - I'll reopen here. Since this fix won't be released for ~2 months (it missed the 2.42.0 cut), we should probably hold off on doing that until then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org