You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/08/25 20:02:11 UTC

[GitHub] [beam] yeandy opened a new issue, #22895: [Bug]: RunInference GPU error: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

yeandy opened a new issue, #22895:
URL: https://github.com/apache/beam/issues/22895

   ### What happened?
   
   When using multiple processes (when we don’t use `no_use_multiple_sdk_containers`) in DataFlow jobs, this error has been observed
   
   `Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!`
   
   The current solution is to add the `experiment=no_use_multiple_sdk_containers` flag.
   
   We should look into how much benefit we get with multiple processes, as opposed to maxing out GPU utilization in one worker. 
   
   ### Issue Priority
   
   Priority: 2
   
   ### Issue Component
   
   Component: sdk-py-core


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] yeandy commented on issue #22895: [Bug]: RunInference GPU error: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

Posted by GitBox <gi...@apache.org>.
yeandy commented on issue #22895:
URL: https://github.com/apache/beam/issues/22895#issuecomment-1228759226

   Not at the moment. You can claim it with the `take-issue` command. See [here](https://github.com/apache/beam/blob/master/.github/ACTIONS.md) for instructions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] Saifuddin53 commented on issue #22895: [Bug]: RunInference GPU error: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

Posted by GitBox <gi...@apache.org>.
Saifuddin53 commented on issue #22895:
URL: https://github.com/apache/beam/issues/22895#issuecomment-1228360338

   is anyone working on this issue. If no you can assign me
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] github-actions[bot] closed issue #22895: [Bug]: RunInference GPU error: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #22895: [Bug]: RunInference GPU error: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!
URL: https://github.com/apache/beam/issues/22895


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] yeandy commented on issue #22895: [Bug]: RunInference GPU error: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

Posted by GitBox <gi...@apache.org>.
yeandy commented on issue #22895:
URL: https://github.com/apache/beam/issues/22895#issuecomment-1227703816

   .add-labels run-inference


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] Saifuddin53 commented on issue #22895: [Bug]: RunInference GPU error: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

Posted by GitBox <gi...@apache.org>.
Saifuddin53 commented on issue #22895:
URL: https://github.com/apache/beam/issues/22895#issuecomment-1231223509

   .close-issue 
   not_planned
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] tvalentyn commented on issue #22895: [Bug]: RunInference GPU error: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #22895:
URL: https://github.com/apache/beam/issues/22895#issuecomment-1232043781

   Likely was closed by mistake.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] yeandy commented on issue #22895: [Bug]: RunInference GPU error: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

Posted by GitBox <gi...@apache.org>.
yeandy commented on issue #22895:
URL: https://github.com/apache/beam/issues/22895#issuecomment-1231545462

   Why did you close this issue?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] Saifuddin53 commented on issue #22895: [Bug]: RunInference GPU error: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

Posted by GitBox <gi...@apache.org>.
Saifuddin53 commented on issue #22895:
URL: https://github.com/apache/beam/issues/22895#issuecomment-1229751629

   .take-issue
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org