You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "damccorm (via GitHub)" <gi...@apache.org> on 2025/10/07 14:10:18 UTC
[PR] Add vllm extra [beam]
damccorm opened a new pull request, #36420:
URL: https://github.com/apache/beam/pull/36420
This will help us define which version of vLLM we support, which can be useful for gpu containers.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Use vLLM extra to generate GPU requirements files [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #36420:
URL: https://github.com/apache/beam/pull/36420#discussion_r2478830714
##########
sdks/python/container/run_generate_requirements.sh:
##########
@@ -76,14 +76,24 @@ python"${PY_VERSION}" -m venv "$ENV_PATH"
source "$ENV_PATH"/bin/activate
pip install --upgrade pip setuptools wheel
+# For non-vllm (non-gpu) requirement files, force downloading torch from CPU wheels
+INDEX_URL_OPTION="--extra-index-url https://download.pytorch.org/whl/cpu"
+if [[ $EXTRAS == *"vllm"* ]]; then
+ # Explicitly install torch to avoid https://github.com/facebookresearch/xformers/issues/740
+ # This should be overwritten later since the vllm extra is installed alongside torch
Review Comment:
Thanks - I agree this was unclear. I was trying to say that the torch version we install here doesn't matter since torch will get installed anyways later as part of the vllm install (and that will determine the actual version which is installed). Hopefully the new comment is clearer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add vllm extra [beam]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #36420:
URL: https://github.com/apache/beam/pull/36420#issuecomment-3453416590
## [Codecov](https://app.codecov.io/gh/apache/beam/pull/36420?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 36.24%. Comparing base ([`80f4564`](https://app.codecov.io/gh/apache/beam/commit/80f4564fbdcfda0cdedf7ffd8570e04f9d7a5e88?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`fb80415`](https://app.codecov.io/gh/apache/beam/commit/fb8041506af0e6ff1d5583ba1ab76ad449bb1e4a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #36420 +/- ##
=========================================
Coverage 36.24% 36.24%
Complexity 1666 1666
=========================================
Files 1060 1060
Lines 165704 165704
Branches 1195 1195
=========================================
Hits 60063 60063
Misses 103465 103465
Partials 2176 2176
```
| [Flag](https://app.codecov.io/gh/apache/beam/pull/36420/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [go](https://app.codecov.io/gh/apache/beam/pull/36420/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `28.52% <ø> (ø)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/beam/pull/36420?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add vllm extra [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #36420:
URL: https://github.com/apache/beam/pull/36420#issuecomment-3381701545
This PR is unrelated to that flakiness, so I wouldn't expect it to make a difference immediately.
In the logs, I now see a bunch of failures like:
```
RuntimeError: self and mat2 must have the same dtype, but got Float and Half
```
I'm not 100% sure what would cause that error, but it may be a consequence of trying to run Gemma on a T4 - could you try updating to an L4 to see if that solves the problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Use vLLM extra to generate GPU requirements files [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #36420:
URL: https://github.com/apache/beam/pull/36420#issuecomment-3457864167
assign set of reviewers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Use vLLM extra to generate GPU requirements files [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on code in PR #36420:
URL: https://github.com/apache/beam/pull/36420#discussion_r2470918186
##########
sdks/python/container/run_generate_requirements.sh:
##########
@@ -76,14 +76,24 @@ python"${PY_VERSION}" -m venv "$ENV_PATH"
source "$ENV_PATH"/bin/activate
pip install --upgrade pip setuptools wheel
+# For non-vllm (non-gpu) requirement files, force downloading torch from CPU wheels
+INDEX_URL_OPTION="--extra-index-url https://download.pytorch.org/whl/cpu"
+if [[ $EXTRAS == *"vllm"* ]]; then
+ # Explicitly install torch to avoid https://github.com/facebookresearch/xformers/issues/740
+ # This should be overwritten later since the vllm extra is installed alongside torch
Review Comment:
> This should be overwritten later
I don't quite follow the comment. Could you please clarify what is being overwritten?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Use vLLM extra to generate GPU requirements files [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm merged PR #36420:
URL: https://github.com/apache/beam/pull/36420
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Use vLLM extra to generate GPU requirements files [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36420:
URL: https://github.com/apache/beam/pull/36420#issuecomment-3457868922
Assigning reviewers:
R: @tvalentyn for label python.
R: @Abacn for label build.
Note: If you would like to opt out of this review, comment `assign to next reviewer`.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add vllm extra [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on PR #36420:
URL: https://github.com/apache/beam/pull/36420#issuecomment-3384189795
> This PR is unrelated to that flakiness, so I wouldn't expect it to make a difference immediately.
>
> In the logs, I now see a bunch of failures like:
>
> ```
> RuntimeError: self and mat2 must have the same dtype, but got Float and Half
> ```
>
> I'm not 100% sure what would cause that error, but it may be a consequence of trying to run Gemma on a T4 - could you try updating to an L4 to see if that solves the problem?
It helped!
PR: https://github.com/apache/beam/pull/36451
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add vllm extra [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on PR #36420:
URL: https://github.com/apache/beam/pull/36420#issuecomment-3381607724
@damccorm I used your changes plus fixed OutOfMemory issue, but there is still an error with starting VLLM server.
Could you please help to understand what happened?
https://github.com/apache/beam/actions/runs/18342904797/job/52242432601
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Use vLLM extra to generate GPU requirements files [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #36420:
URL: https://github.com/apache/beam/pull/36420#discussion_r2478823627
##########
sdks/python/container/run_generate_requirements.sh:
##########
@@ -76,14 +76,24 @@ python"${PY_VERSION}" -m venv "$ENV_PATH"
source "$ENV_PATH"/bin/activate
pip install --upgrade pip setuptools wheel
+# For non-vllm (non-gpu) requirement files, force downloading torch from CPU wheels
Review Comment:
Yes - specifically https://github.com/apache/beam/blob/fe07fe767098663c032f84df74b4b4f7723bf607/sdks/python/container/ml/py310/ml_image_requirements.txt#L217
has a cpu postfix, and some gpu requirements (which are not ASF-license compliant) are omitted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Use vLLM extra to generate GPU requirements files [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36420:
URL: https://github.com/apache/beam/pull/36420#issuecomment-3457829151
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Use vLLM extra to generate GPU requirements files [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on code in PR #36420:
URL: https://github.com/apache/beam/pull/36420#discussion_r2470921491
##########
sdks/python/container/run_generate_requirements.sh:
##########
@@ -76,14 +76,24 @@ python"${PY_VERSION}" -m venv "$ENV_PATH"
source "$ENV_PATH"/bin/activate
pip install --upgrade pip setuptools wheel
+# For non-vllm (non-gpu) requirement files, force downloading torch from CPU wheels
Review Comment:
Does the outcome of the generated requirements file depend on which wheel is installed (cpu vs non-cpu)? If so, is it because the dependencies of torch change?
##########
sdks/python/container/run_generate_requirements.sh:
##########
@@ -76,14 +76,24 @@ python"${PY_VERSION}" -m venv "$ENV_PATH"
source "$ENV_PATH"/bin/activate
pip install --upgrade pip setuptools wheel
+# For non-vllm (non-gpu) requirement files, force downloading torch from CPU wheels
+INDEX_URL_OPTION="--extra-index-url https://download.pytorch.org/whl/cpu"
+if [[ $EXTRAS == *"vllm"* ]]; then
+ # Explicitly install torch to avoid https://github.com/facebookresearch/xformers/issues/740
+ # This should be overwritten later since the vllm extra is installed alongside torch
Review Comment:
> This should be overwritten later
I don't quite follow the comment. Could you please clarify what is being overwritten?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Use vLLM extra to generate GPU requirements files [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on PR #36420:
URL: https://github.com/apache/beam/pull/36420#issuecomment-3473960076
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org