You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ru...@apache.org on 2023/08/07 02:57:22 UTC
[spark] branch master updated: [SPARK-44264][PYTHON][ML][TESTS][FOLLOWUP] Adding Deepspeed To The Test Dockerfile
This is an automated email from the ASF dual-hosted git repository.
ruifengz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 7515061ec23 [SPARK-44264][PYTHON][ML][TESTS][FOLLOWUP] Adding Deepspeed To The Test Dockerfile
7515061ec23 is described below
commit 7515061ec237b0393e2fcc064a309fae29502dff
Author: Mathew Jacob <ma...@databricks.com>
AuthorDate: Mon Aug 7 10:57:02 2023 +0800
[SPARK-44264][PYTHON][ML][TESTS][FOLLOWUP] Adding Deepspeed To The Test Dockerfile
### What changes were proposed in this pull request?
Added tests to the Dockerfile for tests in OSS Spark CI.
### Why are the changes needed?
They'll skip the deepspeed tests otherwise.
### Does this PR introduce _any_ user-facing change?
Nope, testing infra.
### How was this patch tested?
Running the tests on machine.
Closes #42347 from mathewjacob1002/testing_infra.
Authored-by: Mathew Jacob <ma...@databricks.com>
Signed-off-by: Ruifeng Zheng <ru...@apache.org>
---
dev/infra/Dockerfile | 2 ++
1 file changed, 2 insertions(+)
diff --git a/dev/infra/Dockerfile b/dev/infra/Dockerfile
index 9d7b29e25b4..b69e682f239 100644
--- a/dev/infra/Dockerfile
+++ b/dev/infra/Dockerfile
@@ -73,3 +73,5 @@ RUN python3.9 -m pip install grpcio protobuf googleapis-common-protos grpcio-sta
# Add torch as a testing dependency for TorchDistributor
RUN python3.9 -m pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
RUN python3.9 -m pip install torcheval
+# Add Deepspeed as a testing dependency for DeepspeedTorchDistributor
+RUN python3.9 -m pip install deepspeed
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org