You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "ryankert01 (via GitHub)" <gi...@apache.org> on 2026/01/25 10:55:02 UTC

[PR] feat: Add make test & make pre-commit to enhance development flow [mahout]

ryankert01 opened a new pull request, #943:
URL: https://github.com/apache/mahout/pull/943

   Add tests for Rust and Python based on NVIDIA GPU detection.
   
   ### Purpose of PR
   before we have to manually make sure it passes all test by manually do so.
   
   Now, we can just
   ```
   make test && make pre-commit
   ```
   Then, we can go grab coffee.
   
   ### TODO:
   
   - [ ] document this
   
   ### Related Issues or PRs
   <!-- Add links to related issues or PRs. -->
   <!-- - Closes #123  -->
   <!-- - Related to #123   -->
   
   ### Changes Made
   <!-- Please mark one with an "x"   -->
   - [ ] Bug fix
   - [ ] New feature
   - [ ] Refactoring
   - [ ] Documentation
   - [ ] Test
   - [ ] CI/CD pipeline
   - [ ] Other
   
   ### Breaking Changes
   <!-- Does this PR introduce a breaking change? -->
   - [ ] Yes
   - [ ] No
   
   ### Checklist
   <!-- Please mark each item with an "x" when complete -->
   <!-- If not all items are complete, please open this as a **Draft PR**.
   Once all requirements are met, mark as ready for review. -->
   
   - [ ] Added or updated unit tests for all changes
   - [ ] Added or updated documentation for all changes
   - [ ] Successfully built and ran all unit tests or manual tests locally
   - [ ] PR title follows "MAHOUT-XXX: Brief Description" format (if related to an issue)
   - [ ] Code follows ASF guidelines
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat: Add make test & make pre-commit to enhance development flow [mahout]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming merged PR #943:
URL: https://github.com/apache/mahout/pull/943


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat: Add make test & make pre-commit to enhance development flow [mahout]

Posted by "viiccwen (via GitHub)" <gi...@apache.org>.
viiccwen commented on code in PR #943:
URL: https://github.com/apache/mahout/pull/943#discussion_r2725610231


##########
Makefile:
##########
@@ -0,0 +1,40 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Detect NVIDIA GPU
+HAS_NVIDIA := $(shell command -v nvidia-smi >/dev/null 2>&1 && echo yes || echo no)

Review Comment:
   Checking only for `nvidia-smi` in `PATH` can give false positives in CI/containers. Consider also verifying that nvidia-smi can actually enumerate GPUs (e.g. via `nvidia-smi -L`).



##########
Makefile:
##########
@@ -0,0 +1,40 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Detect NVIDIA GPU
+HAS_NVIDIA := $(shell command -v nvidia-smi >/dev/null 2>&1 && echo yes || echo no)
+
+test_rust:
+ifeq ($(HAS_NVIDIA),yes)
+	cd qdp && cargo test
+else
+	@echo "No NVIDIA GPU detected, skipping test_rust"
+endif
+
+test_python:
+	uv sync --group dev
+ifeq ($(HAS_NVIDIA),yes)
+	unset CONDA_PREFIX && uv run --active maturin develop --manifest-path qdp/qdp-python/Cargo.toml
+else
+	@echo "No NVIDIA GPU detected, skipping maturin develop"

Review Comment:
   Add prefix would be better.
   ```bash
   @echo "[SKIP] No NVIDIA GPU detected, skipping maturin develop"
   ```



##########
Makefile:
##########


Review Comment:
   may need to add `.PHONY`
   
   ```makefile
   .PHONY: test_rust test_python tests pre-commit
   ```



##########
Makefile:
##########
@@ -0,0 +1,40 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Detect NVIDIA GPU
+HAS_NVIDIA := $(shell command -v nvidia-smi >/dev/null 2>&1 && echo yes || echo no)
+
+test_rust:
+ifeq ($(HAS_NVIDIA),yes)
+	cd qdp && cargo test
+else
+	@echo "No NVIDIA GPU detected, skipping test_rust"

Review Comment:
   Add prefix would be better.
   ```bash
   @echo "[SKIP] No NVIDIA GPU detected, skipping test_rust"
   ```



##########
Makefile:
##########
@@ -0,0 +1,40 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Detect NVIDIA GPU
+HAS_NVIDIA := $(shell command -v nvidia-smi >/dev/null 2>&1 && echo yes || echo no)
+
+test_rust:
+ifeq ($(HAS_NVIDIA),yes)
+	cd qdp && cargo test
+else
+	@echo "No NVIDIA GPU detected, skipping test_rust"
+endif
+
+test_python:
+	uv sync --group dev

Review Comment:
   separate `uv sync --group dev` into single cmd `setup`, then call it in `test_python`, `pre-commit`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat: Add make test & make pre-commit to enhance development flow [mahout]

Posted by "viiccwen (via GitHub)" <gi...@apache.org>.
viiccwen commented on PR #943:
URL: https://github.com/apache/mahout/pull/943#issuecomment-3796819462

   like this:
   
   ```makefile
   .PHONY: test_rust test_python tests pre-commit setup-python
   
   # Detect NVIDIA GPU
   HAS_NVIDIA := $(shell command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi -L >/dev/null 2>&1 && echo yes || echo no)
   
   setup-python:
   	uv sync --group dev
   
   test_rust:
   ifeq ($(HAS_NVIDIA),yes)
   	cd qdp && cargo test
   else
   	@echo "[SKIP] No NVIDIA GPU detected, skipping test_rust"
   endif
   
   test_python: setup-python
   ifeq ($(HAS_NVIDIA),yes)
   	unset CONDA_PREFIX && uv run --active maturin develop --manifest-path qdp/qdp-python/Cargo.toml
   else
   	@echo "[SKIP] No NVIDIA GPU detected, skipping maturin develop"
   endif
   	uv run pytest
   
   tests: test_rust test_python
   
   pre-commit: setup-python
   	uv run pre-commit run --all-files
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org