You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "kevinjqliu (via GitHub)" <gi...@apache.org> on 2026/01/29 06:09:16 UTC
[PR] infra: allow rerun `make install` without prompt [iceberg-python]
kevinjqliu opened a new pull request, #2979:
URL: https://github.com/apache/iceberg-python/pull/2979
<!--
Thanks for opening a pull request!
-->
<!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. -->
<!-- Closes #${GITHUB_ISSUE_ID} -->
# Rationale for this change
I noticed running `make install` when a `.venv` already existed would show an interactive prompt:
```
➜ make install
uv is already installed.
uv venv
Using CPython 3.12.11 interpreter at: /Users/kevinliu/.pyenv/versions/3.12.11/bin/python3
Creating virtual environment at: .venv
? A virtual environment already exists at `.venv`. Do you want to replace it? [y/n] › yes
hint: Use the `--clear` flag or set `UV_VENV_CLEAR=1` to skip this prompt
```
We dont need this prompt.
This PR only setup a new venv if it does not exist. And also sync dep only when necessary.
This should make the entire `make install` process a lot faster and easier to work with
## Are these changes tested?
Yes running `make install` repeatedly
## Are there any user-facing changes?
<!-- In the case of user-facing changes, please add the changelog label. -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] infra: allow rerun `make install` without prompt [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #2979:
URL: https://github.com/apache/iceberg-python/pull/2979#discussion_r2742351744
##########
Makefile:
##########
@@ -66,11 +66,15 @@ install-uv: ## Ensure uv is installed
echo "uv is already installed."; \
fi
-setup-venv: ## Create virtual environment
- uv venv $(PYTHON_ARG)
+setup-venv: ## Create virtual environment (if not exists)
+ @if [ ! -d ".venv" ]; then \
+ uv venv $(PYTHON_ARG); \
+ else \
+ echo "Virtual environment already exists at .venv"; \
+ fi
install-dependencies: setup-venv ## Install all dependencies including extras
- uv sync $(PYTHON_ARG) --all-extras --reinstall
+ uv sync $(PYTHON_ARG) --all-extras
Review Comment:
yes i totally forgot, this was https://github.com/apache/iceberg-python/pull/2869
i added the conditional logic and a comment for so we dont forget again 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] infra: allow rerun `make install` without prompt [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #2979:
URL: https://github.com/apache/iceberg-python/pull/2979#discussion_r2742345133
##########
Makefile:
##########
@@ -66,11 +66,15 @@ install-uv: ## Ensure uv is installed
echo "uv is already installed."; \
fi
-setup-venv: ## Create virtual environment
- uv venv $(PYTHON_ARG)
+setup-venv: ## Create virtual environment (if not exists)
+ @if [ ! -d ".venv" ]; then \
Review Comment:
updated the pr to get rid of `setup-venv` entirely and let uv handle the virtualenv
this will allow users to customize `UV_PROJECT_ENVIRONMENT` if desired
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] infra: allow rerun `make install` without prompt [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #2979:
URL: https://github.com/apache/iceberg-python/pull/2979#discussion_r2742600457
##########
Makefile:
##########
@@ -66,17 +66,15 @@ install-uv: ## Ensure uv is installed
echo "uv is already installed."; \
fi
-setup-venv: ## Create virtual environment
- uv venv $(PYTHON_ARG)
-
-install-dependencies: setup-venv ## Install all dependencies including extras
- uv sync $(PYTHON_ARG) --all-extras --reinstall
-
-install-hooks: ## Install pre-commit hooks
+install: install-uv ## Install uv, dependencies, and pre-commit hooks
+ uv sync $(PYTHON_ARG) --all-extras
+ @# Reinstall pyiceberg if Cython extensions (.so) are missing after `make clean` (see #2869)
Review Comment:
`@#` is the syntax for comment. Using just `#` will have the sentence echo out when running `make install`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] infra: allow rerun `make install` without prompt [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu merged PR #2979:
URL: https://github.com/apache/iceberg-python/pull/2979
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] infra: allow rerun `make install` without prompt [iceberg-python]
Posted by "rambleraptor (via GitHub)" <gi...@apache.org>.
rambleraptor commented on code in PR #2979:
URL: https://github.com/apache/iceberg-python/pull/2979#discussion_r2740151684
##########
Makefile:
##########
@@ -66,11 +66,15 @@ install-uv: ## Ensure uv is installed
echo "uv is already installed."; \
fi
-setup-venv: ## Create virtual environment
- uv venv $(PYTHON_ARG)
+setup-venv: ## Create virtual environment (if not exists)
+ @if [ ! -d ".venv" ]; then \
Review Comment:
It is possible for you to set a different venv path using `UV_PROJECT_ENVIRONMENT`.
I don't think we should attempt to change this script to handle that case. If users want to do non-standard things, they probably aren't running this command in the first place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] infra: allow rerun `make install` without prompt [iceberg-python]
Posted by "geruh (via GitHub)" <gi...@apache.org>.
geruh commented on code in PR #2979:
URL: https://github.com/apache/iceberg-python/pull/2979#discussion_r2744215289
##########
Makefile:
##########
@@ -66,17 +66,15 @@ install-uv: ## Ensure uv is installed
echo "uv is already installed."; \
fi
-setup-venv: ## Create virtual environment
- uv venv $(PYTHON_ARG)
-
-install-dependencies: setup-venv ## Install all dependencies including extras
- uv sync $(PYTHON_ARG) --all-extras --reinstall
-
-install-hooks: ## Install pre-commit hooks
+install: install-uv ## Install uv, dependencies, and pre-commit hooks
+ uv sync $(PYTHON_ARG) --all-extras
+ @# Reinstall pyiceberg if Cython extensions (.so) are missing after `make clean` (see #2869)
+ @if ! find pyiceberg -name "*.so" 2>/dev/null | grep -q .; then \
Review Comment:
Makes sense can't really think of a way around this until we deprecate cython (if...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] infra: allow rerun `make install` without prompt [iceberg-python]
Posted by "geruh (via GitHub)" <gi...@apache.org>.
geruh commented on code in PR #2979:
URL: https://github.com/apache/iceberg-python/pull/2979#discussion_r2744206272
##########
Makefile:
##########
@@ -66,11 +66,15 @@ install-uv: ## Ensure uv is installed
echo "uv is already installed."; \
fi
-setup-venv: ## Create virtual environment
- uv venv $(PYTHON_ARG)
+setup-venv: ## Create virtual environment (if not exists)
Review Comment:
Niceeee!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] infra: allow rerun `make install` without prompt [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #2979:
URL: https://github.com/apache/iceberg-python/pull/2979#discussion_r2742349303
##########
Makefile:
##########
@@ -66,11 +66,15 @@ install-uv: ## Ensure uv is installed
echo "uv is already installed."; \
fi
-setup-venv: ## Create virtual environment
- uv venv $(PYTHON_ARG)
+setup-venv: ## Create virtual environment (if not exists)
Review Comment:
yea good one, claude suggested to just use `uv sync` since it will create and manage the virtualenv automatically
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] infra: allow rerun `make install` without prompt [iceberg-python]
Posted by "geruh (via GitHub)" <gi...@apache.org>.
geruh commented on code in PR #2979:
URL: https://github.com/apache/iceberg-python/pull/2979#discussion_r2740413594
##########
Makefile:
##########
@@ -66,11 +66,15 @@ install-uv: ## Ensure uv is installed
echo "uv is already installed."; \
fi
-setup-venv: ## Create virtual environment
- uv venv $(PYTHON_ARG)
+setup-venv: ## Create virtual environment (if not exists)
Review Comment:
Nit: shell check is nice but you could also use `uv venv $(PYTHON_ARG) --allow-existing` the allow existing flag will suppress and handle version swaps correctly.
##########
Makefile:
##########
@@ -66,11 +66,15 @@ install-uv: ## Ensure uv is installed
echo "uv is already installed."; \
fi
-setup-venv: ## Create virtual environment
- uv venv $(PYTHON_ARG)
+setup-venv: ## Create virtual environment (if not exists)
+ @if [ ! -d ".venv" ]; then \
+ uv venv $(PYTHON_ARG); \
+ else \
+ echo "Virtual environment already exists at .venv"; \
+ fi
install-dependencies: setup-venv ## Install all dependencies including extras
- uv sync $(PYTHON_ARG) --all-extras --reinstall
+ uv sync $(PYTHON_ARG) --all-extras
Review Comment:
The --reinstall flag was added because of the Cython compiled code. After running `make clean` the .so files are removed and uv sync without --reinstall won't rebuild them.
We could use `--reinstall-package pyiceberg` conditionally instead. I see that it is a bit faster, and potentially only run when `.so` files don't exist.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] infra: allow rerun `make install` without prompt [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #2979:
URL: https://github.com/apache/iceberg-python/pull/2979#discussion_r2742598053
##########
Makefile:
##########
@@ -66,17 +66,15 @@ install-uv: ## Ensure uv is installed
echo "uv is already installed."; \
fi
-setup-venv: ## Create virtual environment
- uv venv $(PYTHON_ARG)
-
-install-dependencies: setup-venv ## Install all dependencies including extras
- uv sync $(PYTHON_ARG) --all-extras --reinstall
-
-install-hooks: ## Install pre-commit hooks
+install: install-uv ## Install uv, dependencies, and pre-commit hooks
+ uv sync $(PYTHON_ARG) --all-extras
+ @# Reinstall pyiceberg if Cython extensions (.so) are missing after `make clean` (see #2869)
+ @if ! find pyiceberg -name "*.so" 2>/dev/null | grep -q .; then \
Review Comment:
since `uv sync` is super fast, its fine to call it (potentially) twice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org