You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "geruh (via GitHub)" <gi...@apache.org> on 2026/03/07 21:19:22 UTC
Re: [PR] Catalog tests run on Polaris on CI [iceberg-python]
geruh commented on code in PR #3106:
URL: https://github.com/apache/iceberg-python/pull/3106#discussion_r2900537600
##########
Makefile:
##########
@@ -123,6 +125,38 @@ test-integration-cleanup: ## Clean up integration test environment
fi
$(CLEANUP_COMMAND)
+test-polaris-setup: install ## Start Docker services for Polaris integration tests
+ docker compose -f dev/docker-compose-polaris.yml kill
+ docker compose -f dev/docker-compose-polaris.yml rm -f
+ docker compose -f dev/docker-compose-polaris.yml up -d --build --wait
+ uv run $(PYTHON_ARG) python dev/provision_polaris.py > dev/polaris_creds.env
+
+test-polaris-exec: ## Run Polaris integration tests
+ @eval $$(cat dev/polaris_creds.env) && \
+ PYICEBERG_TEST_CATALOG="polaris" \
+ PYICEBERG_CATALOG__POLARIS__TYPE="rest" \
+ PYICEBERG_CATALOG__POLARIS__URI="http://localhost:8181/api/catalog" \
+ PYICEBERG_CATALOG__POLARIS__OAUTH2_SERVER_URI="http://localhost:8181/api/catalog/v1/oauth/tokens" \
+ PYICEBERG_CATALOG__POLARIS__CREDENTIAL="$$CLIENT_ID:$$CLIENT_SECRET" \
+ PYICEBERG_CATALOG__POLARIS__SCOPE="PRINCIPAL_ROLE:ALL" \
+ PYICEBERG_CATALOG__POLARIS__WAREHOUSE="polaris" \
+ PYICEBERG_CATALOG__POLARIS__HEADER__X_ICEBERG_ACCESS_DELEGATION="vended-credentials" \
+ PYICEBERG_CATALOG__POLARIS__HEADER__REALM="POLARIS" \
+ PYICEBERG_CATALOG__POLARIS__S3__ENDPOINT="http://localhost:9000" \
+ PYICEBERG_CATALOG__POLARIS__S3__ACCESS_KEY_ID="admin" \
+ PYICEBERG_CATALOG__POLARIS__S3__SECRET_ACCESS_KEY="password" \
+ PYICEBERG_CATALOG__POLARIS__S3__REGION="us-east-1" \
+ $(TEST_RUNNER) pytest tests/integration/test_catalog.py -k "rest_test_catalog and not test_update_namespace_properties" $(PYTEST_ARGS)
+ # Skip test_update_namespace_properties: Polaris triggers a CommitConflictException when updates and removals are in the same request.
+
+
Review Comment:
nit: remove extra new line
##########
Makefile:
##########
@@ -123,6 +125,38 @@ test-integration-cleanup: ## Clean up integration test environment
fi
$(CLEANUP_COMMAND)
+test-polaris-setup: install ## Start Docker services for Polaris integration tests
+ docker compose -f dev/docker-compose-polaris.yml kill
+ docker compose -f dev/docker-compose-polaris.yml rm -f
+ docker compose -f dev/docker-compose-polaris.yml up -d --build --wait
+ uv run $(PYTHON_ARG) python dev/provision_polaris.py > dev/polaris_creds.env
+
+test-polaris-exec: ## Run Polaris integration tests
+ @eval $$(cat dev/polaris_creds.env) && \
+ PYICEBERG_TEST_CATALOG="polaris" \
Review Comment:
Not sure if I like populating the PyIceberg configs here and convoluting the MF. this makes for a cool case to have a persistent `.pyiceberg.yaml` in `/dev` and have all tests point there lol
##########
Makefile:
##########
@@ -123,6 +125,38 @@ test-integration-cleanup: ## Clean up integration test environment
fi
$(CLEANUP_COMMAND)
+test-polaris-setup: install ## Start Docker services for Polaris integration tests
+ docker compose -f dev/docker-compose-polaris.yml kill
+ docker compose -f dev/docker-compose-polaris.yml rm -f
+ docker compose -f dev/docker-compose-polaris.yml up -d --build --wait
+ uv run $(PYTHON_ARG) python dev/provision_polaris.py > dev/polaris_creds.env
+
+test-polaris-exec: ## Run Polaris integration tests
+ @eval $$(cat dev/polaris_creds.env) && \
+ PYICEBERG_TEST_CATALOG="polaris" \
+ PYICEBERG_CATALOG__POLARIS__TYPE="rest" \
+ PYICEBERG_CATALOG__POLARIS__URI="http://localhost:8181/api/catalog" \
+ PYICEBERG_CATALOG__POLARIS__OAUTH2_SERVER_URI="http://localhost:8181/api/catalog/v1/oauth/tokens" \
+ PYICEBERG_CATALOG__POLARIS__CREDENTIAL="$$CLIENT_ID:$$CLIENT_SECRET" \
+ PYICEBERG_CATALOG__POLARIS__SCOPE="PRINCIPAL_ROLE:ALL" \
+ PYICEBERG_CATALOG__POLARIS__WAREHOUSE="polaris" \
+ PYICEBERG_CATALOG__POLARIS__HEADER__X_ICEBERG_ACCESS_DELEGATION="vended-credentials" \
+ PYICEBERG_CATALOG__POLARIS__HEADER__REALM="POLARIS" \
+ PYICEBERG_CATALOG__POLARIS__S3__ENDPOINT="http://localhost:9000" \
+ PYICEBERG_CATALOG__POLARIS__S3__ACCESS_KEY_ID="admin" \
+ PYICEBERG_CATALOG__POLARIS__S3__SECRET_ACCESS_KEY="password" \
+ PYICEBERG_CATALOG__POLARIS__S3__REGION="us-east-1" \
+ $(TEST_RUNNER) pytest tests/integration/test_catalog.py -k "rest_test_catalog and not test_update_namespace_properties" $(PYTEST_ARGS)
+ # Skip test_update_namespace_properties: Polaris triggers a CommitConflictException when updates and removals are in the same request.
Review Comment:
Is this a known Polaris bug or intentional behavior? Is there a tracking issue against this? We should link for future context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org