You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by li...@apache.org on 2025/07/31 12:49:49 UTC
(iceberg-rust) branch main updated: Updates Podman Instructions, Docker-Compose, & Spark Script (#1553)
This is an automated email from the ASF dual-hosted git repository.
liurenjie1024 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/iceberg-rust.git
The following commit(s) were added to refs/heads/main by this push:
new c390ad35 Updates Podman Instructions, Docker-Compose, & Spark Script (#1553)
c390ad35 is described below
commit c390ad354600d04d55fad649387e27345e4abdc0
Author: Kurtis Wright <Ku...@users.noreply.github.com>
AuthorDate: Thu Jul 31 05:49:45 2025 -0700
Updates Podman Instructions, Docker-Compose, & Spark Script (#1553)
## Which issue does this PR close?
- No current PR, was found while testing 0.6.0 RC
## What changes are included in this PR?
`CONTRIBUTING.md`
`docs/contributing/podman.md`
- Podman instructions have been updated for clarity and to emphasis its
usage in a Linux environment.
`crates/integration_tests/testdata/docker-compose.yaml`
- Updated to remove deprecated Link feature, which breaks when used with
Podman.
https://stackoverflow.com/questions/35832095/difference-between-links-and-depends-on-in-docker-compose-yml
`crates/integration_tests/testdata/spark/provision.py`
- Added Create Namespace `default` to fix breaking tests.
## Are these changes tested?
- Tested by running `make build && make test` against the following
platforms:
* Arch Linux x86_64 with Podman && Docker-Compose
* MacOS M1 arm64 with Orbstack
---
CONTRIBUTING.md | 3 +--
.../integration_tests/testdata/docker-compose.yaml | 3 ---
.../integration_tests/testdata/spark/provision.py | 2 ++
docs/contributing/podman.md | 27 +++++++++++++++++-----
4 files changed, 24 insertions(+), 11 deletions(-)
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 8c143ba9..712128f5 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -104,8 +104,7 @@ cargo 1.69.0 (6e9a83356 2023-04-12)
Currently, iceberg-rust uses Docker to set up environment for integration tests. Native Docker has some limitations, please check (https://github.com/apache/iceberg-rust/pull/748). Please use Orbstack or Podman.
For MacOS users, you can install [OrbStack as a docker alternative](docs/contributing/orbstack.md).
-
-For podman, refer to [Using Podman instead of Docker](docs/contributing/podman.md)
+For Podman users, refer to [Using Podman instead of Docker](docs/contributing/podman.md)
## Build
diff --git a/crates/integration_tests/testdata/docker-compose.yaml b/crates/integration_tests/testdata/docker-compose.yaml
index f9851f3c..e2f9b7da 100644
--- a/crates/integration_tests/testdata/docker-compose.yaml
+++ b/crates/integration_tests/testdata/docker-compose.yaml
@@ -80,6 +80,3 @@ services:
- AWS_ACCESS_KEY_ID=admin
- AWS_SECRET_ACCESS_KEY=password
- AWS_REGION=us-east-1
- links:
- - rest:rest
- - minio:minio
diff --git a/crates/integration_tests/testdata/spark/provision.py b/crates/integration_tests/testdata/spark/provision.py
index 1929266b..899a49f6 100755
--- a/crates/integration_tests/testdata/spark/provision.py
+++ b/crates/integration_tests/testdata/spark/provision.py
@@ -30,6 +30,8 @@ spark = (
.getOrCreate()
)
+spark.sql(f"""CREATE NAMESPACE IF NOT EXISTS rest.default""")
+
spark.sql(
f"""
CREATE OR REPLACE TABLE rest.default.test_positional_merge_on_read_deletes (
diff --git a/docs/contributing/podman.md b/docs/contributing/podman.md
index 3281ad4d..eee93a0b 100644
--- a/docs/contributing/podman.md
+++ b/docs/contributing/podman.md
@@ -19,15 +19,16 @@
# Using Podman instead of Docker
-Iceberg-rust does not require containerization, except for integration tests, where "docker" and "docker-compose" are used to start containers for minio and various catalogs. Below instructions setup "rootful podman" and docker's official docker-compose plugin to run integration tests as an alternative to docker or Orbstack.
+Iceberg-rust does not require containerization, except for integration tests, where "docker" and "docker-compose" are used to start containers for minio and various catalogs. Below instructions setup "rootful podman" and docker's official docker-compose plugin to run integration tests as an alternative to docker or Orbstack.
1. Have podman v4 or newer.
- ```console
+ ```shell
$ podman --version
podman version 4.9.4-rhel
```
-
-2. Open file `/usr/bin/docker` and add the below contents:
+2. Create a docker wrapper script:
+
+ * Create a fresh `/usr/bin/docker` file and add the below contents:
```bash
#!/bin/sh
[ -e /etc/containers/nodocker ] || \
@@ -35,8 +36,13 @@ Iceberg-rust does not require containerization, except for integration tests, wh
exec sudo /usr/bin/podman "$@"
```
+ * Set new `/usr/bin/docker` file to executable.
+ ```shell
+ sudo chmod +x /usr/bin/docker
+ ```
+
3. Install the [docker compose plugin](https://docs.docker.com/compose/install/linux). Check for successful installation.
- ```console
+ ```shell
$ docker compose version
Docker Compose version v2.28.1
```
@@ -53,7 +59,7 @@ Iceberg-rust does not require containerization, except for integration tests, wh
```
6. Check that the following symlink exists.
- ```console
+ ```shell
$ ls -al /var/run/docker.sock
lrwxrwxrwx 1 root root 27 Jul 24 12:18 /var/run/docker.sock -> /var/run/podman/podman.sock
```
@@ -83,3 +89,12 @@ As of podman v4, ["To be succinct and simple, when running rootless containers,
* <https://www.redhat.com/sysadmin/container-ip-address-podman>
* <https://github.com/containers/podman/blob/main/docs/tutorials/basic_networking.md>
+
+# Debugging Note:
+ - Fix for error: `Error: short-name "apache/iceberg-rest-fixture" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"`
+ - Add or modify the `/etc/containers/registries.conf` file:
+ ```toml
+ [[registry]]
+ prefix = "docker.io"
+ location = "docker.io"
+ ```