You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yihua (via GitHub)" <gi...@apache.org> on 2026/04/17 18:55:17 UTC
[I] Reduce docker image size for Java 17 [hudi]
yihua opened a new issue, #18523:
URL: https://github.com/apache/hudi/issues/18523
### Task Description
The docker image size is larger for Java 17 build (`./build_docker_images.sh --hadoop-version 3.4.0 --spark-version 4.0.1 --hive-version 3.1.3 --multi-arch`). We should see if there's way to reduce the image size.
E.g., https://hub.docker.com/r/apachehudi/hudi-hadoop_3.4.0-base/tags
### Task Type
Test enhancement
### Related Issues
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Reduce docker image size for Java 17 [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous closed issue #18523: Reduce docker image size for Java 17
URL: https://github.com/apache/hudi/issues/18523
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Reduce docker image size for Java 17 [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on issue #18523:
URL: https://github.com/apache/hudi/issues/18523#issuecomment-4317406703
@kartikeyaagrawal have you tried to build the image with `./build_docker_images.sh --hadoop-version 3.4.0 --spark-version 4.0.1 --hive-version 3.1.3` and see the size difference before and after? The current size is 1.1GB (https://hub.docker.com/r/apachehudi/hudi-hadoop_3.4.0-base) whereas older hudi hadoop base image is 465MB (https://hub.docker.com/r/apachehudi/hudi-hadoop_2.8.4-base). which is different from what is being claimed here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Reduce docker image size for Java 17 [hudi]
Posted by "kartikeyaagrawal (via GitHub)" <gi...@apache.org>.
kartikeyaagrawal commented on issue #18523:
URL: https://github.com/apache/hudi/issues/18523#issuecomment-4290217310
## Image Size Reductions (`base_java17`)
**Starting size:** ~3.56 GB
### Changes & Savings
- **JDK → JRE base image**
`17-jdk` → `17-jre-jammy`
→ **~200 MB saved**
_Removes compiler toolchain since the container only runs Hadoop._
- **Python: source build → apt install**
`python3-minimal` + `python3-pip` (in `spark_base`)
→ **~600 MB saved**
_Avoids installing `build-essential`, dev headers, and compiling CPython with `--enable-optimizations`._
- **Multi-stage build optimizations**
- `--no-install-recommends`
- Cleaned apt lists
- Dropped unused `wget` and `.asc` downloads
→ **~100 MB saved**
_Builder stage isolates curl/ca-certs and Hadoop tarball._
---
### Total Impact
**~3.56 GB → ~2.58 GB**
→ **~900 MB saved (~27% reduction)**
Should I add these ??
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Reduce docker image size for Java 17 [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on issue #18523:
URL: https://github.com/apache/hudi/issues/18523#issuecomment-4701341444
Fixed in https://github.com/apache/hudi/pull/18542
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org