You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "zhangshenghang (via GitHub)" <gi...@apache.org> on 2026/07/29 01:36:04 UTC
[PR] [Improve][Connector-V2] Guard POI Excel reads by file size [seatunnel]
zhangshenghang opened a new pull request, #11591:
URL: https://github.com/apache/seatunnel/pull/11591
### Purpose
Prevent large Excel files from OOMing workers when they are read through the POI engine.
### What changed
- Add `poi_excel_max_file_size` with a 50 MB default for POI Excel reads.
- Fail fast when POI would read an Excel file larger than the configured limit, and suggest `excel_engine = EasyExcel`.
- Keep the default `excel_engine` as POI for compatibility.
- Register Excel reader options in file source option metadata.
- Add tests for POI fail-fast behavior and EasyExcel bypassing the POI limit.
- Update LocalFile EN/ZH docs.
### Compatibility
The default engine remains POI. Users can increase `poi_excel_max_file_size` if POI is required, or switch to `EasyExcel` for large streaming reads.
### Tests
- `./mvnw -pl seatunnel-connectors-v2/connector-file/connector-file-base spotless:apply -DskipTests`
- `./mvnw -pl seatunnel-connectors-v2/connector-file/connector-file-base -Dtest=ExcelReadStrategyTest test`
- `./mvnw -pl seatunnel-connectors-v2/connector-file/connector-file-base -DskipTests install`
- `./mvnw -pl seatunnel-connectors-v2/connector-file/connector-file-local,seatunnel-connectors-v2/connector-file/connector-file-hadoop,seatunnel-connectors-v2/connector-file/connector-file-s3,seatunnel-connectors-v2/connector-file/connector-file-sftp,seatunnel-connectors-v2/connector-file/connector-file-obs,seatunnel-connectors-v2/connector-file/connector-file-oss,seatunnel-connectors-v2/connector-file/connector-file-cos,seatunnel-connectors-v2/connector-file/connector-file-ftp,seatunnel-connectors-v2/connector-file/connector-file-jindo-oss -DskipTests compile`
- `git diff --check`
### Note
A broader `-am` connector compile currently stops in `seatunnel-config-shade` before reaching the changed connector modules, with missing shaded Typesafe Config classes. The targeted connector-module compile above passed after installing the local `connector-file-base` artifact.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improve][Connector-V2] Guard POI Excel reads by file size [seatunnel]
Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo merged PR #11591:
URL: https://github.com/apache/seatunnel/pull/11591
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org