You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "JNSimba (via GitHub)" <gi...@apache.org> on 2026/04/03 08:54:15 UTC
[PR] [Improve] Enable gzip compression by default for StreamLoad writes [doris-spark-connector]
JNSimba opened a new pull request, #356:
URL: https://github.com/apache/doris-spark-connector/pull/356
## Summary
- Enable gzip compression by default for StreamLoad writes to reduce network transfer and improve write performance
- Arrow format is excluded since binary columnar data has low compression benefit
- Fix `compress_type` check from `"gzip"` to `"gz"` to match Doris server-side expected value
- Users can disable compression by setting `doris.sink.properties.compress_type=`
## Changes
- `AbstractStreamLoadProcessor.java`: Add `putIfAbsent("compress_type", "gz")` for non-Arrow formats, simplify compression check
- `DorisWriterITCase.scala`: Add ITCase for explicit gz compression and explicit no-compression
## Test plan
- [ ] Existing CSV/JSON/Arrow write ITCases pass (covers default gzip compression for CSV/JSON, no compression for Arrow)
- [ ] New `testSinkCsvGzCompression` passes (explicit gz compression)
- [ ] New `testSinkCsvNoCompression` passes (explicit disable compression)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improve] Enable gzip compression by default for StreamLoad writes [doris-spark-connector]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba merged PR #356:
URL: https://github.com/apache/doris-spark-connector/pull/356
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org