You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@parquet.apache.org by "iemejia (via GitHub)" <gi...@apache.org> on 2026/05/23 10:07:25 UTC
[PR] Simplify CI: collapse codec matrix and add zstd coverage [parquet-java]
iemejia opened a new pull request, #3580:
URL: https://github.com/apache/parquet-java/pull/3580
## Summary
Remove the codec matrix dimension from the CI Hadoop 3 workflow. The codec split only parameterized a single integration test (`FileEncodingsIT`) while duplicating the entire build+test suite across 4 jobs instead of 2.
## Motivation
Analysis of recent CI runs shows:
- The `verify` step takes ~12 min (JDK 17) / ~16 min (JDK 11) **regardless of which codecs are set**
- The difference between codec variants for the same JDK is <1 minute
- Each redundant job wastes ~6 min of `before_install` + `install` overhead
- Total savings: **~46% fewer billable CI minutes** (from ~81 to ~44 min per run)
## Changes
- Remove the `codes` matrix dimension (was: `['uncompressed,brotli', 'gzip,snappy']`)
- Set `TEST_CODECS` to all codecs in a single string: `uncompressed,brotli,gzip,snappy,zstd`
- Add `zstd` to the tested codecs (was previously missing)
- Jobs go from 4 (2 JDKs x 2 codec groups) to 2 (2 JDKs)
## Impact
- Wall-clock CI time: unchanged (jobs already ran in parallel)
- Billable minutes: ~46% reduction
- Test coverage: improved (adds zstd)
- `FileEncodingsIT` now tests 8 types x 5 codecs = 40 parameterized cases in one shot
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Simplify CI: collapse codec matrix and add zstd coverage [parquet-java]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko merged PR #3580:
URL: https://github.com/apache/parquet-java/pull/3580
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Simplify CI: collapse codec matrix and add zstd coverage [parquet-java]
Posted by "iemejia (via GitHub)" <gi...@apache.org>.
iemejia commented on PR #3580:
URL: https://github.com/apache/parquet-java/pull/3580#issuecomment-4689253265
Gentle ping @Fokko. This one is tiny but the impact for the contributors is huge (Notice in the logs that now we cover even more conmpression types in less time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org