You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "fallintoplace (via GitHub)" <gi...@apache.org> on 2026/08/13 12:07:37 UTC
[PR] perf(table): precompute data writer invariants [iceberg-go]
fallintoplace opened a new pull request, #1813:
URL: https://github.com/apache/iceberg-go/pull/1813
## What changed
- Precompute the stats plan, current partition spec, Parquet row-group size, write properties, schema options, and file extension when the writer is created.
- Reuse those values for every write task.
- Add a focused writer benchmark.
## Benchmark
Apple M1 Pro, `GOMAXPROCS=1`, median of 5 runs with `-benchtime=1s`:
- `origin/main`: 8.53 us/op, 97 allocs/op, 6.49 KB/op
- this branch: 6.38 us/op, 72 allocs/op, 5.01 KB/op
- about 1.34x faster and 26% fewer allocations
The benchmark uses a no-op file format so it measures the per-task writer setup and schema conversion without filesystem or Parquet I/O.
## Tests
- `go test ./...`
- `go test -race ./table/...`
- `go vet ./table/...`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] perf(table): precompute data writer invariants [iceberg-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade merged PR #1813:
URL: https://github.com/apache/iceberg-go/pull/1813
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] perf(table): precompute data writer invariants [iceberg-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #1813:
URL: https://github.com/apache/iceberg-go/pull/1813#issuecomment-5319771624
Since this is a draft, i'll hold off on further review until it's marked ready
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org