You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "fallintoplace (via GitHub)" <gi...@apache.org> on 2026/08/29 11:41:55 UTC
[PR] perf(table): fetch orphan-cleanup manifest lists concurrently [iceberg-go]
fallintoplace opened a new pull request, #1969:
URL: https://github.com/apache/iceberg-go/pull/1969
## Summary
- **Reads snapshot manifest lists concurrently** using the existing cleanup concurrency limit.
- Stores each snapshot result separately, then folds results in snapshot order before deduplicating manifests.
- Calls `metadata.Snapshots()` once and adds regression and benchmark coverage.
## Benchmark
Representative `-benchtime=1x` results with a 1 ms artificial `Open` delay on an Apple M1 Pro:
| Snapshots | Concurrency 1 | Concurrency 16 |
| --- | ---: | ---: |
| 16 | 22.9 ms | 3.4 ms |
| 128 | 171.2 ms | 13.3 ms |
| 1,024 | 1.40 s | 99.6 ms |
## Tests
- `go test ./table/... -count=1`
- `go test -race ./table/... -count=1`
- `go test ./... -run '^$' -count=1`
- `go vet ./...`
- `golangci-lint run --timeout=10m`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] perf(table): fetch orphan-cleanup manifest lists concurrently [iceberg-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade merged PR #1969:
URL: https://github.com/apache/iceberg-go/pull/1969
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org