[PR] perf(parquet): encode RLE levels in batches [arrow-go]

["fallintoplace (via GitHub)" <gi...@apache.org>]

fallintoplace opened a new pull request, #1177:
URL: https://github.com/apache/arrow-go/pull/1177

   ### What changed
   
   - add a batch path for repetition and definition level encoding
   - scan repeated runs once instead of calling the scalar encoder for every level
   - keep the scalar path around literal boundaries
   - preserve the existing encoded bytes across split streaming calls
   - add level encoder and scalar-versus-batch benchmarks
   
   ### Benchmarks
   
   Apple M1 Pro:
   
   | RLE workload | Before | After | Change |
   | --- | ---: | ---: | ---: |
   | 65,536 all defined | 151,051 ns/op | 31,143 ns/op | -79% |
   | 65,536 mostly defined | 324,228 ns/op | 252,108 ns/op | -22% |
   | 65,536 alternating | 339,128 ns/op | 341,149 ns/op | +0.6% |
   
   The alternating case is the fallback path. Allocations remain at zero.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] perf(parquet): encode RLE levels in batches [arrow-go]

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade merged PR #1177:
URL: https://github.com/apache/arrow-go/pull/1177


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org