[GitHub] [arrow-rs] bjchambers opened a new issue #725: `append_nulls` and `append_trusted_len_iter` for PrimitiveBuilder

[GitBox <gi...@apache.org>]

bjchambers opened a new issue #725:
URL: https://github.com/apache/arrow-rs/issues/725


   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   
   I have cases where I've identified a range of numbers to add to a builder. Or a number of nulls to add. My only options with the existing PrimitiveBuilder interfaces is to either (a) add each value/null individually or (b) create a *vector* containing the numbers/nulls I wish to add, and then use `append_values` or `append_slice`.
   
   Both of these extra work than is strictly necessary -- in one case, capacity checks after every value and in the other materializing an intermediate vector.
   
   **Describe the solution you'd like**
   
   * `pub fn append_nulls(&mut self, num_nulls: usize) -> Result<(), ArrowError>`. Same as calling `append_nulls` `num_nulls` times.
   * `pub fn append_trusted_len_iter_values(&mut self, iter: impl IntoIterator<T::Native>) -> Result<(), ArrowError>`. Behaves similarly to collecting the items into a vector and then calling `append_slice(&vector)`.
   * Could also add a version for `Option<T::Native>`.
   
   **Describe alternatives you've considered**
   As noted in the description -- multiple calls to `append_value`/`append_null` (not ideal due to checking the capacity after each call) or creating a vector and then calling `append_values` or `append_slice` (not ideal due to the need to collect/materialize values first).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] bjchambers commented on issue #725: `append_nulls` and `append_trusted_len_iter` for PrimitiveBuilder

[GitBox <gi...@apache.org>]

bjchambers commented on issue #725:
URL: https://github.com/apache/arrow-rs/issues/725#issuecomment-907558472


   I have a prototype of these methods, that improved the performance of a 2-way merge kernel 20-30% in some 100k element microbenchmarks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] alamb closed issue #725: `append_nulls` and `append_trusted_len_iter` for PrimitiveBuilder

[GitBox <gi...@apache.org>]

alamb closed issue #725:
URL: https://github.com/apache/arrow-rs/issues/725


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] bjchambers edited a comment on issue #725: `append_nulls` and `append_trusted_len_iter` for PrimitiveBuilder

[GitBox <gi...@apache.org>]

bjchambers edited a comment on issue #725:
URL: https://github.com/apache/arrow-rs/issues/725#issuecomment-907558472


   I have a prototype of these methods, that improved the performance of a 2-way merge kernel 20-30% in some 100k element microbenchmarks.
   
   https://github.com/bjchambers/arrow-rs/tree/725-append_nulls_iter


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org