[GitHub] [arrow] mbrubeck commented on pull request #9235: ARROW-11291: [Rust] Add extend to MutableBuffer (-25% for aritmetic, -97% for length)

[GitBox <gi...@apache.org>]

mbrubeck commented on pull request #9235:
URL: https://github.com/apache/arrow/pull/9235#issuecomment-763846523


   I was able to speed up the fast path of `extend_from_iter` by a few percent, by splitting the loop into two parts.  This version is only about 8% slower than `extend_from_trusted_len_iter`.  (Previously it was about 12% slower.)  Both are still much slower than `extend_from_slice`.
   
   <details>
   
   ```rust
   #[inline]
   fn extend_from_iter<T: ArrowNativeType, I: Iterator<Item = T>>(
       &mut self,
       mut iterator: I,
   ) {
       let size = std::mem::size_of::<T>();
       let (lower, _) = iterator.size_hint();
       let additional = lower * size;
       self.reserve(additional);
   
       // this is necessary because of https://github.com/rust-lang/rust/issues/32155
       let mut len = SetLenOnDrop::new(&mut self.len);
       let mut dst = unsafe { self.data.as_ptr().add(len.local_len) as *mut T };
       let capacity = self.capacity;
   
       while len.local_len + size <= capacity {
           if let Some(item) = iterator.next() {
               unsafe {
                   std::ptr::write(dst, item);
                   dst = dst.add(1);
               }
               len.local_len += size;
           } else {
               break;
           }
       }
       drop(len);
   
       iterator.for_each(|item| self.push(item));
   }
   ```
   </details>
   
   I haven't benchmarked the slow path (the second loop), which I implemented using safe code.  It's possible that it could be made faster with unsafe code, but I don't expect large gains there.
   
   My complete code including benchmarks is on this branch: https://github.com/apache/arrow/compare/master...mbrubeck:length_faster


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org