[GitHub] [arrow] westonpace opened a new issue, #35793: [C++] MemoryPool::Allocate returns an error when the user provides small values for alignment

["westonpace (via GitHub)" <gi...@apache.org>]

westonpace opened a new issue, #35793:
URL: https://github.com/apache/arrow/issues/35793

   ### Describe the enhancement requested
   
   The various allocators have minimum requirements for what can be passed to their "allocate aligned" utility.
   
   For example, `posix_memalign` states:
   
   >  The address of the allocated memory will be a multiple of alignment, which must be a power of two and a multiple of sizeof(void *)
   
   First, we should probably do our own check that alignment is a power of 2 and reject it with a clear error message.  We should also add this requirement to the MemoryPool::Allocate method's documentation.
   
   Second, if the user passes in a value that is a power of two, but is too small for the underlying allocator, then we should silently raise it to the first valid power of two.  For example, if the user provides 4 then we should just raise that to 8.  Any pointer that is 8-byte aligned is also 4-byte aligned so we fulfilled the user's request.  This means that users don't have to know the details of the underlying allocator.
   
   ### Component(s)
   
   C++


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on issue #35793: [C++] MemoryPool::Allocate returns an error when the user provides small values for alignment

["mapleFU (via GitHub)" <gi...@apache.org>]

mapleFU commented on issue #35793:
URL: https://github.com/apache/arrow/issues/35793#issuecomment-1565227943

   Seems that `velox` will force align the data:
   
   ```
     FOLLY_ALWAYS_INLINE int64_t sizeAlign(int64_t size) {
       const auto remainder = size % alignment_;
       return (remainder == 0) ? size : (size + alignment_ - remainder);
     }
   
   void* MemoryPoolImpl::allocate(int64_t size) {
     CHECK_AND_INC_MEM_OP_STATS(Allocs);
     const auto alignedSize = sizeAlign(size);
     reserve(alignedSize);
     void* buffer = allocator_->allocateBytes(alignedSize, alignment_);
     if (FOLLY_UNLIKELY(buffer == nullptr)) {
       release(alignedSize);
       VELOX_MEM_ALLOC_ERROR(fmt::format(
           "{} failed with {} bytes from {}", __FUNCTION__, size, toString()));
     }
     return buffer;
   }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] westonpace commented on issue #35793: [C++] MemoryPool::Allocate returns an error when the user provides small values for alignment

["westonpace (via GitHub)" <gi...@apache.org>]

westonpace commented on issue #35793:
URL: https://github.com/apache/arrow/issues/35793#issuecomment-1583261884

   > Seems that velox will force align the data:
   
   Probably.  We do something similar in our memory pool (everything is always 64-byte aligned).  However, unaligned buffers can still occur when data comes from users.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org