You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "timmyb32r (via GitHub)" <gi...@apache.org> on 2026/10/23 19:29:07 UTC

[PR] Fix memory reservation leak in Buffer::into_vec [arrow-rs]

timmyb32r opened a new pull request, #11878:
URL: https://github.com/apache/arrow-rs/pull/11878

   ### Which issue does this PR close?
   
   None — standalone bug fix.
   
   ### Rationale for this change
   
   `Buffer::into_vec` calls `mem::forget(bytes)` to transfer ownership of the allocation to a `Vec`. With the `pool` feature enabled, this also leaks the memory reservation: its destructor never runs, so the pool continues reporting the allocation as used even after the vector is dropped.
   
   ### What changes are included in this PR?
   
   Release the memory reservation before forgetting `Bytes` and transferring the allocation to the vector.
   
   ### Are these changes tested?
   
   Added a regression test verifying that:
   - Successful conversion releases the reservation and preserves the allocation and its contents.
   - Failed conversion of a shared buffer keeps the reservation intact.
   
   The test fails before the fix and passes afterward.
   
   Validation passed:
   - `cargo test -p arrow-buffer`
   - `cargo test -p arrow-buffer --all-features`
   - `cargo test --workspace`
   - `cargo clippy --workspace --all-targets --all-features -- -D warnings`
   - Formatting and spelling checks.
   
   ### Are there any significant user-facing changes?
   
   With the `pool` feature enabled, converting a buffer into a vector now releases its memory reservation.
   
   ### AI assistance
   
   Codex identified the issue, generated the fix and regression test, ran validation, and drafted this PR description.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix memory reservation leak in Buffer::into_vec [arrow-rs]

Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #11878:
URL: https://github.com/apache/arrow-rs/pull/11878#issuecomment-6386562476

   cc @cetra3 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix memory reservation leak in Buffer::into_vec [arrow-rs]

Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #11878:
URL: https://github.com/apache/arrow-rs/pull/11878#issuecomment-6455838668

   taking a look 👀 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix memory reservation leak in Buffer::into_vec [arrow-rs]

Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on code in PR #11878:
URL: https://github.com/apache/arrow-rs/pull/11878#discussion_r4395202324


##########
arrow-buffer/src/buffer/immutable.rs:
##########
@@ -927,6 +929,31 @@ mod tests {
         buffer.slice_with_length(2, usize::MAX);
     }
 
+    #[test]
+    #[cfg(feature = "pool")]
+    fn test_into_vec_releases_reservation() {
+        use crate::pool::{MemoryPool, TrackingMemoryPool};

Review Comment:
   can we place this import at the test mod levle



##########
arrow-buffer/src/buffer/immutable.rs:
##########


Review Comment:
   might be worth explaining that when bool is enabled successful conversions release the reservation and the returned vec is not tracked.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix memory reservation leak in Buffer::into_vec [arrow-rs]

Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid merged PR #11878:
URL: https://github.com/apache/arrow-rs/pull/11878


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix memory reservation leak in Buffer::into_vec [arrow-rs]

Posted by "timmyb32r (via GitHub)" <gi...@apache.org>.
timmyb32r commented on PR #11878:
URL: https://github.com/apache/arrow-rs/pull/11878#issuecomment-6481324874

   > thanks @timmyb32r & @cetra3
   > 
   > left two comments, should be good to go once those are addressed
   
   Thanks! Both review comments are addressed in the latest commit.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix memory reservation leak in Buffer::into_vec [arrow-rs]

Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #11878:
URL: https://github.com/apache/arrow-rs/pull/11878#issuecomment-6472770029

   will queue when threads are addressed 🙇 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org