You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "jarohen (via GitHub)" <gi...@apache.org> on 2025/11/05 16:52:10 UTC
[I] Calling `retain` on a closed ArrowBuf revives it, into an undefined state [arrow-java]
jarohen opened a new issue, #906:
URL: https://github.com/apache/arrow-java/issues/906
### Describe the bug, including details regarding any error messages, version, and platform.
```kotlin
@Test
fun testArrowBufRetainBug() {
RootAllocator().use { al ->
val buf = al.buffer(10)
buf.close()
// correctly throws
assertThrows<IllegalArgumentException> { buf.referenceManager.retain() }
assertEquals(0, buf.refCnt()) // nope, it's 1
// doesn't throw, ref-count is 1 - so the caller assumes they've successfully taken a reference
// but the underlying memory has already been reclaimed and re-used
assertThrows<IllegalArgumentException> { buf.referenceManager.retain() }
}
}
```
Caused by the `getAndAdd` in `BufferLedger.retain(int)` - this is what leaves the ref-count positive, so on the next call, this doesn't fail.
Some kind of compareAndSet instead, perhaps?
Cheers,
James
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Calling `retain` on a closed ArrowBuf revives it, into an undefined state [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #906:
URL: https://github.com/apache/arrow-java/issues/906#issuecomment-3509039992
Would resetting the count before throwing be a reasonable solution for your case? That way we only touch the error path
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Calling `retain` on a closed ArrowBuf revives it, into an undefined state [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #906:
URL: https://github.com/apache/arrow-java/issues/906#issuecomment-3494354483
Hmm, I'd argue that after the first exception the buffer is already in an undefined state. compareAndSet would require a CAS loop - might be a pessimization for the common case? We could reset back to 0 before throwing instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Calling `retain` on a closed ArrowBuf revives it, into an undefined state [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #906:
URL: https://github.com/apache/arrow-java/issues/906#issuecomment-3499946705
I'm not too sure if we have benchmarks - coverage is spotty for this repo. There's also the question of the right benchmark (frankly, I would _assume_ an application should not be spamming reference count updates, in which case a CAS loop would presumably not cost anything over an atomic add?).
I would love to use the new FFM APIs instead...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Calling `retain` on a closed ArrowBuf revives it, into an undefined state [arrow-java]
Posted by "jarohen (via GitHub)" <gi...@apache.org>.
jarohen commented on issue #906:
URL: https://github.com/apache/arrow-java/issues/906#issuecomment-3499183823
> I'd argue that after the first exception the buffer is already in an undefined state.
That's completely reasonable 🙂
> compareAndSet would require a CAS loop - might be a pessimization for the common case?
Yep, understood. You folks got benchmarks we could check this against? Suspect ours are a bit high-level, might get lost in noise.
No urgency on our part, we've put more guards in 🙂
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Calling `retain` on a closed ArrowBuf revives it, into an undefined state [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #906:
URL: https://github.com/apache/arrow-java/issues/906#issuecomment-3499947392
@jbonofre not sure if you have opinions or experience from your arrow-java application :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Calling `retain` on a closed ArrowBuf revives it, into an undefined state [arrow-java]
Posted by "jarohen (via GitHub)" <gi...@apache.org>.
jarohen commented on issue #906:
URL: https://github.com/apache/arrow-java/issues/906#issuecomment-3501651292
> I would love to use the new FFM APIs instead...
That's what we're starting to do here 🙂
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Calling `retain` on a closed ArrowBuf revives it, into an undefined state [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #906:
URL: https://github.com/apache/arrow-java/issues/906#issuecomment-3509040298
> > I would love to use the new FFM APIs instead...
>
> That's what we're starting to do here 🙂
I would like to hear more if there's anything that can be shared :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Calling `retain` on a closed ArrowBuf revives it, into an undefined state [arrow-java]
Posted by "jarohen (via GitHub)" <gi...@apache.org>.
jarohen commented on issue #906:
URL: https://github.com/apache/arrow-java/issues/906#issuecomment-3548022212
hey @lidavidm - turns out we had to put this fix in in the end anyway (see linked commits) because it led to problems on our longer running environments.
fwiw the CAS loop didn't seem to add anything to our benchmarks AFAICT 🙂
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org