[GitHub] [arrow] paleolimbot commented on pull request #36307: GH-35649: [R] Always call `RecordBatchReader::ReadNext()` from DuckDB from the main R thread

["paleolimbot (via GitHub)" <gi...@apache.org>]

paleolimbot commented on PR #36307:
URL: https://github.com/apache/arrow/pull/36307#issuecomment-1609954314

   It's a good thing for all of our sanity!
   
   `SafeCallIntoR()` either evaluates the expression it contains once on the main R thread OR errors it can't. The `RunWithCapturedR()` wrapper launches a thread (via the IO thread pools), sets up a "listener" on the main R thread that waits for requests to evaluate R code from `SafeCallIntoR()`. It's not a perfect demo, but there's some test code that demonstrates this:
   
   https://github.com/apache/arrow/blob/8b4a548baf862c4333fab8868b416fcf30612956/r/src/safe-call-into-r-impl.cpp#L62-L76
   
   The main difference with actual code is that the call to `SafeCallIntoR()` is usually buried deep in some utility code (e.g., read bytes from an R connection by calling the base R `readBin()` function) and the top-level call usually knows nothing about this (e.g., "Read this parquet file").
   
   If `SafeCallIntoR()` is invoked from the R thread, it just evaluates the function without any of that dance. Basically, it is used everywhere we call anything from the R API (as in `#include <R.h>`) or cpp11 (because cpp11 wraps a lot of calls to R.h functions with the assumption that all of this is happening on the main thread).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org