[GitHub] [arrow] romainfrancois commented on pull request #9615: ARROW-3316: [R] Multi-threaded conversion from R data.frame to Arrow table / record batch

[GitBox <gi...@apache.org>]

romainfrancois commented on pull request #9615:
URL: https://github.com/apache/arrow/pull/9615#issuecomment-789772368


   ```cpp
   bool CanExtendParallel(SEXP x, const std::shared_ptr<arrow::DataType>& type) {
     // TODO: identify when it's ok to do things in parallel
     return false;
   }
   ```
   
   turning this to `true` entirely makes everything fail dramatically. So we'll need to pay careful attention about what can and cannot be done concurrently. We might have to dial down the use of the `cpp11` package because I believe when we create a `cpp11` vector to shell a `SEXP` this uses a central resource for the protection. 
   
   It would probably be better to move the "can this be done in parallel" to a virtual method of `RConverter`, but we would need to move capture `converter` in the `task` lambda or at least figure out some way for the converter to still be alive when the task is run. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org