[GitHub] [arrow] romainfrancois commented on a change in pull request #8533: ARROW-10080: [R] Call gc() and try again in MemoryPool

[GitBox <gi...@apache.org>]

romainfrancois commented on a change in pull request #8533:
URL: https://github.com/apache/arrow/pull/8533#discussion_r513265845



##########
File path: r/src/memorypool.cpp
##########
@@ -18,11 +18,55 @@
 #include "./arrow_types.h"
 #if defined(ARROW_R_WITH_ARROW)
 #include <arrow/memory_pool.h>
+#include <arrow/util/mutex.h>
+
+class GcMemoryPool : public arrow::MemoryPool {
+ public:
+  GcMemoryPool() : pool_(arrow::default_memory_pool()) {}
+
+  arrow::Status Allocate(int64_t size, uint8_t** out) override {
+    return GcAndTryAgain([&] { return pool_->Allocate(size, out); });
+  }
+
+  arrow::Status Reallocate(int64_t old_size, int64_t new_size, uint8_t** ptr) override {
+    return GcAndTryAgain([&] { return pool_->Reallocate(old_size, new_size, ptr); });
+  }
+
+  void Free(uint8_t* buffer, int64_t size) override { pool_->Free(buffer, size); }
+
+  int64_t bytes_allocated() const override { return pool_->bytes_allocated(); }
+
+  int64_t max_memory() const override { return pool_->max_memory(); }
+
+  std::string backend_name() const override { return pool_->backend_name() + "-gc"; }
+
+ private:
+  template <typename Call>
+  arrow::Status GcAndTryAgain(const Call& call) {
+    if (call().ok()) {
+      return arrow::Status::OK();
+    } else {
+      auto lock = mutex_.Lock();
+
+      // ARROW-10080: Allocation may fail spuriously since the garbage collector is lazy.
+      // Force it to run then try again in case any reusable allocations have been freed.
+      static cpp11::function gc = cpp11::package("base")["gc"];

Review comment:
       Maybe make `gc()` a member of `GcMemoryPool` instead of repeating it for each `GcAndTryAgain` ?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org