[GitHub] [arrow] westonpace commented on a diff in pull request #13640: ARROW-14635: [Python][C++] add O_DIRECT support to writes

[GitBox <gi...@apache.org>]

westonpace commented on code in PR #13640:
URL: https://github.com/apache/arrow/pull/13640#discussion_r924958407


##########
cpp/src/arrow/io/file.cc:
##########
@@ -378,6 +378,77 @@ Status FileOutputStream::Write(const void* data, int64_t length) {
 
 int FileOutputStream::file_descriptor() const { return impl_->fd(); }
 
+// ----------------------------------------------------------------------
+// DirectFileOutputStream, change the Open, Write and Close methods from FileOutputStream
+// Uses DirectIO for writes. Will only write out things in 4096 byte blocks. Buffers leftover bytes
+// in an internal data structure, which will be padded to 4096 bytes and flushed upon call to close.
+
+class DirectFileOutputStream::DirectFileOutputStreamImpl : public OSFile {
+ public:
+  Status Open(const std::string& path, bool append) {
+    const bool truncate = !append;
+    return OpenWritable(path, truncate, append, true /* write_only */, true);
+  }
+  Status Open(int fd) { return OpenWritable(fd); }
+};
+
+DirectFileOutputStream::DirectFileOutputStream() { 
+  uintptr_t mask = (uintptr_t)(4095);
+  uint8_t *mem = static_cast<uint8_t *>(malloc(4096 + 4095));
+  cached_data = reinterpret_cast<uint8_t *>( reinterpret_cast<uintptr_t>(mem+4095) & ~(mask));

Review Comment:
   I ran the experiments myself with 1GB of data for timing.  Oddly, on an HHD, I didn't observe much difference.  On an SSD, the difference was rather considerable (I reran the ssd tests a number of times and the difference was consistent):
   
   Method | Disk | Time (ms)
   --- | --- | ---
   direct | hhd | 41290.55257
   direct | ssd | 4641.38446
   fadvise1 | hhd | 41179.51826
   fadvise1 | ssd | 6959.85842
   
   The direct I/O results roughly match what I get for a disk benchmark on the SSD.  However, I feel the test is still cheating.  The `direct.cpp` example is assuming the memory is previously unaligned.  Can you create a version that does not assume this (e.g. memcpy into a temp buffer before writing)?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org