[GitHub] [arrow] lidavidm commented on a diff in pull request #13709: Arrow-16000: [C++][Python] Dataset: Added transcoding function option to CSV scanner

[GitBox <gi...@apache.org>]

lidavidm commented on code in PR #13709:
URL: https://github.com/apache/arrow/pull/13709#discussion_r929931695


##########
cpp/src/arrow/python/io.h:
##########
@@ -112,5 +112,8 @@ std::shared_ptr<::arrow::io::InputStream> MakeTransformInputStream(
     std::shared_ptr<::arrow::io::InputStream> wrapped, TransformInputStreamVTable vtable,
     PyObject* arg);
 
+std::function<Result<std::shared_ptr<io::InputStream>>(std::shared_ptr<io::InputStream>)>
+makeStreamTransformFunc(TransformInputStreamVTable vtable, PyObject* handler);

Review Comment:
   nit: UpperCamelCase?



##########
python/pyarrow/_dataset.pyx:
##########
@@ -1237,6 +1238,40 @@ cdef class CsvFileFormat(FileFormat):
         return f"<CsvFileFormat parse_options={self.parse_options}>"
 
 
+# From io.pxi
+def py_buffer(object obj):
+    """
+    Construct an Arrow buffer from a Python bytes-like or buffer-like object
+
+    Parameters
+    ----------
+    obj : object
+        the object from which the buffer should be constructed.
+    """
+    cdef shared_ptr[CBuffer] buf
+    buf = GetResultValue(PyBuffer.FromPyObject(obj))
+    return pyarrow_wrap_buffer(buf)
+
+
+# From io.pxi
+cdef void _cb_transform(transform_func, const shared_ptr[CBuffer]& src,

Review Comment:
   I _think_ you could make this `cimport`able if placed into `lib.pxd` but it's probably not a big deal to just inline it here



##########
python/pyarrow/_dataset.pyx:
##########
@@ -1237,6 +1238,40 @@ cdef class CsvFileFormat(FileFormat):
         return f"<CsvFileFormat parse_options={self.parse_options}>"
 
 
+# From io.pxi
+def py_buffer(object obj):

Review Comment:
   This should be importable as just `from pyarrow.lib import py_buffer` (you can see _plasma.pyx and _cuda.pyx do the same)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org