[GitHub] [arrow] g302ge opened a new issue, #14526: If I can cimport the cython .pxd files from pyarrow directly

[GitBox <gi...@apache.org>]

g302ge opened a new issue, #14526:
URL: https://github.com/apache/arrow/issues/14526

   Recent days, I have develop a new dataset filesystem used in my company scope. But in the python environment my colleague use the dataset api and inject the filesystem which I have wrote. Hence I have to make a new python wrapper for the new filesystem. But I am a newbie of cython I don't know if I can cimport the .pxd files from the pyarrow I have installed (built with my filesystem)
   
   example is that 
   
   ```python
   # my/path/to/filesystem.pyx
   
   # I don't know if this statement is valid ?
   from pyarrow.includes.common cimport *  
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] ljluestc commented on issue #14526: If I can cimport the cython .pxd files from pyarrow directly

["ljluestc (via GitHub)" <gi...@apache.org>]

ljluestc commented on issue #14526:
URL: https://github.com/apache/arrow/issues/14526#issuecomment-1661613715

   ```
   # my_custom_filesystem.py
   
   # Implement your custom filesystem using PyArrow interfaces here
   # For example:
   import pyarrow as pa
   
   class MyCustomFileSystem(pa.FileSystem):
       def __init__(self):
           # Initialize your custom filesystem here
           pass
   
       def open_input_file(self, path):
           # Implement your open_input_file method
           pass
   
       # ... other methods ...
   
   ```
   ```
   # my_path_to_filesystem.pyx
   
   # Import required PyArrow modules and symbols from the .pxd files
   from pyarrow.includes.common cimport *
   from my_custom_filesystem cimport MyCustomFileSystem
   
   # Your Cython code using PyArrow with your custom filesystem goes here...
   # For example:
   cdef void my_function():
       cdef FileSystem fs = MyCustomFileSystem()
       # Use the PyArrow API with your custom filesystem
       # For example:
       cdef File file = fs.open_input_file("my_file.txt")
       # ... other PyArrow operations with the custom filesystem ...
   ```
   ```
   # setup.py
   
   from distutils.core import setup
   from distutils.extension import Extension
   from Cython.Build import cythonize
   
   # Include directories for PyArrow headers if needed
   include_dirs = ["/path/to/pyarrow/headers"]
   
   # Library directories for PyArrow libraries if needed
   library_dirs = ["/path/to/pyarrow/libraries"]
   
   # Link libraries for PyArrow if needed
   libraries = ["arrow", "parquet"]
   
   ext_modules = [
       Extension("my_path_to_filesystem", ["my_path_to_filesystem.pyx"],
                 include_dirs=include_dirs,
                 library_dirs=library_dirs,
                 libraries=libraries)
   ]
   
   setup(
       name='MyCustomFilesystem',
       ext_modules=cythonize(ext_modules)
   )
   
   ```
   `python setup.py build_ext --inplace
   `


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org