You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/10/14 15:20:58 UTC

[GitHub] [arrow] rdbisme opened a new issue, #14421: How to expose missing C++ API to Cython?

rdbisme opened a new issue, #14421:
URL: https://github.com/apache/arrow/issues/14421

   Hello, 
   not sure if this is 100% a question for Arrow, but I wanted to experiment a bit with Cython, `nogil` and `arrow`. So I started writing a small function that uses `GetColumnByName`, that is not exposed for what I can see in `https://github.com/apache/arrow/blob/master/python/pyarrow/includes/libarrow.pxd`. 
   
   ```
   import pyarrow as pa
   
   from libcpp.string cimport string
   from libcpp.memory cimport shared_ptr
   
   from pyarrow.lib cimport CTable, pyarrow_unwrap_table(sa_table)
   
   cdef string SCORES_COLUMN = b"SCORES_COLUMNS"
   
   cdef extern from "arrow/api.h" namespace "arrow" nogil:
       cdef cppclass CTable" arrow::Table":
           shared_ptr[CChunkedArray] GetColumnByName(const string&)
   
   def normalize_sa(sa_table: pa.Table) -> None:
       unwrapped_table: shared_ptr[CTable] = pyarrow_unwrap_table(sa_table)
   
       unwrapped_table.get().GetColumnByName(SCORES_COLUMN)
   
   ```
   
   But when compiling this, I get:
   ```
   Error compiling Cython file:
   ------------------------------------------------------------
   ...
   def normalize_sa(sa_table: pa.Table) -> None:
       unwrapped_table: shared_ptr[CTable] = pyarrow_unwrap_table(sa_table)
   
       unwrapped_table.get().GetColumnByName(SCORES_COLUMN)
                           ^
   ------------------------------------------------------------
   
   src\pkg\cutils\_normalize.pyx:35:25: Object of type 'CTable' has no attribute 'GetColumnByName'
   ```
   
   Could you please help me understand why I can't expose `GetColumnByName` or why cython can't find it?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] drin commented on issue #14421: How to expose missing C++ API to Cython?

Posted by GitBox <gi...@apache.org>.
drin commented on issue #14421:
URL: https://github.com/apache/arrow/issues/14421#issuecomment-1279341772

   also you maybe want to try importing `CChunkedArray`?
   
   I don't really see any problems otherwise though. Maybe you can try moving the python code to a `.pyx` file and see if that helps with compilation somehow? Doesn't seem like it would help, but nothing comes to mind. I'll try playing around with it later.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] drin commented on issue #14421: How to expose missing C++ API to Cython?

Posted by GitBox <gi...@apache.org>.
drin commented on issue #14421:
URL: https://github.com/apache/arrow/issues/14421#issuecomment-1279327288

   while I'm simmering on this code, I think one of your imports is malformed, specifically:
   ```python
   from pyarrow.lib cimport CTable, pyarrow_unwrap_table(sa_table) # I don't think you want to pass `sa_table` here?
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] rdbisme commented on issue #14421: How to expose missing C++ API to Cython?

Posted by GitBox <gi...@apache.org>.
rdbisme commented on issue #14421:
URL: https://github.com/apache/arrow/issues/14421#issuecomment-1279774800

   Hi @drin, thanks a lot for your help. 
   
   > while I'm simmering on this code, I think one of your imports is malformed, specifically:
   > 
   > ```python
   > from pyarrow.lib cimport CTable, pyarrow_unwrap_table # I don't think you want to pass `sa_table` here?
   > ```
   
   Yep, copy / pasting to remove unrelated stuff I left a wrong import (fixed in this post).
   
   
   
   > also you maybe want to try importing `CChunkedArray`?
   > 
   > I don't really see any problems otherwise though. Maybe you can try moving the python code to a `.pyx` file and see if that helps with compilation somehow? Doesn't seem like it would help, but nothing comes to mind. I'll try playing around with it later.
   
   I tried also with a `shared_ptr[CChunkedArray]` return value, but still I get the same error. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org