You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Andrew Wong (Code Review)" <ge...@cloudera.org> on 2020/10/18 03:47:41 UTC

[kudu-CR] python: support for passing arrow data to python

Andrew Wong has uploaded a new patch set (#3) to the change originally created by Todd Lipcon. ( http://gerrit.cloudera.org:8080/15661 )

Change subject: python: support for passing arrow data to python
......................................................................

python: support for passing arrow data to python

This adds some support to pass arrow-formatted data to pyarrow. This
takes the form of Arrow's C-structs, as well as helper classes to manage
their memory. These structs are exposed in the Cython wrapper files.

This also adds Python APIs that match the existing Python APIs. Namely,
batch-level APIs that return one scan batch at a time, as well as a
to_pandas() method that materializes the entire scan result as a
DataFrame.

Change-Id: I5df4714bf5ac339d675f2a9169f8aeff06d30eed
---
M python/kudu/__init__.py
M python/kudu/client.pyx
M python/kudu/libkudu_client.pxd
M python/kudu/schema.pyx
M python/kudu/tests/test_scanner.py
M python/kudu/tests/util.py
M src/kudu/client/CMakeLists.txt
A src/kudu/client/arrow-internal.h
A src/kudu/client/arrow.h
M src/kudu/client/client-test.cc
M src/kudu/client/client.cc
M src/kudu/client/client.h
M src/kudu/client/columnar_scan_batch.cc
M src/kudu/client/columnar_scan_batch.h
M src/kudu/client/scanner-internal.cc
M src/kudu/client/scanner-internal.h
M src/kudu/client/schema.cc
M src/kudu/client/schema.h
M src/kudu/util/memory/arena.h
19 files changed, 540 insertions(+), 35 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/15661/3
-- 
To view, visit http://gerrit.cloudera.org:8080/15661
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5df4714bf5ac339d675f2a9169f8aeff06d30eed
Gerrit-Change-Number: 15661
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Wes McKinney <we...@apache.org>