[arrow] branch master updated: ARROW-2711: [Python] Fix inference from Pandas column with first empty list

[ap...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

apitrou pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
     new 141b2c3  ARROW-2711: [Python] Fix inference from Pandas column with first empty list
141b2c3 is described below

commit 141b2c39de45dc4c3a702ea82e3d9764c87d80d7
Author: Antoine Pitrou <an...@python.org>
AuthorDate: Mon Jul 23 18:52:32 2018 +0200

    ARROW-2711: [Python] Fix inference from Pandas column with first empty list
    
    Author: Antoine Pitrou <an...@python.org>
    
    Closes #2309 from pitrou/ARROW-2711-infer-empty-list and squashes the following commits:
    
    859ca93 <Antoine Pitrou> Fix test on Python 2.7
    f5549fd <Antoine Pitrou> ARROW-2711:  Fix inference from Pandas column with first empty list
---
 cpp/src/arrow/python/numpy_to_arrow.cc      | 4 ++++
 python/pyarrow/tests/test_convert_pandas.py | 8 ++++++++
 2 files changed, 12 insertions(+)

diff --git a/cpp/src/arrow/python/numpy_to_arrow.cc b/cpp/src/arrow/python/numpy_to_arrow.cc
index c0a2649..41bda76 100644
--- a/cpp/src/arrow/python/numpy_to_arrow.cc
+++ b/cpp/src/arrow/python/numpy_to_arrow.cc
@@ -1128,6 +1128,10 @@ Status NumPyConverter::ConvertObjectsInfer() {
     } else if (PyObject_IsInstance(obj, decimal_type_.obj()) == 1) {
       return ConvertDecimals();
     } else if (PyList_Check(obj)) {
+      if (PyList_Size(obj) == 0 && i < length_ - 1) {
+        // Iterate until we find a non-empty list or the enclosing sequence is empty
+        continue;
+      }
       std::shared_ptr<DataType> inferred_type;
       RETURN_NOT_OK(InferArrowType(obj, &inferred_type));
       return ConvertLists(inferred_type);
diff --git a/python/pyarrow/tests/test_convert_pandas.py b/python/pyarrow/tests/test_convert_pandas.py
index ceedf1a..e9f2e4b 100644
--- a/python/pyarrow/tests/test_convert_pandas.py
+++ b/python/pyarrow/tests/test_convert_pandas.py
@@ -1537,6 +1537,14 @@ class TestListTypes(object):
         assert arr.equals(expected)
         assert arr.type == pa.list_(pa.null())
 
+    def test_nested_list_first_empty(self):
+        # ARROW-2711
+        data = pd.Series([[], [u"a"]])
+        arr = pa.array(data)
+        expected = pa.array(list(data))
+        assert arr.equals(expected)
+        assert arr.type == pa.list_(pa.string())
+
     def test_nested_smaller_ints(self):
         # ARROW-1345, ARROW-2008, there were some type inference bugs happening
         # before