You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by we...@apache.org on 2017/05/23 21:22:25 UTC
[1/4] arrow-site git commit: Update Python documentation
Repository: arrow-site
Updated Branches:
refs/heads/asf-site 0f10eadd8 -> 820f0af64
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/searchindex.js
----------------------------------------------------------------------
diff --git a/docs/python/searchindex.js b/docs/python/searchindex.js
index d74759f..b45747b 100644
--- a/docs/python/searchindex.js
+++ b/docs/python/searchindex.js
@@ -1 +1 @@
-Search.setIndex({docnames:["api","data","development","filesystems","generated/pyarrow.Array","generated/pyarrow.ArrayValue","generated/pyarrow.BinaryArray","generated/pyarrow.BinaryValue","generated/pyarrow.BooleanArray","generated/pyarrow.BooleanValue","generated/pyarrow.Buffer","generated/pyarrow.BufferReader","generated/pyarrow.ChunkedArray","generated/pyarrow.Column","generated/pyarrow.DataType","generated/pyarrow.Date32Array","generated/pyarrow.Date32Value","generated/pyarrow.Date64Array","generated/pyarrow.Date64Value","generated/pyarrow.DecimalArray","generated/pyarrow.DecimalType","generated/pyarrow.DecimalValue","generated/pyarrow.DictionaryArray","generated/pyarrow.DictionaryType","generated/pyarrow.DoubleValue","generated/pyarrow.Field","generated/pyarrow.FileReader","generated/pyarrow.FileWriter","generated/pyarrow.FixedSizeBinaryArray","generated/pyarrow.FixedSizeBinaryType","generated/pyarrow.FixedSizeBinaryValue","generated/pyarrow.FloatValue","generated/pyarrow.Floa
tingPointArray","generated/pyarrow.InMemoryOutputStream","generated/pyarrow.Int16Array","generated/pyarrow.Int16Value","generated/pyarrow.Int32Array","generated/pyarrow.Int32Value","generated/pyarrow.Int64Array","generated/pyarrow.Int64Value","generated/pyarrow.Int8Array","generated/pyarrow.Int8Value","generated/pyarrow.IntegerArray","generated/pyarrow.ListArray","generated/pyarrow.ListValue","generated/pyarrow.MemoryMappedFile","generated/pyarrow.MemoryPool","generated/pyarrow.NA","generated/pyarrow.NAType","generated/pyarrow.NativeFile","generated/pyarrow.NullArray","generated/pyarrow.NumericArray","generated/pyarrow.PythonFile","generated/pyarrow.RecordBatch","generated/pyarrow.Scalar","generated/pyarrow.Schema","generated/pyarrow.StreamReader","generated/pyarrow.StreamWriter","generated/pyarrow.StringArray","generated/pyarrow.StringValue","generated/pyarrow.Table","generated/pyarrow.Tensor","generated/pyarrow.Time32Array","generated/pyarrow.Time32Type","generated/pyarrow.Time64A
rray","generated/pyarrow.Time64Type","generated/pyarrow.TimestampArray","generated/pyarrow.TimestampType","generated/pyarrow.TimestampValue","generated/pyarrow.UInt16Array","generated/pyarrow.UInt16Value","generated/pyarrow.UInt32Array","generated/pyarrow.UInt32Value","generated/pyarrow.UInt64Array","generated/pyarrow.UInt64Value","generated/pyarrow.UInt8Array","generated/pyarrow.UInt8Value","generated/pyarrow.array","generated/pyarrow.binary","generated/pyarrow.bool_","generated/pyarrow.create_memory_map","generated/pyarrow.date32","generated/pyarrow.date64","generated/pyarrow.decimal","generated/pyarrow.default_memory_pool","generated/pyarrow.dictionary","generated/pyarrow.field","generated/pyarrow.float16","generated/pyarrow.float32","generated/pyarrow.float64","generated/pyarrow.from_numpy_dtype","generated/pyarrow.get_record_batch_size","generated/pyarrow.get_tensor_size","generated/pyarrow.int16","generated/pyarrow.int32","generated/pyarrow.int64","generated/pyarrow.int8","gen
erated/pyarrow.jemalloc_memory_pool","generated/pyarrow.list_","generated/pyarrow.memory_map","generated/pyarrow.null","generated/pyarrow.parquet.ParquetDataset","generated/pyarrow.parquet.ParquetFile","generated/pyarrow.parquet.read_table","generated/pyarrow.parquet.write_metadata","generated/pyarrow.parquet.write_table","generated/pyarrow.read_tensor","generated/pyarrow.schema","generated/pyarrow.set_memory_pool","generated/pyarrow.string","generated/pyarrow.struct","generated/pyarrow.time32","generated/pyarrow.time64","generated/pyarrow.timestamp","generated/pyarrow.total_allocated_bytes","generated/pyarrow.uint16","generated/pyarrow.uint32","generated/pyarrow.uint64","generated/pyarrow.uint8","generated/pyarrow.write_tensor","getting_involved","index","install","ipc","jemalloc","memory","pandas","parquet"],envversion:51,filenames:["api.rst","data.rst","development.rst","filesystems.rst","generated/pyarrow.Array.rst","generated/pyarrow.ArrayValue.rst","generated/pyarrow.BinaryArr
ay.rst","generated/pyarrow.BinaryValue.rst","generated/pyarrow.BooleanArray.rst","generated/pyarrow.BooleanValue.rst","generated/pyarrow.Buffer.rst","generated/pyarrow.BufferReader.rst","generated/pyarrow.ChunkedArray.rst","generated/pyarrow.Column.rst","generated/pyarrow.DataType.rst","generated/pyarrow.Date32Array.rst","generated/pyarrow.Date32Value.rst","generated/pyarrow.Date64Array.rst","generated/pyarrow.Date64Value.rst","generated/pyarrow.DecimalArray.rst","generated/pyarrow.DecimalType.rst","generated/pyarrow.DecimalValue.rst","generated/pyarrow.DictionaryArray.rst","generated/pyarrow.DictionaryType.rst","generated/pyarrow.DoubleValue.rst","generated/pyarrow.Field.rst","generated/pyarrow.FileReader.rst","generated/pyarrow.FileWriter.rst","generated/pyarrow.FixedSizeBinaryArray.rst","generated/pyarrow.FixedSizeBinaryType.rst","generated/pyarrow.FixedSizeBinaryValue.rst","generated/pyarrow.FloatValue.rst","generated/pyarrow.FloatingPointArray.rst","generated/pyarrow.InMemoryOu
tputStream.rst","generated/pyarrow.Int16Array.rst","generated/pyarrow.Int16Value.rst","generated/pyarrow.Int32Array.rst","generated/pyarrow.Int32Value.rst","generated/pyarrow.Int64Array.rst","generated/pyarrow.Int64Value.rst","generated/pyarrow.Int8Array.rst","generated/pyarrow.Int8Value.rst","generated/pyarrow.IntegerArray.rst","generated/pyarrow.ListArray.rst","generated/pyarrow.ListValue.rst","generated/pyarrow.MemoryMappedFile.rst","generated/pyarrow.MemoryPool.rst","generated/pyarrow.NA.rst","generated/pyarrow.NAType.rst","generated/pyarrow.NativeFile.rst","generated/pyarrow.NullArray.rst","generated/pyarrow.NumericArray.rst","generated/pyarrow.PythonFile.rst","generated/pyarrow.RecordBatch.rst","generated/pyarrow.Scalar.rst","generated/pyarrow.Schema.rst","generated/pyarrow.StreamReader.rst","generated/pyarrow.StreamWriter.rst","generated/pyarrow.StringArray.rst","generated/pyarrow.StringValue.rst","generated/pyarrow.Table.rst","generated/pyarrow.Tensor.rst","generated/pyarrow
.Time32Array.rst","generated/pyarrow.Time32Type.rst","generated/pyarrow.Time64Array.rst","generated/pyarrow.Time64Type.rst","generated/pyarrow.TimestampArray.rst","generated/pyarrow.TimestampType.rst","generated/pyarrow.TimestampValue.rst","generated/pyarrow.UInt16Array.rst","generated/pyarrow.UInt16Value.rst","generated/pyarrow.UInt32Array.rst","generated/pyarrow.UInt32Value.rst","generated/pyarrow.UInt64Array.rst","generated/pyarrow.UInt64Value.rst","generated/pyarrow.UInt8Array.rst","generated/pyarrow.UInt8Value.rst","generated/pyarrow.array.rst","generated/pyarrow.binary.rst","generated/pyarrow.bool_.rst","generated/pyarrow.create_memory_map.rst","generated/pyarrow.date32.rst","generated/pyarrow.date64.rst","generated/pyarrow.decimal.rst","generated/pyarrow.default_memory_pool.rst","generated/pyarrow.dictionary.rst","generated/pyarrow.field.rst","generated/pyarrow.float16.rst","generated/pyarrow.float32.rst","generated/pyarrow.float64.rst","generated/pyarrow.from_numpy_dtype.rst
","generated/pyarrow.get_record_batch_size.rst","generated/pyarrow.get_tensor_size.rst","generated/pyarrow.int16.rst","generated/pyarrow.int32.rst","generated/pyarrow.int64.rst","generated/pyarrow.int8.rst","generated/pyarrow.jemalloc_memory_pool.rst","generated/pyarrow.list_.rst","generated/pyarrow.memory_map.rst","generated/pyarrow.null.rst","generated/pyarrow.parquet.ParquetDataset.rst","generated/pyarrow.parquet.ParquetFile.rst","generated/pyarrow.parquet.read_table.rst","generated/pyarrow.parquet.write_metadata.rst","generated/pyarrow.parquet.write_table.rst","generated/pyarrow.read_tensor.rst","generated/pyarrow.schema.rst","generated/pyarrow.set_memory_pool.rst","generated/pyarrow.string.rst","generated/pyarrow.struct.rst","generated/pyarrow.time32.rst","generated/pyarrow.time64.rst","generated/pyarrow.timestamp.rst","generated/pyarrow.total_allocated_bytes.rst","generated/pyarrow.uint16.rst","generated/pyarrow.uint32.rst","generated/pyarrow.uint64.rst","generated/pyarrow.uin
t8.rst","generated/pyarrow.write_tensor.rst","getting_involved.rst","index.rst","install.rst","ipc.rst","jemalloc.rst","memory.rst","pandas.rst","parquet.rst"],objects:{"pyarrow.Array":{__init__:[4,1,1,""]},"pyarrow.ArrayValue":{__init__:[5,1,1,""]},"pyarrow.BinaryArray":{__init__:[6,1,1,""]},"pyarrow.BinaryValue":{__init__:[7,1,1,""]},"pyarrow.BooleanArray":{__init__:[8,1,1,""]},"pyarrow.BooleanValue":{__init__:[9,1,1,""]},"pyarrow.Buffer":{__init__:[10,1,1,""]},"pyarrow.BufferReader":{__init__:[11,1,1,""]},"pyarrow.ChunkedArray":{__init__:[12,1,1,""]},"pyarrow.Column":{__init__:[13,1,1,""]},"pyarrow.DataType":{__init__:[14,1,1,""]},"pyarrow.Date32Array":{__init__:[15,1,1,""]},"pyarrow.Date32Value":{__init__:[16,1,1,""]},"pyarrow.Date64Array":{__init__:[17,1,1,""]},"pyarrow.Date64Value":{__init__:[18,1,1,""]},"pyarrow.DecimalArray":{__init__:[19,1,1,""]},"pyarrow.DecimalType":{__init__:[20,1,1,""]},"pyarrow.DecimalValue":{__init__:[21,1,1,""]},"pyarrow.DictionaryArray":{__init__:[2
2,1,1,""]},"pyarrow.DictionaryType":{__init__:[23,1,1,""]},"pyarrow.DoubleValue":{__init__:[24,1,1,""]},"pyarrow.Field":{__init__:[25,1,1,""]},"pyarrow.FileReader":{__init__:[26,1,1,""]},"pyarrow.FileWriter":{__init__:[27,1,1,""]},"pyarrow.FixedSizeBinaryArray":{__init__:[28,1,1,""]},"pyarrow.FixedSizeBinaryType":{__init__:[29,1,1,""]},"pyarrow.FixedSizeBinaryValue":{__init__:[30,1,1,""]},"pyarrow.FloatValue":{__init__:[31,1,1,""]},"pyarrow.FloatingPointArray":{__init__:[32,1,1,""]},"pyarrow.InMemoryOutputStream":{__init__:[33,1,1,""]},"pyarrow.Int16Array":{__init__:[34,1,1,""]},"pyarrow.Int16Value":{__init__:[35,1,1,""]},"pyarrow.Int32Array":{__init__:[36,1,1,""]},"pyarrow.Int32Value":{__init__:[37,1,1,""]},"pyarrow.Int64Array":{__init__:[38,1,1,""]},"pyarrow.Int64Value":{__init__:[39,1,1,""]},"pyarrow.Int8Array":{__init__:[40,1,1,""]},"pyarrow.Int8Value":{__init__:[41,1,1,""]},"pyarrow.IntegerArray":{__init__:[42,1,1,""]},"pyarrow.ListArray":{__init__:[43,1,1,""]},"pyarrow.ListVal
ue":{__init__:[44,1,1,""]},"pyarrow.MemoryMappedFile":{__init__:[45,1,1,""]},"pyarrow.MemoryPool":{__init__:[46,1,1,""]},"pyarrow.NAType":{__init__:[48,1,1,""]},"pyarrow.NativeFile":{__init__:[49,1,1,""]},"pyarrow.NullArray":{__init__:[50,1,1,""]},"pyarrow.NumericArray":{__init__:[51,1,1,""]},"pyarrow.PythonFile":{__init__:[52,1,1,""]},"pyarrow.RecordBatch":{__init__:[53,1,1,""]},"pyarrow.Scalar":{__init__:[54,1,1,""]},"pyarrow.Schema":{__init__:[55,1,1,""]},"pyarrow.StreamReader":{__init__:[56,1,1,""]},"pyarrow.StreamWriter":{__init__:[57,1,1,""]},"pyarrow.StringArray":{__init__:[58,1,1,""]},"pyarrow.StringValue":{__init__:[59,1,1,""]},"pyarrow.Table":{__init__:[60,1,1,""]},"pyarrow.Tensor":{__init__:[61,1,1,""]},"pyarrow.Time32Array":{__init__:[62,1,1,""]},"pyarrow.Time32Type":{__init__:[63,1,1,""]},"pyarrow.Time64Array":{__init__:[64,1,1,""]},"pyarrow.Time64Type":{__init__:[65,1,1,""]},"pyarrow.TimestampArray":{__init__:[66,1,1,""]},"pyarrow.TimestampType":{__init__:[67,1,1,""]},
"pyarrow.TimestampValue":{__init__:[68,1,1,""]},"pyarrow.UInt16Array":{__init__:[69,1,1,""]},"pyarrow.UInt16Value":{__init__:[70,1,1,""]},"pyarrow.UInt32Array":{__init__:[71,1,1,""]},"pyarrow.UInt32Value":{__init__:[72,1,1,""]},"pyarrow.UInt64Array":{__init__:[73,1,1,""]},"pyarrow.UInt64Value":{__init__:[74,1,1,""]},"pyarrow.UInt8Array":{__init__:[75,1,1,""]},"pyarrow.UInt8Value":{__init__:[76,1,1,""]},"pyarrow.parquet":{ParquetDataset:[101,0,1,""],ParquetFile:[102,0,1,""],read_table:[103,3,1,""],write_metadata:[104,3,1,""],write_table:[105,3,1,""]},"pyarrow.parquet.ParquetDataset":{__init__:[101,1,1,""]},"pyarrow.parquet.ParquetFile":{__init__:[102,1,1,""]},pyarrow:{"null":[100,3,1,""],Array:[4,0,1,""],ArrayValue:[5,0,1,""],BinaryArray:[6,0,1,""],BinaryValue:[7,0,1,""],BooleanArray:[8,0,1,""],BooleanValue:[9,0,1,""],Buffer:[10,0,1,""],BufferReader:[11,0,1,""],ChunkedArray:[12,0,1,""],Column:[13,0,1,""],DataType:[14,0,1,""],Date32Array:[15,0,1,""],Date32Value:[16,0,1,""],Date64Array
:[17,0,1,""],Date64Value:[18,0,1,""],DecimalArray:[19,0,1,""],DecimalType:[20,0,1,""],DecimalValue:[21,0,1,""],DictionaryArray:[22,0,1,""],DictionaryType:[23,0,1,""],DoubleValue:[24,0,1,""],Field:[25,0,1,""],FileReader:[26,0,1,""],FileWriter:[27,0,1,""],FixedSizeBinaryArray:[28,0,1,""],FixedSizeBinaryType:[29,0,1,""],FixedSizeBinaryValue:[30,0,1,""],FloatValue:[31,0,1,""],FloatingPointArray:[32,0,1,""],InMemoryOutputStream:[33,0,1,""],Int16Array:[34,0,1,""],Int16Value:[35,0,1,""],Int32Array:[36,0,1,""],Int32Value:[37,0,1,""],Int64Array:[38,0,1,""],Int64Value:[39,0,1,""],Int8Array:[40,0,1,""],Int8Value:[41,0,1,""],IntegerArray:[42,0,1,""],ListArray:[43,0,1,""],ListValue:[44,0,1,""],MemoryMappedFile:[45,0,1,""],MemoryPool:[46,0,1,""],NA:[47,2,1,""],NAType:[48,0,1,""],NativeFile:[49,0,1,""],NullArray:[50,0,1,""],NumericArray:[51,0,1,""],PythonFile:[52,0,1,""],RecordBatch:[53,0,1,""],Scalar:[54,0,1,""],Schema:[55,0,1,""],StreamReader:[56,0,1,""],StreamWriter:[57,0,1,""],StringArray:[58,
0,1,""],StringValue:[59,0,1,""],Table:[60,0,1,""],Tensor:[61,0,1,""],Time32Array:[62,0,1,""],Time32Type:[63,0,1,""],Time64Array:[64,0,1,""],Time64Type:[65,0,1,""],TimestampArray:[66,0,1,""],TimestampType:[67,0,1,""],TimestampValue:[68,0,1,""],UInt16Array:[69,0,1,""],UInt16Value:[70,0,1,""],UInt32Array:[71,0,1,""],UInt32Value:[72,0,1,""],UInt64Array:[73,0,1,""],UInt64Value:[74,0,1,""],UInt8Array:[75,0,1,""],UInt8Value:[76,0,1,""],array:[77,3,1,""],binary:[78,3,1,""],bool_:[79,3,1,""],create_memory_map:[80,3,1,""],date32:[81,3,1,""],date64:[82,3,1,""],decimal:[83,3,1,""],default_memory_pool:[84,3,1,""],dictionary:[85,3,1,""],field:[86,3,1,""],float16:[87,3,1,""],float32:[88,3,1,""],float64:[89,3,1,""],from_numpy_dtype:[90,3,1,""],get_record_batch_size:[91,3,1,""],get_tensor_size:[92,3,1,""],int16:[93,3,1,""],int32:[94,3,1,""],int64:[95,3,1,""],int8:[96,3,1,""],jemalloc_memory_pool:[97,3,1,""],list_:[98,3,1,""],memory_map:[99,3,1,""],read_tensor:[106,3,1,""],schema:[107,3,1,""],set_mem
ory_pool:[108,3,1,""],string:[109,3,1,""],struct:[110,3,1,""],time32:[111,3,1,""],time64:[112,3,1,""],timestamp:[113,3,1,""],total_allocated_bytes:[114,3,1,""],uint16:[115,3,1,""],uint32:[116,3,1,""],uint64:[117,3,1,""],uint8:[118,3,1,""],write_tensor:[119,3,1,""]}},objnames:{"0":["py","class","Python class"],"1":["py","method","Python method"],"2":["py","data","Python data"],"3":["py","function","Python function"]},objtypes:{"0":"py:class","1":"py:method","2":"py:data","3":"py:function"},terms:{"0x2ab45d3b2950":[],"0x2ab45db7fdc8":[],"0x2ab45de6af80":[],"0x2ab45de85068":[],"0x2ab45dec31f0":[],"0x2ae789605098":[],"0x2ae78961a0e8":[],"0x2ae78961a908":[],"0x2aead9398a50":[],"0x2aead9b67d08":[],"0x2aead9e8e730":[],"0x2aead9ea9c38":[],"0x2aead9f41340":[],"0x2afb4f6cb230":[],"0x2afb4feb27c8":[],"0x2afb4ff00b90":[],"0x2b01dc5bad08":[],"0x2b01dc5c0cc8":[],"0x2b029559cc30":[],"0x2b02a3902a78":[],"0x2b02a5231888":[],"0x2b02a5513df8":[],"0x2b02a5552d88":[],"0x2b07aa92b7c8":[],"0x2b07aa964290"
:[],"0x2b07aa97cc38":[],"0x2b07aabe9ae8":[],"0x2b07aaeaa110":[],"0x2b0bcda1ef48":125,"0x2b0bcda54ed0":1,"0x2b0bcda54f00":1,"0x2b0bcda58ab0":125,"0x2b0bcdd11318":1,"0x2b0bcdd19588":1,"0x2b0bcdd29818":1,"0x2b0bcdd29908":1,"0x2b0bcdd36ea8":1,"0x2b0bcdd36f98":1,"0x2b0bcdd4b3b8":1,"0x2b0bcddc06f8":125,"0x2b0bcdde94c8":127,"0x2b0bcde04378":125,"0x2b0bcde7c538":125,"0x2b0bce0dd9a8":127,"0x2b0d086a8d68":[],"0x2b0d086e2ef8":[],"0x2b0d086f6908":[],"0x2b22e6a400b0":[],"0x2b22e6a42588":[],"0x2b22e6a5b848":[],"0x2b22e6d82298":[],"0x2b22e6d82688":[],"0x2b2350e31a08":[],"0x2b2350e66b10":[],"0x2b2350e66b40":[],"0x2b2350e7fca8":[],"0x2b2351104cc8":[],"0x2b23511253b8":[],"0x2b2351125408":[],"0x2b2351137b88":[],"0x2b2351137f48":[],"0x2b235113ee08":[],"0x2b235114b150":[],"0x2b2351159618":[],"0x2b2351164848":[],"0x2b23511967d8":[],"0x2b235142ec78":[],"0x2b2351430e48":[],"0x2b2f897d2228":[],"0x2b2f897d2ef8":[],"0x2b2f897de048":[],"0x2b328ee8c3b8":[],"0x2b328ee8c818":[],"0x2b328eea74f8":[],"0x2b373d998250
":[],"0x2b373e1b8588":[],"0x2b373e1d0a40":[],"0x2b45dca1e1b0":[],"0x2b45dca1e4b0":[],"0x2b45dcccb548":[],"0x2b45dcce9188":[],"0x2b45dcce9278":[],"0x2b45dccf1048":[],"0x2b4e6e8f1408":[],"0x2b4e6eb83db8":[],"0x2b58d9984d08":[],"0x2b58d99c18b8":[],"0x2b58da5e7510":[],"0x2b58da751068":[],"0x2b58da7516f8":[],"0x2b5bbfce5510":[],"0x2b5bbfce5b40":[],"0x2b5bbfcfec48":[],"0x2b5bbff85548":[],"0x2b5bbff9e0e8":[],"0x2b5bbff9e548":[],"0x2b5bbffc13b8":[],"0x2b5bbffc1688":[],"0x2b5bbffd4ae8":[],"0x2b772e911bd0":[],"0x2b772f124ac8":[],"0x2b772f45c768":[],"0x2b772f4f9e30":[],"0x2b772f509c70":[],"0x2b787b3cc3b8":[],"0x2b787b3db138":[],"0x2b787b3e6408":[],"0x2b80df428c78":[],"0x2b8d3ab842c8":[],"0x2b8d3ab99f48":[],"0x2b8d3abb3958":[],"0x2b92b1b0c4a8":[],"0x2b92b1b27098":[],"0x2b92b1b270e8":[],"0x2b94e1524510":[],"0x2b94e1ceeb88":[],"0x2b94e1d56b90":[],"0x2ba1c3986d68":[],"0x2ba1c3997948":[],"2306c71fef75":[],"33451edb9706":[],"33e8af46152c":[],"3866a5b0a55c":[],"5199a1c92584":[],"52a1af38d8a2":[],"56e
a00246f64":[],"67203deacf03":[],"74905e723e96":[],"7b10db728cbc":[],"81c851451745":[],"81d6c8f19eeb":[],"94a852cad77a":[],"957f74a84022":[],"abstract":125,"boolean":[1,86,101,127],"byte":[11,26,86,102,103,119,123,125],"case":120,"class":[1,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,101,102,121,125,127],"default":[3,26,77,78,86,99,101,102,103,104,105,124,125,127],"export":[2,3,125],"final":2,"float":126,"function":[1,121,125,127],"import":[1,3,123,124,125,126,127],"int":[26,78,80,83,103,105,119,126],"new":[1,123,124,125],"null":[1,126],"return":[1,77,78,80,86,91,92,97,99,103,106,107,119,123],"true":[1,86,101,105,123,126,127],"while":[124,125,126,127],For:[1,102,103,121,123,125],One:126,The:[1,27,57,104,105,123,124,125,126,127],There:125,These:[1,125,127],Use:[101,102],Using:[121,124,125],With:126,__cinit__:[],__init__:[
4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,101,102],_arrai:1,_error:[],_io:125,_memori:125,_parquet:127,_tabl:1,abcdefghijklmnopqrstuvwxyz:125,abl:[2,125],about:2,abov:1,absolut:127,acceler:121,accept:127,access:[121,125],accompani:1,accur:[4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,48,49,50,51,52,53,54,55,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76],achiev:126,acquir:125,activ:[2,120],actual:124,adapt:127,add:121,addit:[1,121,125],adopt:127,after:[2,125,127],agnost:1,algorithm:121,align:124,all:[1,2,101,123,125,126,127],alloc:[77,97,106,123,124,125],allow:124,along:121,alreadi:[124,126],also:[1,3,121,123,124,125,126,127],altern:101,although:[124,126],alwai:126,amount:125,analogu:1,analysi:127,analyt:121,ani:[2,106,123,125,127],apach
:[1,2,120,125],api:[1,121,123,124,125],appear:1,applic:1,apr:2,apt:2,arbitrari:[123,126],archiv:120,argument:127,arr:1,arrai:[12,60,85,121,123,124,125,126],arrow:[1,2,11,26,27,56,57,60,120,122,123,124,125,127],arrow_build_toolchain:2,arrow_build_typ:2,arrow_hom:2,arrow_libhdfs_dir:3,arrowioerror:[],artifact:122,assign:1,assist:125,assum:[101,122],atom:1,attribut:[1,4,6,8,10,12,13,15,17,19,20,22,25,26,28,29,32,34,36,38,40,42,43,44,50,51,53,55,56,58,60,61,62,63,64,65,66,67,69,71,73,75,102],attributeerror:[],audienc:120,avail:[122,125],ba2a6e2ee1e7:[],back:[12,126,127],bar:[1,123,127],base:[3,97,124,125],basi:[105,127],batch2:1,batch:[26,53,91,121,123],baz:[1,123,127],becaus:[1,2,123],been:[123,125,127],begin:123,behav:125,being:127,below:1,best:127,better:127,between:[123,125],big:121,bin:[2,3],binari:[1,26,27,56,57,122,123],bind:[3,121,127],bit:[1,2],bool:[1,86,105,123,126,127],boost:2,both:126,brotli:[2,127],buf:[123,125],buffer:[1,11,102,103,121,123],bufferread:[102,103,123,125],bu
ild:127,build_ext:2,built:[125,127],bundl:127,byte_arrai:127,bytes_alloc:125,bytes_written:119,bytesio:125,c1bc967efc27:[],call:[12,13,53,60,126],can:[1,2,3,97,103,123,124,125,126,127],cannot:99,canon:121,categor:[1,85,126],categori:1,chang:99,chat:120,cheapli:125,check:[2,101,123],check_statu:[],child:125,children:1,choic:[],choos:125,chunk:[1,12],chunk_siz:[],chunkedarrai:1,circumst:125,clang:2,classpath:3,clean:125,client:3,clone:2,close:[123,127],cmake:2,code:127,codec:105,coerc:1,coercibl:86,collect:[1,2,60,107,125],column:[1,53,103,105,123,126,127],columnar:[1,121,127],com:2,come:[1,3],common:[121,127],commonli:127,commun:121,compat:[101,121],compil:2,complet:[101,123],compos:1,compress:[105,121],comput:103,concaten:[1,127],concept:125,concurr:127,conda:[121,127],conda_env:2,conda_prefix:2,config:2,configur:2,connect:3,consid:[1,125],consist:[1,101,126,127],constitut:127,construct:[107,125],constructor:[12,13,25,53,60,121],consum:126,contain:[1,3,123,126,127],content:103,conti
gu:1,contribut:120,conveni:[],convers:125,convert:[1,11,90,125,126],convienc:126,copi:[1,11,106,123,124,125],correspond:1,could:123,count:1,cpp:[2,127],creat:[1,2,27,77,80,86,123,125,127],created_bi:127,ctest:2,current:[2,77,106,119,122,126],custom:[121,125],cxx:2,cython:2,darrow_build_test:2,darrow_python:2,dat:125,data:[25,26,27,57,77,120,121,123,124,125,126,127],datafram:[121,127],dataset:[1,101,103,121],dataset_nam:127,datatyp:[1,77,83,85,86,90,98],date:[1,126],datetim:126,datetime64:126,db281ae27c00:[],dcmake_build_typ:2,dcmake_install_prefix:2,dealloc:125,decim:1,decod:127,def:[],defin:[1,123,125,127],depend:[2,127],describ:1,descriptor:125,deseri:125,design:[121,126],dest:119,destin:126,detail:[101,121,127],dev:[2,120],develop:[120,121,122,127],df_new:126,dict:[86,105],dict_arrai:1,dictionari:[105,126,127],dictionaryarrai:1,did:2,differ:[1,121,123,125],directli:[1,12,13,25,53,60],directori:[2,101,103,127],discov:127,discuss:[3,125],disk:[101,121,127],distinct:1,distribut:3,di
vid:101,dll:2,document:[1,121],doe:[106,123,125],doesn:124,done:126,doubl:[126,127],dparquet_build_benchmark:2,dparquet_build_execut:2,dparquet_build_test:2,dparquet_zlib_vendor:2,drill:127,driver:3,drwxrwxr:2,dtype:[1,90],due:127,dure:126,each:[1,101,105,125,127],effici:[1,125],either:[26,27,56,57,105,124,127],element:13,elsewher:101,email:120,embed:[1,26],enabl:[1,120,124,127],encapsul:101,encod:[1,85,105,121],end:[26,123,125],engin:[],entir:123,entri:126,env:2,environ:3,equal:[1,13,53,60,78,123,126],equival:126,especi:127,even:120,examin:1,exampl:[1,125,127],example2:[125,127],exe:2,exist:[102,122,124],expect:125,expens:127,explain:[1,125],explicit:[3,124],explicitli:124,expos:1,extend:124,extens:127,factor:1,factori:[1,121],fail:[],fall:127,fals:[1,101,123,126,127],fast:121,faster:[124,127],featur:[121,122,125,127],field:[107,110,121],file:[1,26,27,56,57,80,99,101,102,103,104,105,106,119,121],file_obj:125,filemetadata:[101,103,127],filenam:[],fileread:123,filesystem:[101,121],fi
lesytem:125,filewrit:123,finer:121,first:[2,123],fit:1,fix:[1,78,123,126],fixed_size_binari:1,flag:[],flat:[121,126],flatbuff:2,float32:126,float64:[1,126],follow:[120,123,125],foo:[1,123,125,127],footer_offset:26,forg:[2,122],format:[1,26,27,56,57,103,104,105,120,121,123],format_vers:127,found:101,fraction:[],free:[124,125],freed:125,friend:125,from:[1,2,3,11,26,77,102,103,104,106,107,121,123,125,126,127],from_:[53,60],from_arrai:[1,123],from_batch:1,from_panda:[126,127],from_pandas_seri:126,from_pylist:124,frombuff:125,full:[1,126],futur:125,garbag:125,gcc:2,gener:[105,127],get:[2,121],get_batch:123,get_read:[],get_result:[123,125],gil:125,git:2,github:2,given:[1,2,80],glob:3,global:124,grain:121,greater:78,group:[101,127],grow:127,guid:2,gzip:127,hadoop:[121,127],hadoop_hom:3,handl:[1,120],happi:120,has:[1,3,123,124,125,126,127],have:[1,120,123,125,127],hdf:121,hdfs3:3,hdfsclient:3,hdfsfile:125,hello:125,help:[1,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,28,29,30
,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,48,49,50,51,52,53,54,55,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76],here:[1,121,123,126],hierarch:121,high:127,higher:[2,125,127],hive:127,hold:1,home:2,host:3,hous:121,how:127,http:2,idea:120,ideal:127,immut:1,impala:127,implement:[121,125,127],improv:1,includ:[1,91,92,125,127],increas:[],increment:125,incub:127,index:1,index_typ:85,indic:[1,80,127],indirectli:120,individu:[101,127],infer:[77,127],inifil:2,initi:[4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,48,49,50,51,52,53,54,55,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76],inmemoryoutputstream:[123,125],inplac:2,input:[1,121,123],instal:[2,3,121,127],instanc:[1,77,86,125,127],instead:[53,60],int32:1,int32_field:1,int64:[1,123],int64arrai:1,integ:1,integr:120,intend:125,interact:125,interest:125,interfac:[102,121,124,126],intern:[120,125],interoper:120,interprocess:121,interrel:1,intro
duc:125,invers:126,involv:121,ioerror:[],ipc:[1,121],ipython:[],issu:126,item:[1,2],iter:107,its:[119,124],itself:120,jar:3,java:3,java_hom:3,jemalloc:[2,97,121],jemalloc_memory_pool:[124,125],jemalloc_pool:124,jira:120,jni:3,join:120,kei:86,kerb_ticket:3,kernel:124,kind:125,know:[2,123],known:1,kwarg:105,lab:3,languag:[1,121],larg:127,larger:26,last:[],later:1,latest:122,layer:[121,127],layout:127,ld_library_path:3,leaf:121,learn:[120,127],least:[],len:[1,123],length:[1,53,60,78,123,125,126],let:[1,2,123,125,127],level:[60,125,127],lib:[2,3],libhdf:3,libhdfs3:3,librari:[2,3,121,125],lifetim:125,like:[1,3,26,27,56,57,77,78,122,123,124,125,127],linear:[124,126],link:3,linux:122,list:[1,101,103,105,107,120,127],list_:1,load:3,local:101,locat:[3,103],logic:1,look:127,maco:122,magnitud:124,mai:[1,3,124,125,127],mail:120,main:[121,126],major:125,make:[1,2,123,124,125],malloc:125,manag:125,mani:[125,127],manipul:121,manylinux1:122,map:[80,99,106,121,123,124,127],mark:126,mask:126,maximum:
105,mean:125,memori:[12,77,80,97,99,106,121,123,124,126,127],memory_map:125,memory_pool:[77,124],memorymappedfil:[80,99,125],memorypool:[77,108,121,125],memoryview:125,messag:[1,121,125],metadata:[25,86,91,92,101,102,103,104,121,123,127],method:[4,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,48,49,50,51,52,53,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,101,102,123,125],might:126,millisecond:126,mimic:125,miniconda:2,minim:[],minimum:125,miss:126,mkdir:2,mmap:[80,99,125],mode:[45,99],model:121,modern:2,modul:[],month:127,more:[1,12,121,125,126,127],most:[120,126,127],much:127,multipl:[1,101,121],multithread:121,must:[1,2,3,86,123,127],my_pool:125,my_schema:1,name:[1,13,25,60,86,101,103,123,126,127],nameerror:[],nan:[1,127],nanosecond:126,nativ:[3,121,127],nativefil:[26,27,56,57,102,103,104,105,106,119,123,125,127],need:[1,2,122,125,127],nest:[1,126,127],nested_arr:1,non:[],none:[1,26,77,86
,101,102,103,105,113,123,125,126,127],normal:[1,125],note:25,noth:101,now:[2,120,123],nthread:[103,127],null_count:1,nullabl:[25,86,126],num_chunk:1,num_column:[1,123,127],num_record_batch:123,num_row:[1,123,127],num_row_group:127,number:[1,103,105,119,123,127],numpi:[1,2,90,127],obj:11,object:[1,11,26,27,56,57,77,80,102,103,106,119,123,124,125,126,127],obtain:[2,101,121],occupi:1,off:2,offset:26,often:[1,3,125],old:124,omit:2,one:[1,12,53,60,127],ones:1,onli:[1,103,104,105,121,122,123,124,125,126],open:[80,99,125,127],oper:[106,123,124,125],option:[1,2,3,25,77,78,125,126,127],org:120,orient:1,origin:[123,127],osfil:125,other:[1,3,120,121,123,124,125,126,127],otherwis:122,our:[2,120],out:[1,123,125,127],output:121,over:124,own:125,packag:2,pad:[91,92],page:[121,124,127],paio:[],panda:[1,2,121,127],parallel:[103,127],paramet:[11,26,27,56,57,77,78,80,86,99,101,102,103,104,105,106,107,119,126],parent:125,parq:127,parquet:[2,121,125,126],parquet_build_toolchain:2,parquet_fil:127,parquet
_hom:2,parquetdataset:127,parquetfil:127,parquetfilemetadata:102,parquetread:[],parquetschema:127,parquetwrit:127,part:[1,122],particular:[1,123,125,127],particularli:[1,127],partit:[101,121],pass:[1,2,77,97,101,102,103,123,124,125,127],path:[2,26,27,56,57,80,99,101,127],path_or_path:101,payload:123,peopl:120,per:[1,105,127],perform:[1,127],permit:125,persist:[],pf2:127,physic:1,piec:[1,101],pip:[121,127],pivot:3,plain:127,plan:120,platform:2,plu:1,pluggi:2,point:123,pointer:125,pool:[77,108,121,124],popd:2,port:3,posit:[106,119],posix:124,possibl:[101,125,126],potenti:[],practic:127,precis:[83,125],prefer:125,present:127,preserv:125,previou:124,previous:[],primari:[120,125],primit:1,print:[1,125],process:123,produc:[125,127],project:[120,127],properti:[],protocol:125,provid:[121,124,125,126,127],pull:122,pushd:2,put:2,pyarrow:[1,2,3,121,123,124],pybyt:78,pypi:122,pytest:2,python35:2,python:[1,11,26,27,56,57,77,102,103,122,123,125,127],pythonfil:125,pythonfileinterfac:[102,103],pyth
onpath:2,pyx:[],queri:[],random:[121,125],rang:[123,125,127],rapidjson:2,rather:[1,3,102],read:[26,101,102,103,106,120,121],read_buff:125,read_multiple_fil:[],read_row_group:127,read_tabl:127,readabl:[26,56,102],reader:[11,56,102,123,125],realloc:124,receiv:1,recent:122,recommend:2,record:[26,121,123],recordbatch:[1,91,123],refer:[121,125],referenc:[1,125],region:124,regular:125,relationship:125,releas:2,relev:1,reli:3,repo:2,repositori:2,repres:[25,126],represent:121,request:[124,125],requir:[1,103,123,125],resolut:126,respect:127,result:127,retriev:122,right:120,rle:127,root:3,rootdir:2,routin:126,row:[53,101,105,127],row_group_s:105,rowgroup:105,runtim:3,safe:125,same:[1,101,123,126],save:1,scalar:[1,121],scale:83,schema:[27,57,101,104,121,123,127],sdk:3,second:2,section:[1,3,123,125],see:[1,2,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,48,49,50,51,52,53,54,55,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,
74,75,76,102,103,121,122,127],seek:[123,125],select:1,self:[4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,48,49,50,51,52,53,54,55,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76],semant:125,send:[120,123],sent:123,separ:103,sequenc:[1,77,123],seri:[1,121],serial:[91,92,121,125],serialized_s:127,session:2,set:[2,3,77,121,124,126,127],set_memory_pool:[97,124],setuptool:2,sever:125,share:[1,120,121,127],should:[2,105,127],side:126,signatur:[4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,48,49,50,51,52,53,54,55,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76],significantli:127,similar:[1,125,126],similarli:127,simpl:[1,127],simpli:[2,85,124],sinc:[3,123],singl:[1,101,102,103,105,121,124],sink:[27,57,123],site:2,six:2,size:[1,78,80,91,92,99,123,124,125],skip:2,slack:120,slice:[1,125],slightli:1,small:[1,123],smaller:127,snappi:[2,1
05,127],snippet:[],socket:[1,123],some:[1,2,3,26,105,123,125,126,127],someplac:2,somewher:3,sourc:[2,26,27,56,57,97,101,102,103,104,105,106,121,123,126,127],spark:127,special:1,specif:[1,120],specifi:[105,124,127],speed:127,split_row_group:101,sssssssssssssss:2,standard:[125,127],start:[2,121,123],still:[124,126],storag:[1,127],store:[],str:[26,27,56,57,101,102,103,105,126],strategi:124,stream:[1,56,57,121,125],streamread:123,streamwrit:123,string:[1,80,86,99,103,104,105,123,125,126,127],stringarrai:1,struct:1,structur:[1,121,126,127],studio:2,subdirectori:101,subpool:125,subscrib:120,subset:127,sudo:2,suffici:2,suppli:126,support:[1,45,106,121,122,123,124,125,126],sure:[2,123],system:[120,121,124,125,126,127],tabl:[103,105,121,124,126,127],table2:127,tag:122,take:[],target:2,tcmalloc:124,temp:[],tensor:[92,106,119,121],test_arrai:2,test_convert_builtin:2,test_convert_panda:2,test_feath:2,test_hdf:2,test_io:2,test_ipc:2,test_jemalloc:2,test_parquet:2,test_scalar:2,test_schema:2,test
_tabl:2,test_tensor:2,than:[1,3,78,102,125,126,127],thee:1,thei:[1,125,127],them:[1,126,127],thi:[1,2,3,12,13,25,26,53,60,106,121,122,123,124,125,126,127],thirdparti:[2,3],those:127,though:1,thought:1,threadsaf:103,three:127,thrift:2,through:[1,120,124],throughput:127,thu:[123,124,126,127],ticket_cache_path:3,time:[1,3,126],timestamp:[1,126],timestamps_to_m:126,to_panda:[1,126,127],to_pybyt:125,todo:1,toggl:127,too:127,tool:[1,125],toolchain:2,top:60,total:[2,91,92,119],total_allocated_byt:125,touch:124,traceback:[],track:125,transmit:1,transport:127,tricki:2,two:[123,125,127],type:[2,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,48,49,50,51,52,53,54,55,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,85,86,121,123,125],typeerror:[],typic:[],ubuntu:2,uncompress:127,underli:103,unicod:126,union:1,unit:126,unit_str:[111,112,113],unmodifi:126,usag:121,use:[1,3,25,53,60,105,120,123,124,125,127],use_dict
ionari:[105,127],used:[1,123,125,126,127],useful:123,user:[1,2,3],uses:[3,124,125,127],using:[1,2,3,122,125,126,127],utf8:[109,127],util:[],valid:101,validate_schema:101,valu:[1,86,121,126],value_typ:98,variabl:[1,2,3,78],variou:[1,126],vector:[13,126],veri:[26,123],version:[2,104,105,122,127],via:12,view:125,visual:2,wai:[1,125,127],want:[],well:127,wesm:2,when:[1,123,125,127],where:[1,104,105,125,127],whether:127,which:[1,97,123,125,127],whole:127,width:78,win64:2,window:121,within:127,without:[1,125],work:[2,126],worst:127,wrangl:1,wrap:125,write:[104,105,119,121],write_batch:123,write_t:127,writeabl:[27,57,80,123],writer:[27,57,123,125,127],written:[27,57,119,127],x86_64:[],xcode:2,year:127,yield:127,you:[1,2,3,120,122,124,125,126,127],your:[2,3,120,125],zero:[11,106,123,125],zip:2,zlib:2},titles:["API Reference","In-Memory Data Model","Development","Filesystem Interfaces","pyarrow.Array","pyarrow.ArrayValue","pyarrow.BinaryArray","pyarrow.BinaryValue","pyarrow.BooleanArray","py
arrow.BooleanValue","pyarrow.Buffer","pyarrow.BufferReader","pyarrow.ChunkedArray","pyarrow.Column","pyarrow.DataType","pyarrow.Date32Array","pyarrow.Date32Value","pyarrow.Date64Array","pyarrow.Date64Value","pyarrow.DecimalArray","pyarrow.DecimalType","pyarrow.DecimalValue","pyarrow.DictionaryArray","pyarrow.DictionaryType","pyarrow.DoubleValue","pyarrow.Field","pyarrow.FileReader","pyarrow.FileWriter","pyarrow.FixedSizeBinaryArray","pyarrow.FixedSizeBinaryType","pyarrow.FixedSizeBinaryValue","pyarrow.FloatValue","pyarrow.FloatingPointArray","pyarrow.InMemoryOutputStream","pyarrow.Int16Array","pyarrow.Int16Value","pyarrow.Int32Array","pyarrow.Int32Value","pyarrow.Int64Array","pyarrow.Int64Value","pyarrow.Int8Array","pyarrow.Int8Value","pyarrow.IntegerArray","pyarrow.ListArray","pyarrow.ListValue","pyarrow.MemoryMappedFile","pyarrow.MemoryPool","pyarrow.NA","pyarrow.NAType","pyarrow.NativeFile","pyarrow.NullArray","pyarrow.NumericArray","pyarrow.PythonFile","pyarrow.RecordBatch","pya
rrow.Scalar","pyarrow.Schema","pyarrow.StreamReader","pyarrow.StreamWriter","pyarrow.StringArray","pyarrow.StringValue","pyarrow.Table","pyarrow.Tensor","pyarrow.Time32Array","pyarrow.Time32Type","pyarrow.Time64Array","pyarrow.Time64Type","pyarrow.TimestampArray","pyarrow.TimestampType","pyarrow.TimestampValue","pyarrow.UInt16Array","pyarrow.UInt16Value","pyarrow.UInt32Array","pyarrow.UInt32Value","pyarrow.UInt64Array","pyarrow.UInt64Value","pyarrow.UInt8Array","pyarrow.UInt8Value","pyarrow.array","pyarrow.binary","pyarrow.bool_","pyarrow.create_memory_map","pyarrow.date32","pyarrow.date64","pyarrow.decimal","pyarrow.default_memory_pool","pyarrow.dictionary","pyarrow.field","pyarrow.float16","pyarrow.float32","pyarrow.float64","pyarrow.from_numpy_dtype","pyarrow.get_record_batch_size","pyarrow.get_tensor_size","pyarrow.int16","pyarrow.int32","pyarrow.int64","pyarrow.int8","pyarrow.jemalloc_memory_pool","pyarrow.list_","pyarrow.memory_map","pyarrow.null","pyarrow.parquet.ParquetDatas
et","pyarrow.parquet.ParquetFile","pyarrow.parquet.read_table","pyarrow.parquet.write_metadata","pyarrow.parquet.write_table","pyarrow.read_tensor","pyarrow.schema","pyarrow.set_memory_pool","pyarrow.string","pyarrow.struct","pyarrow.time32","pyarrow.time64","pyarrow.timestamp","pyarrow.total_allocated_bytes","pyarrow.uint16","pyarrow.uint32","pyarrow.uint64","pyarrow.uint8","pyarrow.write_tensor","Getting Involved","Apache Arrow (Python)","Install PyArrow","IPC: Fast Streaming and Serialization","jemalloc MemoryPool","Memory and IO Interfaces","Using PyArrow with pandas","Reading and Writing the Apache Parquet Format"],titleterms:{"class":0,"function":0,"null":100,Using:126,access:123,apach:[0,121,127],api:0,arrai:[0,1,4,77],arrayvalu:5,arrow:[121,126],batch:[0,1],binari:78,binaryarrai:6,binaryvalu:7,bool_:79,booleanarrai:8,booleanvalu:9,buffer:[10,125],bufferread:11,build:2,chunkedarrai:12,column:13,commun:0,compat:127,compress:127,conda:[2,122],constructor:0,convers:126,create_me
mory_map:80,custom:1,data:1,datafram:126,dataset:127,datatyp:14,date32:81,date32arrai:15,date32valu:16,date64:82,date64arrai:17,date64valu:18,decim:83,decimalarrai:19,decimaltyp:20,decimalvalu:21,default_memory_pool:84,develop:2,dictionari:[1,85],dictionaryarrai:22,dictionarytyp:23,differ:126,disk:125,doublevalu:24,encod:127,environ:2,factori:0,fast:123,field:[1,25,86],file:[3,123,125,127],fileread:26,filesystem:[3,125],filewrit:27,finer:127,fixedsizebinaryarrai:28,fixedsizebinarytyp:29,fixedsizebinaryvalu:30,float16:87,float32:88,float64:89,floatingpointarrai:32,floatvalu:31,format:127,from:122,from_numpy_dtyp:90,get:120,get_record_batch_s:91,get_tensor_s:92,grain:127,hadoop:[3,125],hdf:3,inmemoryoutputstream:33,input:0,instal:122,int16:93,int16arrai:34,int16valu:35,int32:94,int32arrai:36,int32valu:37,int64:95,int64arrai:38,int64valu:39,int8:96,int8arrai:40,int8valu:41,integerarrai:42,interfac:[3,125],interprocess:0,involv:120,ipc:123,jemalloc:124,jemalloc_memory_pool:97,linux:2,li
st_:98,listarrai:43,listvalu:44,maco:2,map:125,memori:[0,1,125],memory_map:99,memorymappedfil:45,memorypool:[46,124],messag:0,metadata:1,model:1,multipl:127,multithread:127,nativ:125,nativefil:49,natyp:48,nullarrai:50,numericarrai:51,obtain:127,output:0,panda:126,parquet:[0,101,102,103,104,105,127],parquetdataset:101,parquetfil:102,partit:127,pip:122,pool:[0,125],pyarrow:[4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,122,125,126,127],python:[2,121],pythonfil:52,random:123,read:[123,125,127],read_tabl:103,read_tensor:106,record:[0,1],recordbatch:53,refer:0,requir:2,run:2,scalar:[0,54],schema:[0,1,55,107],seri:126,serial:123,set_memory_pool:108,setup:2,share:0,singl:127,sour
c:122,stream:123,streamread:56,streamwrit:57,string:109,stringarrai:58,stringvalu:59,struct:110,support:127,system:[2,3],tabl:[0,1,60],tensor:[0,61],test:2,time32:111,time32arrai:62,time32typ:63,time64:112,time64arrai:64,time64typ:65,timestamp:113,timestamparrai:66,timestamptyp:67,timestampvalu:68,total_allocated_byt:114,type:[0,1,126],uint16:115,uint16arrai:69,uint16valu:70,uint32:116,uint32arrai:71,uint32valu:72,uint64:117,uint64arrai:73,uint64valu:74,uint8:118,uint8arrai:75,uint8valu:76,unit:2,valu:0,window:2,write:[123,125,127],write_metadata:104,write_t:105,write_tensor:119}})
\ No newline at end of file
+Search.setIndex({docnames:["api","data","development","filesystems","generated/pyarrow.Array","generated/pyarrow.ArrayValue","generated/pyarrow.BinaryArray","generated/pyarrow.BinaryValue","generated/pyarrow.BooleanArray","generated/pyarrow.BooleanValue","generated/pyarrow.Buffer","generated/pyarrow.BufferOutputStream","generated/pyarrow.BufferReader","generated/pyarrow.ChunkedArray","generated/pyarrow.Column","generated/pyarrow.DataType","generated/pyarrow.Date32Array","generated/pyarrow.Date32Value","generated/pyarrow.Date64Array","generated/pyarrow.Date64Value","generated/pyarrow.DecimalArray","generated/pyarrow.DecimalType","generated/pyarrow.DecimalValue","generated/pyarrow.DictionaryArray","generated/pyarrow.DictionaryType","generated/pyarrow.DoubleValue","generated/pyarrow.Field","generated/pyarrow.FixedSizeBinaryArray","generated/pyarrow.FixedSizeBinaryType","generated/pyarrow.FixedSizeBinaryValue","generated/pyarrow.FloatValue","generated/pyarrow.FloatingPointArray","genera
ted/pyarrow.Int16Array","generated/pyarrow.Int16Value","generated/pyarrow.Int32Array","generated/pyarrow.Int32Value","generated/pyarrow.Int64Array","generated/pyarrow.Int64Value","generated/pyarrow.Int8Array","generated/pyarrow.Int8Value","generated/pyarrow.IntegerArray","generated/pyarrow.ListArray","generated/pyarrow.ListValue","generated/pyarrow.MemoryMappedFile","generated/pyarrow.MemoryPool","generated/pyarrow.NA","generated/pyarrow.NAType","generated/pyarrow.NativeFile","generated/pyarrow.NullArray","generated/pyarrow.NumericArray","generated/pyarrow.PythonFile","generated/pyarrow.RecordBatch","generated/pyarrow.RecordBatchFileReader","generated/pyarrow.RecordBatchFileWriter","generated/pyarrow.RecordBatchStreamReader","generated/pyarrow.RecordBatchStreamWriter","generated/pyarrow.Scalar","generated/pyarrow.Schema","generated/pyarrow.StringArray","generated/pyarrow.StringValue","generated/pyarrow.Table","generated/pyarrow.Tensor","generated/pyarrow.Time32Array","generated/pyar
row.Time32Type","generated/pyarrow.Time64Array","generated/pyarrow.Time64Type","generated/pyarrow.TimestampArray","generated/pyarrow.TimestampType","generated/pyarrow.TimestampValue","generated/pyarrow.UInt16Array","generated/pyarrow.UInt16Value","generated/pyarrow.UInt32Array","generated/pyarrow.UInt32Value","generated/pyarrow.UInt64Array","generated/pyarrow.UInt64Value","generated/pyarrow.UInt8Array","generated/pyarrow.UInt8Value","generated/pyarrow.array","generated/pyarrow.binary","generated/pyarrow.bool_","generated/pyarrow.create_memory_map","generated/pyarrow.date32","generated/pyarrow.date64","generated/pyarrow.decimal","generated/pyarrow.default_memory_pool","generated/pyarrow.dictionary","generated/pyarrow.field","generated/pyarrow.float16","generated/pyarrow.float32","generated/pyarrow.float64","generated/pyarrow.from_numpy_dtype","generated/pyarrow.get_record_batch_size","generated/pyarrow.get_tensor_size","generated/pyarrow.int16","generated/pyarrow.int32","generated/py
arrow.int64","generated/pyarrow.int8","generated/pyarrow.jemalloc_memory_pool","generated/pyarrow.list_","generated/pyarrow.memory_map","generated/pyarrow.null","generated/pyarrow.open_file","generated/pyarrow.open_stream","generated/pyarrow.parquet.ParquetDataset","generated/pyarrow.parquet.ParquetFile","generated/pyarrow.parquet.read_table","generated/pyarrow.parquet.write_metadata","generated/pyarrow.parquet.write_table","generated/pyarrow.read_tensor","generated/pyarrow.schema","generated/pyarrow.set_memory_pool","generated/pyarrow.string","generated/pyarrow.struct","generated/pyarrow.time32","generated/pyarrow.time64","generated/pyarrow.timestamp","generated/pyarrow.total_allocated_bytes","generated/pyarrow.uint16","generated/pyarrow.uint32","generated/pyarrow.uint64","generated/pyarrow.uint8","generated/pyarrow.write_tensor","getting_involved","index","install","ipc","jemalloc","memory","pandas","parquet"],envversion:51,filenames:["api.rst","data.rst","development.rst","filesy
stems.rst","generated/pyarrow.Array.rst","generated/pyarrow.ArrayValue.rst","generated/pyarrow.BinaryArray.rst","generated/pyarrow.BinaryValue.rst","generated/pyarrow.BooleanArray.rst","generated/pyarrow.BooleanValue.rst","generated/pyarrow.Buffer.rst","generated/pyarrow.BufferOutputStream.rst","generated/pyarrow.BufferReader.rst","generated/pyarrow.ChunkedArray.rst","generated/pyarrow.Column.rst","generated/pyarrow.DataType.rst","generated/pyarrow.Date32Array.rst","generated/pyarrow.Date32Value.rst","generated/pyarrow.Date64Array.rst","generated/pyarrow.Date64Value.rst","generated/pyarrow.DecimalArray.rst","generated/pyarrow.DecimalType.rst","generated/pyarrow.DecimalValue.rst","generated/pyarrow.DictionaryArray.rst","generated/pyarrow.DictionaryType.rst","generated/pyarrow.DoubleValue.rst","generated/pyarrow.Field.rst","generated/pyarrow.FixedSizeBinaryArray.rst","generated/pyarrow.FixedSizeBinaryType.rst","generated/pyarrow.FixedSizeBinaryValue.rst","generated/pyarrow.FloatValue.
rst","generated/pyarrow.FloatingPointArray.rst","generated/pyarrow.Int16Array.rst","generated/pyarrow.Int16Value.rst","generated/pyarrow.Int32Array.rst","generated/pyarrow.Int32Value.rst","generated/pyarrow.Int64Array.rst","generated/pyarrow.Int64Value.rst","generated/pyarrow.Int8Array.rst","generated/pyarrow.Int8Value.rst","generated/pyarrow.IntegerArray.rst","generated/pyarrow.ListArray.rst","generated/pyarrow.ListValue.rst","generated/pyarrow.MemoryMappedFile.rst","generated/pyarrow.MemoryPool.rst","generated/pyarrow.NA.rst","generated/pyarrow.NAType.rst","generated/pyarrow.NativeFile.rst","generated/pyarrow.NullArray.rst","generated/pyarrow.NumericArray.rst","generated/pyarrow.PythonFile.rst","generated/pyarrow.RecordBatch.rst","generated/pyarrow.RecordBatchFileReader.rst","generated/pyarrow.RecordBatchFileWriter.rst","generated/pyarrow.RecordBatchStreamReader.rst","generated/pyarrow.RecordBatchStreamWriter.rst","generated/pyarrow.Scalar.rst","generated/pyarrow.Schema.rst","gene
rated/pyarrow.StringArray.rst","generated/pyarrow.StringValue.rst","generated/pyarrow.Table.rst","generated/pyarrow.Tensor.rst","generated/pyarrow.Time32Array.rst","generated/pyarrow.Time32Type.rst","generated/pyarrow.Time64Array.rst","generated/pyarrow.Time64Type.rst","generated/pyarrow.TimestampArray.rst","generated/pyarrow.TimestampType.rst","generated/pyarrow.TimestampValue.rst","generated/pyarrow.UInt16Array.rst","generated/pyarrow.UInt16Value.rst","generated/pyarrow.UInt32Array.rst","generated/pyarrow.UInt32Value.rst","generated/pyarrow.UInt64Array.rst","generated/pyarrow.UInt64Value.rst","generated/pyarrow.UInt8Array.rst","generated/pyarrow.UInt8Value.rst","generated/pyarrow.array.rst","generated/pyarrow.binary.rst","generated/pyarrow.bool_.rst","generated/pyarrow.create_memory_map.rst","generated/pyarrow.date32.rst","generated/pyarrow.date64.rst","generated/pyarrow.decimal.rst","generated/pyarrow.default_memory_pool.rst","generated/pyarrow.dictionary.rst","generated/pyarrow.
field.rst","generated/pyarrow.float16.rst","generated/pyarrow.float32.rst","generated/pyarrow.float64.rst","generated/pyarrow.from_numpy_dtype.rst","generated/pyarrow.get_record_batch_size.rst","generated/pyarrow.get_tensor_size.rst","generated/pyarrow.int16.rst","generated/pyarrow.int32.rst","generated/pyarrow.int64.rst","generated/pyarrow.int8.rst","generated/pyarrow.jemalloc_memory_pool.rst","generated/pyarrow.list_.rst","generated/pyarrow.memory_map.rst","generated/pyarrow.null.rst","generated/pyarrow.open_file.rst","generated/pyarrow.open_stream.rst","generated/pyarrow.parquet.ParquetDataset.rst","generated/pyarrow.parquet.ParquetFile.rst","generated/pyarrow.parquet.read_table.rst","generated/pyarrow.parquet.write_metadata.rst","generated/pyarrow.parquet.write_table.rst","generated/pyarrow.read_tensor.rst","generated/pyarrow.schema.rst","generated/pyarrow.set_memory_pool.rst","generated/pyarrow.string.rst","generated/pyarrow.struct.rst","generated/pyarrow.time32.rst","generated
/pyarrow.time64.rst","generated/pyarrow.timestamp.rst","generated/pyarrow.total_allocated_bytes.rst","generated/pyarrow.uint16.rst","generated/pyarrow.uint32.rst","generated/pyarrow.uint64.rst","generated/pyarrow.uint8.rst","generated/pyarrow.write_tensor.rst","getting_involved.rst","index.rst","install.rst","ipc.rst","jemalloc.rst","memory.rst","pandas.rst","parquet.rst"],objects:{"pyarrow.Array":{__init__:[4,1,1,""]},"pyarrow.ArrayValue":{__init__:[5,1,1,""]},"pyarrow.BinaryArray":{__init__:[6,1,1,""]},"pyarrow.BinaryValue":{__init__:[7,1,1,""]},"pyarrow.BooleanArray":{__init__:[8,1,1,""]},"pyarrow.BooleanValue":{__init__:[9,1,1,""]},"pyarrow.Buffer":{__init__:[10,1,1,""]},"pyarrow.BufferOutputStream":{__init__:[11,1,1,""]},"pyarrow.BufferReader":{__init__:[12,1,1,""]},"pyarrow.ChunkedArray":{__init__:[13,1,1,""]},"pyarrow.Column":{__init__:[14,1,1,""]},"pyarrow.DataType":{__init__:[15,1,1,""]},"pyarrow.Date32Array":{__init__:[16,1,1,""]},"pyarrow.Date32Value":{__init__:[17,1,1,""
]},"pyarrow.Date64Array":{__init__:[18,1,1,""]},"pyarrow.Date64Value":{__init__:[19,1,1,""]},"pyarrow.DecimalArray":{__init__:[20,1,1,""]},"pyarrow.DecimalType":{__init__:[21,1,1,""]},"pyarrow.DecimalValue":{__init__:[22,1,1,""]},"pyarrow.DictionaryArray":{__init__:[23,1,1,""]},"pyarrow.DictionaryType":{__init__:[24,1,1,""]},"pyarrow.DoubleValue":{__init__:[25,1,1,""]},"pyarrow.Field":{__init__:[26,1,1,""]},"pyarrow.FixedSizeBinaryArray":{__init__:[27,1,1,""]},"pyarrow.FixedSizeBinaryType":{__init__:[28,1,1,""]},"pyarrow.FixedSizeBinaryValue":{__init__:[29,1,1,""]},"pyarrow.FloatValue":{__init__:[30,1,1,""]},"pyarrow.FloatingPointArray":{__init__:[31,1,1,""]},"pyarrow.Int16Array":{__init__:[32,1,1,""]},"pyarrow.Int16Value":{__init__:[33,1,1,""]},"pyarrow.Int32Array":{__init__:[34,1,1,""]},"pyarrow.Int32Value":{__init__:[35,1,1,""]},"pyarrow.Int64Array":{__init__:[36,1,1,""]},"pyarrow.Int64Value":{__init__:[37,1,1,""]},"pyarrow.Int8Array":{__init__:[38,1,1,""]},"pyarrow.Int8Value":{_
_init__:[39,1,1,""]},"pyarrow.IntegerArray":{__init__:[40,1,1,""]},"pyarrow.ListArray":{__init__:[41,1,1,""]},"pyarrow.ListValue":{__init__:[42,1,1,""]},"pyarrow.MemoryMappedFile":{__init__:[43,1,1,""]},"pyarrow.MemoryPool":{__init__:[44,1,1,""]},"pyarrow.NAType":{__init__:[46,1,1,""]},"pyarrow.NativeFile":{__init__:[47,1,1,""]},"pyarrow.NullArray":{__init__:[48,1,1,""]},"pyarrow.NumericArray":{__init__:[49,1,1,""]},"pyarrow.PythonFile":{__init__:[50,1,1,""]},"pyarrow.RecordBatch":{__init__:[51,1,1,""]},"pyarrow.RecordBatchFileReader":{__init__:[52,1,1,""]},"pyarrow.RecordBatchFileWriter":{__init__:[53,1,1,""]},"pyarrow.RecordBatchStreamReader":{__init__:[54,1,1,""]},"pyarrow.RecordBatchStreamWriter":{__init__:[55,1,1,""]},"pyarrow.Scalar":{__init__:[56,1,1,""]},"pyarrow.Schema":{__init__:[57,1,1,""]},"pyarrow.StringArray":{__init__:[58,1,1,""]},"pyarrow.StringValue":{__init__:[59,1,1,""]},"pyarrow.Table":{__init__:[60,1,1,""]},"pyarrow.Tensor":{__init__:[61,1,1,""]},"pyarrow.Time32
Array":{__init__:[62,1,1,""]},"pyarrow.Time32Type":{__init__:[63,1,1,""]},"pyarrow.Time64Array":{__init__:[64,1,1,""]},"pyarrow.Time64Type":{__init__:[65,1,1,""]},"pyarrow.TimestampArray":{__init__:[66,1,1,""]},"pyarrow.TimestampType":{__init__:[67,1,1,""]},"pyarrow.TimestampValue":{__init__:[68,1,1,""]},"pyarrow.UInt16Array":{__init__:[69,1,1,""]},"pyarrow.UInt16Value":{__init__:[70,1,1,""]},"pyarrow.UInt32Array":{__init__:[71,1,1,""]},"pyarrow.UInt32Value":{__init__:[72,1,1,""]},"pyarrow.UInt64Array":{__init__:[73,1,1,""]},"pyarrow.UInt64Value":{__init__:[74,1,1,""]},"pyarrow.UInt8Array":{__init__:[75,1,1,""]},"pyarrow.UInt8Value":{__init__:[76,1,1,""]},"pyarrow.parquet":{ParquetDataset:[103,0,1,""],ParquetFile:[104,0,1,""],read_table:[105,3,1,""],write_metadata:[106,3,1,""],write_table:[107,3,1,""]},"pyarrow.parquet.ParquetDataset":{__init__:[103,1,1,""]},"pyarrow.parquet.ParquetFile":{__init__:[104,1,1,""]},pyarrow:{"null":[100,3,1,""],Array:[4,0,1,""],ArrayValue:[5,0,1,""],Bina
ryArray:[6,0,1,""],BinaryValue:[7,0,1,""],BooleanArray:[8,0,1,""],BooleanValue:[9,0,1,""],Buffer:[10,0,1,""],BufferOutputStream:[11,0,1,""],BufferReader:[12,0,1,""],ChunkedArray:[13,0,1,""],Column:[14,0,1,""],DataType:[15,0,1,""],Date32Array:[16,0,1,""],Date32Value:[17,0,1,""],Date64Array:[18,0,1,""],Date64Value:[19,0,1,""],DecimalArray:[20,0,1,""],DecimalType:[21,0,1,""],DecimalValue:[22,0,1,""],DictionaryArray:[23,0,1,""],DictionaryType:[24,0,1,""],DoubleValue:[25,0,1,""],Field:[26,0,1,""],FixedSizeBinaryArray:[27,0,1,""],FixedSizeBinaryType:[28,0,1,""],FixedSizeBinaryValue:[29,0,1,""],FloatValue:[30,0,1,""],FloatingPointArray:[31,0,1,""],Int16Array:[32,0,1,""],Int16Value:[33,0,1,""],Int32Array:[34,0,1,""],Int32Value:[35,0,1,""],Int64Array:[36,0,1,""],Int64Value:[37,0,1,""],Int8Array:[38,0,1,""],Int8Value:[39,0,1,""],IntegerArray:[40,0,1,""],ListArray:[41,0,1,""],ListValue:[42,0,1,""],MemoryMappedFile:[43,0,1,""],MemoryPool:[44,0,1,""],NA:[45,2,1,""],NAType:[46,0,1,""],NativeFile:
[47,0,1,""],NullArray:[48,0,1,""],NumericArray:[49,0,1,""],PythonFile:[50,0,1,""],RecordBatch:[51,0,1,""],RecordBatchFileReader:[52,0,1,""],RecordBatchFileWriter:[53,0,1,""],RecordBatchStreamReader:[54,0,1,""],RecordBatchStreamWriter:[55,0,1,""],Scalar:[56,0,1,""],Schema:[57,0,1,""],StringArray:[58,0,1,""],StringValue:[59,0,1,""],Table:[60,0,1,""],Tensor:[61,0,1,""],Time32Array:[62,0,1,""],Time32Type:[63,0,1,""],Time64Array:[64,0,1,""],Time64Type:[65,0,1,""],TimestampArray:[66,0,1,""],TimestampType:[67,0,1,""],TimestampValue:[68,0,1,""],UInt16Array:[69,0,1,""],UInt16Value:[70,0,1,""],UInt32Array:[71,0,1,""],UInt32Value:[72,0,1,""],UInt64Array:[73,0,1,""],UInt64Value:[74,0,1,""],UInt8Array:[75,0,1,""],UInt8Value:[76,0,1,""],array:[77,3,1,""],binary:[78,3,1,""],bool_:[79,3,1,""],create_memory_map:[80,3,1,""],date32:[81,3,1,""],date64:[82,3,1,""],decimal:[83,3,1,""],default_memory_pool:[84,3,1,""],dictionary:[85,3,1,""],field:[86,3,1,""],float16:[87,3,1,""],float32:[88,3,1,""],float64:
[89,3,1,""],from_numpy_dtype:[90,3,1,""],get_record_batch_size:[91,3,1,""],get_tensor_size:[92,3,1,""],int16:[93,3,1,""],int32:[94,3,1,""],int64:[95,3,1,""],int8:[96,3,1,""],jemalloc_memory_pool:[97,3,1,""],list_:[98,3,1,""],memory_map:[99,3,1,""],open_file:[101,3,1,""],open_stream:[102,3,1,""],read_tensor:[108,3,1,""],schema:[109,3,1,""],set_memory_pool:[110,3,1,""],string:[111,3,1,""],struct:[112,3,1,""],time32:[113,3,1,""],time64:[114,3,1,""],timestamp:[115,3,1,""],total_allocated_bytes:[116,3,1,""],uint16:[117,3,1,""],uint32:[118,3,1,""],uint64:[119,3,1,""],uint8:[120,3,1,""],write_tensor:[121,3,1,""]}},objnames:{"0":["py","class","Python class"],"1":["py","method","Python method"],"2":["py","data","Python data"],"3":["py","function","Python function"]},objtypes:{"0":"py:class","1":"py:method","2":"py:data","3":"py:function"},terms:{"0x7fce62e29670":127,"0x7fce6395c4c8":127,"0x7fce8cb28cc8":129,"0x7fce8ed5e1f0":127,"0x7fce8ed80260":127,"0x7fce8edb7ea0":127,"0x7fce8ede44c8":129,"
0x7fce8f9e3c00":1,"0x7fce8f9e3e10":1,"0x7fce90480768":1,"0x7fce90483278":1,"0x7fce90491408":1,"0x7fce904914f8":1,"0x7fce904a6638":1,"0x7fce904b9138":1,"0x7fce904b9818":1,"abstract":127,"boolean":[1,86,103,129],"byte":[12,52,86,101,102,104,105,121,125,127],"case":122,"class":[1,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,103,104,123,127,129],"default":[3,52,77,78,86,99,101,102,103,104,105,106,107,126,127,129],"export":[2,3,127],"final":2,"float":128,"function":[1,123,125,127,129],"import":[1,3,125,126,127,128,129],"int":[52,78,80,83,101,102,105,107,121,128],"new":[1,125,126,127],"null":[1,128,129],"return":[1,77,78,80,86,91,92,97,99,101,102,105,108,109,121,125],"true":[1,2,86,103,107,125,128,129],"while":[126,127,128,129],For:[1,104,105,123,125,127],One:128,The:[1,53,55,106,107,125,126,127,128,129],Then:2,There:127,These
:[1,127,129],Use:[103,104],Using:[123,126,127],With:128,__index_level_0__:129,__init__:[4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,103,104],_parquet:129,abcdefghijklmnopqrstuvwxyz:127,abl:[2,127],about:2,abov:1,absolut:129,acceler:123,accept:129,access:[123,127],accompani:1,accur:[4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76],achiev:128,acquir:127,activ:[2,122],actual:126,adapt:129,add:123,addit:[1,123,127],adopt:129,after:[2,127,129],agnost:1,algorithm:123,align:126,all:[1,2,103,125,127,128,129],alloc:[77,97,108,125,126,127],allow:126,along:123,alreadi:[126,128],also:[1,3,123,125,126,127,128,129],altern:103,although:[126,128],alwai:128,amount:127,analogu:1,analysi:129
,analyt:123,ani:[2,108,125,127,129],apach:[1,2,122,127],api:[1,123,125,126,127],appear:1,applic:1,apr:2,apt:2,arbitrari:[125,128],archiv:122,argument:129,arr:1,arrai:[13,60,85,123,125,126,127,128],arrow:[1,2,12,52,53,54,55,60,101,102,122,124,125,126,127,129],arrow_build_toolchain:2,arrow_build_typ:2,arrow_hom:2,arrow_libhdfs_dir:3,artifact:124,assign:1,assist:127,assum:[103,124],atom:1,attribut:[1,4,6,8,10,13,14,16,18,20,21,23,26,27,28,31,32,34,36,38,40,41,42,48,49,51,52,54,57,58,60,61,62,63,64,65,66,67,69,71,73,75,104],audienc:122,avail:[124,127],back:[13,128,129],bar:[1,125,129],base:[3,97,126,127],basi:[107,129],batch2:1,batch:[51,52,91,123,125],baz:[1,125,129],becaus:[1,2,125],been:[125,127,129],begin:125,behav:127,being:129,below:1,best:129,better:129,between:[125,127],big:123,bin:[2,3],binari:[1,52,53,54,55,124,125],bind:[3,123,129],bit:[1,2],bool:[1,86,107,125,128,129],boost:2,bootstrap:2,both:128,brotli:[2,129],buf:[125,127],buffer:[1,12,104,105,123,125],bufferoutputstream:[
125,127],bufferread:[104,105,125,127],build:129,build_ext:2,built:[127,129],bundl:129,byte_arrai:129,bytes_alloc:127,bytes_written:121,bytesio:127,call:[13,14,51,60,128],can:[1,2,3,97,105,125,126,127,128,129],cannot:99,canon:123,categor:[1,85,128],categori:1,chang:99,chat:122,cheapli:127,check:[2,103,125],child:127,children:1,choos:127,chunk:[1,13],chunkedarrai:1,circumst:127,clang:2,classpath:3,clean:127,client:3,clone:2,close:[125,129],cmake:2,code:129,codec:107,coerc:1,coercibl:86,collect:[1,2,60,109,127],column:[1,51,105,107,125,128,129],columnar:[1,123,129],com:2,come:[1,3],command:2,common:[123,129],commonli:129,commun:123,compat:[103,123],compil:2,complet:[103,125],compos:1,compress:[107,123],comput:105,concaten:[1,129],concept:127,concurr:129,conda:[123,129],conda_env:2,conda_prefix:2,config:2,configur:2,connect:3,consid:[1,127],consist:[1,103,128,129],constitut:129,construct:[109,127],constructor:[13,14,26,51,60,123],consum:128,contain:[1,3,125,128,129],content:105,contigu:
1,contribut:122,conveni:125,convers:127,convert:[1,12,90,127,128],convienc:128,copi:[1,12,108,125,126,127],core:2,correspond:1,could:125,count:1,cpp:[2,129],creat:[1,2,53,77,80,86,101,102,125,127,129],created_bi:129,ctest:2,current:[2,77,108,121,124,128],custom:[123,127],cxx:2,cython:2,darrow_build_test:2,darrow_python:2,dat:127,data:[26,52,53,55,77,101,102,122,123,125,126,127,128,129],datafram:[123,129],dataset:[1,103,105,123],dataset_nam:129,datatyp:[1,77,83,85,86,90,98],date:[1,128],datetim:128,datetime64:128,dcmake_build_typ:2,dcmake_install_prefix:2,dealloc:127,decim:1,decod:129,defin:[1,125,127,129],depend:[2,129],describ:1,descriptor:127,deseri:127,design:[123,128],dest:121,destin:128,detail:[103,123,129],dev:[2,122],develop:[122,123,124,129],df_new:128,dict:[86,107],dict_arrai:1,dictionari:[107,128,129],dictionaryarrai:1,did:2,differ:[1,123,125,127],directli:[1,13,14,26,51,60],directori:[2,103,105,129],discov:129,discuss:[3,127],disk:[103,123,129],distinct:1,distribut:3,divi
d:103,dll:2,document:[1,123],doe:[108,125,127],doesn:126,done:128,doubl:[128,129],dparquet_build_benchmark:2,dparquet_build_execut:2,dparquet_build_test:2,dparquet_zlib_vendor:2,drill:129,driver:3,drwxrwxr:2,dtype:[1,90],due:129,dure:128,each:[1,103,107,127,129],effici:[1,127],either:[52,53,54,55,101,102,107,126,129],element:14,elsewher:103,email:122,embed:[1,52,101,102],enabl:[1,122,126,129],encapsul:103,encod:[1,85,107,123],end:[52,101,102,125,127],entir:125,entri:128,env:2,environ:3,equal:[1,14,51,60,78,125,128],equival:128,especi:129,even:122,examin:1,exampl:[1,127,129],example2:[127,129],exe:2,exist:[104,124,126],expect:127,expens:129,explain:[1,127],explicit:[3,126],explicitli:126,expos:1,extend:126,extens:129,factor:1,factori:[1,123],fall:129,fals:[1,103,125,128,129],fast:123,faster:[126,129],featur:[123,124,127,129],field:[109,112,123],file:[1,52,53,54,55,80,99,101,102,103,104,105,106,107,108,121,123],file_obj:127,filemetadata:[103,105,129],filesystem:[103,123],filesytem:127
,finer:123,first:[2,125],fit:1,fix:[1,78,125,128],fixed_size_binari:1,flat:[123,128],flatbuff:2,float32:128,float64:[1,128,129],follow:[122,125,127],foo:[1,125,127,129],footer_offset:[52,101,102],forg:[2,124],format:[1,52,53,54,55,101,102,105,106,107,122,123,125],format_vers:129,found:103,free:[126,127],freed:127,fresh:2,friend:127,from:[1,2,3,12,52,77,104,105,106,108,109,123,125,127,128,129],from_:[51,60],from_arrai:[1,125],from_batch:1,from_panda:[128,129],from_pandas_seri:128,from_pylist:126,frombuff:127,full:[1,128],futur:127,garbag:127,gcc:2,gener:[107,129],get:[2,123],get_batch:125,get_result:[125,127],gil:127,git:2,github:2,given:[1,2,80],glob:3,global:126,grain:123,greater:78,group:[103,129],grow:129,guid:2,gzip:129,hadoop:[123,129],hadoop_hom:3,handl:[1,122],happi:122,has:[1,3,125,126,127,128,129],have:[1,122,125,127,129],hdf:123,hdfs3:3,hdfsclient:3,hdfsfile:127,hello:127,help:[1,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,3
7,38,39,40,41,42,43,44,46,47,48,49,50,51,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76],here:[1,123,125,128],hierarch:123,high:129,higher:[2,127,129],hive:129,hold:1,home:2,host:3,hous:123,housekeep:2,how:129,http:2,idea:122,ideal:129,immut:1,impala:129,implement:[123,127,129],improv:1,includ:[1,2,91,92,127,129],increment:127,incub:129,index:1,index_column:129,index_typ:85,indic:[1,80,129],indirectli:122,individu:[103,129],infer:[77,129],inifil:2,initi:[4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76],inplac:2,input:[1,123,125],instal:[2,3,123,129],instanc:[1,77,86,127,129],instead:[51,60],instruct:2,int32:1,int32_field:1,int64:[1,125,129],int64arrai:1,integ:1,integr:122,intend:127,interact:127,interest:127,interfac:[104,123,126,128],intern:[122,127],interoper:122,interprocess:123,interrel:1,introduc:127,invers:128,inv
olv:123,ipc:[1,123],issu:128,item:[1,2],iter:109,its:[121,126],itself:122,jar:3,java:3,java_hom:3,jemalloc:[2,97,123],jemalloc_memory_pool:[126,127],jemalloc_pool:126,jira:122,jni:3,join:122,kei:86,kerb_ticket:3,kernel:126,kind:127,know:[2,125],known:1,kwarg:107,lab:3,languag:[1,123],larg:129,larger:[52,101,102],later:1,latest:124,layer:[123,129],layout:129,ld_library_path:3,leaf:123,learn:[122,129],len:[1,125],length:[1,51,60,78,125,127,128],let:[1,2,125,127,129],level:[60,127,129],lib:[1,2,3,127],libhdf:3,libhdfs3:3,librari:[2,3,123,127],lifetim:127,like:[1,3,52,53,54,55,77,78,101,102,124,125,126,127,129],linear:[126,128],link:3,linux:124,list:[1,103,105,107,109,122,129],list_:1,load:3,local:103,locat:[3,105],logic:1,look:129,maco:124,magnitud:126,mai:[1,3,126,127,129],mail:122,main:[123,128],major:127,make:[1,2,125,126,127],malloc:127,manag:127,mani:[127,129],manipul:123,manylinux1:124,map:[80,99,108,123,125,126,129],mark:128,mask:128,maximum:107,mean:127,memori:[13,77,80,97,99,1
08,123,125,126,128,129],memory_map:127,memory_pool:[77,126],memorymappedfil:[80,99,127],memorypool:[77,110,123,127],memoryview:127,messag:[1,123,127],metadata:[26,86,91,92,103,104,105,106,123,125,129],method:[4,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,52,53,54,55,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,103,104,125,127],might:128,millisecond:128,mimic:127,miniconda:2,minimum:127,miss:128,mkdir:2,mmap:[80,99,127],mode:[43,99],model:123,modern:2,month:129,more:[1,13,123,127,128,129],most:[122,128,129],much:129,multipl:[1,103,123],multithread:123,must:[1,2,3,86,125,129],my_pool:127,my_schema:1,name:[1,14,26,60,86,103,105,125,128,129],nan:[1,129],nanosecond:128,nativ:[3,123,129],nativefil:[52,53,54,55,101,102,104,105,106,107,108,121,125,127,129],need:[1,2,124,127,129],nest:[1,128,129],nested_arr:1,none:[1,52,77,86,101,102,103,104,105,107,115,125,127,128,129],normal:[1,127],note:2
6,noth:103,now:[2,122,125],nthread:[105,129],null_count:1,nullabl:[26,86,128],num_chunk:1,num_column:[1,125,129],num_record_batch:125,num_row:[1,125,129],num_row_group:129,number:[1,105,107,121,125,129],numpi:[1,2,90,129],numpy_typ:129,obj:12,object:[1,12,52,53,54,55,77,80,101,102,104,105,108,121,125,126,127,128,129],obtain:[2,103,123],occupi:1,off:2,offset:[52,101,102],often:[1,3,127],old:126,omit:2,one:[1,2,13,51,60,129],ones:1,onli:[1,105,106,107,123,124,125,126,127,128],open:[80,99,125,127,129],open_fil:125,open_stream:125,oper:[108,125,126,127],option:[1,2,3,26,77,78,127,128,129],org:122,orient:1,origin:[125,129],osfil:127,other:[1,3,122,123,125,126,127,128,129],otherwis:124,our:[2,122],out:[1,125,127,129],output:123,over:126,own:127,packag:2,pad:[91,92],page:[123,126,129],panda:[1,2,123,129],pandas_typ:129,pandas_vers:129,parallel:[105,129],paramet:[12,52,53,54,55,77,78,80,86,99,101,102,103,104,105,106,107,108,109,121,128],parent:127,parq:129,parquet:[2,123,127,128],parquet_bu
ild_toolchain:2,parquet_fil:129,parquet_hom:2,parquetdataset:129,parquetfil:129,parquetfilemetadata:104,parquetschema:129,parquetwrit:129,part:[1,124],particular:[1,125,127,129],particularli:[1,129],partit:[103,123],pass:[1,2,77,97,103,104,105,125,126,127,129],path:[2,52,53,54,55,80,99,101,102,103,129],path_or_path:103,payload:125,peopl:122,per:[1,107,129],perform:[1,129],permit:127,pf2:129,physic:1,piec:[1,103],pip:[123,129],pivot:3,place:2,plain:129,plan:122,platform:2,plu:1,pluggi:2,point:125,pointer:127,pool:[77,110,123,126],popd:2,port:3,posit:[108,121],posix:126,possibl:[103,127,128],practic:129,precis:[83,127],prefer:127,present:129,preserv:127,previou:126,primari:[122,127],primit:1,print:[1,127],process:125,produc:[127,129],project:[122,129],protocol:127,provid:[123,126,127,128,129],pull:124,pushd:2,put:2,pyarrow:[1,2,3,123,125,126],pybyt:78,pypi:124,pytest:2,python35:2,python:[1,12,52,53,54,55,77,101,102,104,105,124,125,127,129],pythonfil:127,pythonfileinterfac:[104,105],py
thonpath:2,random:[123,127],rang:[125,127,129],rapidjson:2,rather:[1,3,104],read:[52,103,104,105,108,122,123],read_buff:127,read_row_group:129,read_tabl:129,readabl:[52,54,101,102,104],reader:[12,54,101,102,104,125,127],realloc:126,receiv:1,recent:124,recommend:2,record:[52,123,125],recordbatch:[1,91,125],recordbatchfileread:[101,125],recordbatchfilewrit:125,recordbatchstreamread:[102,125],recordbatchstreamwrit:125,refer:[123,127],referenc:[1,127],region:126,regular:127,relationship:127,releas:2,relev:1,reli:3,repo:2,repositori:2,repres:[26,128],represent:123,request:[126,127],requir:[1,105,125,127],resolut:128,respect:129,result:[2,129],retriev:124,right:122,rle:129,root:3,rootdir:2,routin:128,row:[51,103,107,129],row_group_s:107,rowgroup:107,runtim:3,safe:127,same:[1,2,103,125,128],save:1,scalar:[1,123],scale:83,schema:[53,55,103,106,123,125,129],sdk:3,second:2,section:[1,3,125,127],see:[1,2,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,
36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,104,105,123,124,129],seek:[125,127],select:1,self:[4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76],semant:127,send:[122,125],sent:125,separ:105,sequenc:[1,77,125],seri:[1,123],serial:[91,92,123,127],serialized_s:129,session:2,set:[2,3,77,123,126,128,129],set_memory_pool:[97,126],setuptool:2,sever:127,share:[1,122,123,129],should:[2,107,129],side:128,signatur:[4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76],significantli:129,similar:[1,2,127,128],similarli:129,simpl:[1,129],simpli:[2,85,126],sinc:[3,125],singl:[1,103,104,105,107,123,126],sink:[53,55,125],site:2,six:2,size:[1,78,80,
91,92,99,125,126,127],skip:2,slack:122,slice:[1,127],slightli:1,small:[1,125],smaller:129,snappi:[2,107,129],socket:[1,125],some:[1,2,3,52,101,102,107,125,127,128,129],somewher:3,sourc:[2,52,53,54,55,97,101,102,103,104,105,106,107,108,123,125,128,129],spark:129,special:1,specif:[1,122],specifi:[107,126,129],speed:129,split_row_group:103,sssssssssssssss:2,standard:[127,129],start:[2,123,125],still:[126,128],storag:[1,129],str:[52,53,54,55,101,102,103,104,105,107,128],strategi:126,stream:[1,54,55,102,123,127],streamwrit:125,string:[1,80,86,99,105,106,107,125,127,128,129],stringarrai:1,struct:1,structur:[1,123,128,129],studio:2,subdirectori:103,subpool:127,subscrib:122,subset:129,sudo:2,suffici:2,suppli:128,support:[1,43,108,123,124,125,126,127,128],sure:125,symlink:2,system:[122,123,126,127,128,129],tabl:[105,107,123,126,128,129],table2:129,tag:124,target:2,tcmalloc:126,tensor:[92,108,121,123],test_arrai:2,test_convert_builtin:2,test_convert_panda:2,test_feath:2,test_hdf:2,test_io:2,t
est_ipc:2,test_jemalloc:2,test_parquet:2,test_scalar:2,test_schema:2,test_tabl:2,test_tensor:2,than:[1,3,78,104,127,128,129],thee:1,thei:[1,127,129],them:[1,128,129],thi:[1,2,3,13,14,26,51,52,60,101,102,108,123,124,125,126,127,128,129],thirdparti:[2,3],those:129,though:1,thought:1,threadsaf:105,three:129,thrift:2,through:[1,122,126],throughput:129,thu:[125,126,128,129],ticket_cache_path:3,time:[1,3,128],timestamp:[1,128],timestamps_to_m:128,to_panda:[1,128,129],to_pybyt:127,todo:1,toggl:129,too:129,tool:[1,127],toolchain:2,top:60,total:[2,91,92,121],total_allocated_byt:127,touch:126,track:127,transmit:1,transport:129,tricki:2,two:[125,127,129],type:[2,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,85,86,123,125,127],ubuntu:2,uncompress:129,underli:105,unicod:[128,129],union:1,unit:128,unit_str:[113,114,115],unmodifi:128,usag:123
,use:[1,3,26,51,60,107,122,125,126,127,129],use_dictionari:[107,129],used:[1,125,127,128,129],useful:125,user:[1,2,3],uses:[3,126,127,129],using:[1,2,3,124,127,128,129],utf8:[111,129],valid:103,validate_schema:103,valu:[1,86,123,128],value_typ:98,variabl:[1,2,3,78],variou:[1,128],vector:[14,128],veri:[52,101,102,125],version:[2,106,107,124,129],via:13,view:127,visual:2,wai:[1,127,129],well:129,wesm:2,when:[1,125,127,129],where:[1,106,107,127,129],whether:129,which:[1,97,125,127,129],whole:129,width:78,win64:2,window:123,within:129,without:[1,127],work:[2,128],worst:129,wrangl:1,wrap:127,write:[106,107,121,123],write_batch:125,write_t:129,writeabl:[53,55,80,125],writer:[53,55,125,127,129],written:[53,55,121,129],xcode:2,year:129,yield:129,you:[1,2,3,122,124,126,127,128,129],your:[2,3,122,127],zero:[12,108,125,127],zip:2,zlib:2},titles:["API Reference","In-Memory Data Model","Development","Filesystem Interfaces","pyarrow.Array","pyarrow.ArrayValue","pyarrow.BinaryArray","pyarrow.Binar
yValue","pyarrow.BooleanArray","pyarrow.BooleanValue","pyarrow.Buffer","pyarrow.BufferOutputStream","pyarrow.BufferReader","pyarrow.ChunkedArray","pyarrow.Column","pyarrow.DataType","pyarrow.Date32Array","pyarrow.Date32Value","pyarrow.Date64Array","pyarrow.Date64Value","pyarrow.DecimalArray","pyarrow.DecimalType","pyarrow.DecimalValue","pyarrow.DictionaryArray","pyarrow.DictionaryType","pyarrow.DoubleValue","pyarrow.Field","pyarrow.FixedSizeBinaryArray","pyarrow.FixedSizeBinaryType","pyarrow.FixedSizeBinaryValue","pyarrow.FloatValue","pyarrow.FloatingPointArray","pyarrow.Int16Array","pyarrow.Int16Value","pyarrow.Int32Array","pyarrow.Int32Value","pyarrow.Int64Array","pyarrow.Int64Value","pyarrow.Int8Array","pyarrow.Int8Value","pyarrow.IntegerArray","pyarrow.ListArray","pyarrow.ListValue","pyarrow.MemoryMappedFile","pyarrow.MemoryPool","pyarrow.NA","pyarrow.NAType","pyarrow.NativeFile","pyarrow.NullArray","pyarrow.NumericArray","pyarrow.PythonFile","pyarrow.RecordBatch","pyarrow.Recor
dBatchFileReader","pyarrow.RecordBatchFileWriter","pyarrow.RecordBatchStreamReader","pyarrow.RecordBatchStreamWriter","pyarrow.Scalar","pyarrow.Schema","pyarrow.StringArray","pyarrow.StringValue","pyarrow.Table","pyarrow.Tensor","pyarrow.Time32Array","pyarrow.Time32Type","pyarrow.Time64Array","pyarrow.Time64Type","pyarrow.TimestampArray","pyarrow.TimestampType","pyarrow.TimestampValue","pyarrow.UInt16Array","pyarrow.UInt16Value","pyarrow.UInt32Array","pyarrow.UInt32Value","pyarrow.UInt64Array","pyarrow.UInt64Value","pyarrow.UInt8Array","pyarrow.UInt8Value","pyarrow.array","pyarrow.binary","pyarrow.bool_","pyarrow.create_memory_map","pyarrow.date32","pyarrow.date64","pyarrow.decimal","pyarrow.default_memory_pool","pyarrow.dictionary","pyarrow.field","pyarrow.float16","pyarrow.float32","pyarrow.float64","pyarrow.from_numpy_dtype","pyarrow.get_record_batch_size","pyarrow.get_tensor_size","pyarrow.int16","pyarrow.int32","pyarrow.int64","pyarrow.int8","pyarrow.jemalloc_memory_pool","pyar
row.list_","pyarrow.memory_map","pyarrow.null","pyarrow.open_file","pyarrow.open_stream","pyarrow.parquet.ParquetDataset","pyarrow.parquet.ParquetFile","pyarrow.parquet.read_table","pyarrow.parquet.write_metadata","pyarrow.parquet.write_table","pyarrow.read_tensor","pyarrow.schema","pyarrow.set_memory_pool","pyarrow.string","pyarrow.struct","pyarrow.time32","pyarrow.time64","pyarrow.timestamp","pyarrow.total_allocated_bytes","pyarrow.uint16","pyarrow.uint32","pyarrow.uint64","pyarrow.uint8","pyarrow.write_tensor","Getting Involved","Apache Arrow (Python)","Install PyArrow","IPC: Fast Streaming and Serialization","jemalloc MemoryPool","Memory and IO Interfaces","Using PyArrow with pandas","Reading and Writing the Apache Parquet Format"],titleterms:{"class":0,"function":0,"null":100,Using:128,access:125,apach:[0,123,129],api:0,arrai:[0,1,4,77],arrayvalu:5,arrow:[123,128],batch:[0,1],binari:78,binaryarrai:6,binaryvalu:7,bool_:79,booleanarrai:8,booleanvalu:9,buffer:[10,127],bufferoutput
stream:11,bufferread:12,build:2,chunkedarrai:13,column:14,commun:0,compat:129,compress:129,conda:[2,124],constructor:0,convers:128,create_memory_map:80,custom:1,data:1,datafram:128,dataset:129,datatyp:15,date32:81,date32arrai:16,date32valu:17,date64:82,date64arrai:18,date64valu:19,decim:83,decimalarrai:20,decimaltyp:21,decimalvalu:22,default_memory_pool:84,develop:2,dictionari:[1,85],dictionaryarrai:23,dictionarytyp:24,differ:128,disk:127,doublevalu:25,encod:129,environ:2,factori:0,fast:125,field:[1,26,86],file:[3,125,127,129],filesystem:[3,127],finer:129,fixedsizebinaryarrai:27,fixedsizebinarytyp:28,fixedsizebinaryvalu:29,float16:87,float32:88,float64:89,floatingpointarrai:31,floatvalu:30,format:129,from:124,from_numpy_dtyp:90,get:122,get_record_batch_s:91,get_tensor_s:92,grain:129,hadoop:[3,127],hdf:3,input:0,instal:124,int16:93,int16arrai:32,int16valu:33,int32:94,int32arrai:34,int32valu:35,int64:95,int64arrai:36,int64valu:37,int8:96,int8arrai:38,int8valu:39,integerarrai:40,interf
ac:[3,127],interprocess:0,involv:122,ipc:125,jemalloc:126,jemalloc_memory_pool:97,linux:2,list_:98,listarrai:41,listvalu:42,maco:2,map:127,memori:[0,1,127],memory_map:99,memorymappedfil:43,memorypool:[44,126],messag:0,metadata:1,model:1,multipl:129,multithread:129,nativ:127,nativefil:47,natyp:46,nullarrai:48,numericarrai:49,obtain:129,open_fil:101,open_stream:102,output:0,panda:128,parquet:[0,103,104,105,106,107,129],parquetdataset:103,parquetfil:104,partit:129,pip:124,pool:[0,127],pyarrow:[4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,124,127,128,129],python:[2,123],pythonfil:50,random:125,read:[125,127,129],read_tabl:105,read_tensor:108,record:[0,1],recordbatch:5
1,recordbatchfileread:52,recordbatchfilewrit:53,recordbatchstreamread:54,recordbatchstreamwrit:55,refer:0,requir:2,run:2,scalar:[0,56],schema:[0,1,57,109],seri:128,serial:125,set_memory_pool:110,setup:2,share:0,singl:129,sourc:124,stream:125,string:111,stringarrai:58,stringvalu:59,struct:112,support:129,system:[2,3],tabl:[0,1,60],tensor:[0,61],test:2,time32:113,time32arrai:62,time32typ:63,time64:114,time64arrai:64,time64typ:65,timestamp:115,timestamparrai:66,timestamptyp:67,timestampvalu:68,total_allocated_byt:116,type:[0,1,128],uint16:117,uint16arrai:69,uint16valu:70,uint32:118,uint32arrai:71,uint32valu:72,uint64:119,uint64arrai:73,uint64valu:74,uint8:120,uint8arrai:75,uint8valu:76,unit:2,valu:0,window:2,write:[125,127,129],write_metadata:106,write_t:107,write_tensor:121}})
\ No newline at end of file
[4/4] arrow-site git commit: Update Python documentation
Posted by we...@apache.org.
Update Python documentation
Project: http://git-wip-us.apache.org/repos/asf/arrow-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/arrow-site/commit/820f0af6
Tree: http://git-wip-us.apache.org/repos/asf/arrow-site/tree/820f0af6
Diff: http://git-wip-us.apache.org/repos/asf/arrow-site/diff/820f0af6
Branch: refs/heads/asf-site
Commit: 820f0af644db98cfb67372d925bd4a146a0c2fc9
Parents: 0f10ead
Author: Wes McKinney <we...@twosigma.com>
Authored: Tue May 23 17:12:29 2017 -0400
Committer: Wes McKinney <we...@twosigma.com>
Committed: Tue May 23 17:12:29 2017 -0400
----------------------------------------------------------------------
docs/python/_modules/index.html | 7 +-
docs/python/_modules/pyarrow.html | 157 ++++++++++-------
docs/python/_modules/pyarrow/ipc.html | 97 ++++++++++-
docs/python/_modules/pyarrow/parquet.html | 77 +++++++--
docs/python/_sources/api.rst.txt | 12 +-
docs/python/_sources/development.rst.txt | 51 +++++-
.../pyarrow.BufferOutputStream.rst.txt | 31 ++++
.../pyarrow.RecordBatchFileReader.rst.txt | 31 ++++
.../pyarrow.RecordBatchFileWriter.rst.txt | 24 +++
.../pyarrow.RecordBatchStreamReader.rst.txt | 30 ++++
.../pyarrow.RecordBatchStreamWriter.rst.txt | 24 +++
.../_sources/generated/pyarrow.Schema.rst.txt | 1 +
.../generated/pyarrow.open_file.rst.txt | 6 +
.../generated/pyarrow.open_stream.rst.txt | 6 +
.../pyarrow.parquet.ParquetFile.rst.txt | 1 +
docs/python/_sources/ipc.rst.txt | 28 +--
docs/python/_sources/memory.rst.txt | 12 +-
docs/python/api.html | 16 +-
docs/python/data.html | 30 ++--
docs/python/development.html | 41 ++++-
.../generated/pyarrow.BufferOutputStream.html | 164 ++++++++++++++++++
docs/python/generated/pyarrow.BufferReader.html | 12 +-
docs/python/generated/pyarrow.MemoryPool.html | 10 +-
docs/python/generated/pyarrow.NativeFile.html | 10 +-
docs/python/generated/pyarrow.PythonFile.html | 10 +-
docs/python/generated/pyarrow.RecordBatch.html | 2 +-
.../pyarrow.RecordBatchFileReader.html | 171 +++++++++++++++++++
.../pyarrow.RecordBatchFileWriter.html | 155 +++++++++++++++++
.../pyarrow.RecordBatchStreamReader.html | 163 ++++++++++++++++++
.../pyarrow.RecordBatchStreamWriter.html | 155 +++++++++++++++++
docs/python/generated/pyarrow.Schema.html | 5 +-
docs/python/generated/pyarrow.Table.html | 4 +-
docs/python/generated/pyarrow.open_file.html | 136 +++++++++++++++
docs/python/generated/pyarrow.open_stream.html | 136 +++++++++++++++
.../generated/pyarrow.parquet.ParquetFile.html | 5 +-
docs/python/genindex.html | 61 ++++---
docs/python/ipc.html | Bin 17201 -> 17716 bytes
docs/python/memory.html | Bin 25705 -> 25681 bytes
docs/python/objects.inv | Bin 3385 -> 3425 bytes
docs/python/parquet.html | 12 +-
docs/python/searchindex.js | 2 +-
41 files changed, 1704 insertions(+), 191 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_modules/index.html
----------------------------------------------------------------------
diff --git a/docs/python/_modules/index.html b/docs/python/_modules/index.html
index 88617d8..63ef96b 100644
--- a/docs/python/_modules/index.html
+++ b/docs/python/_modules/index.html
@@ -60,11 +60,8 @@
<h1>All modules for which code is available</h1>
<ul><li><a href="pyarrow.html">pyarrow</a></li>
-<ul><li><a href="pyarrow/_array.html">pyarrow._array</a></li>
-<li><a href="pyarrow/_io.html">pyarrow._io</a></li>
-<li><a href="pyarrow/_memory.html">pyarrow._memory</a></li>
-<li><a href="pyarrow/_table.html">pyarrow._table</a></li>
-<li><a href="pyarrow/ipc.html">pyarrow.ipc</a></li>
+<ul><li><a href="pyarrow/ipc.html">pyarrow.ipc</a></li>
+<li><a href="pyarrow/lib.html">pyarrow.lib</a></li>
<li><a href="pyarrow/parquet.html">pyarrow.parquet</a></li>
</ul></ul>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_modules/pyarrow.html
----------------------------------------------------------------------
diff --git a/docs/python/_modules/pyarrow.html b/docs/python/_modules/pyarrow.html
index aed49c6..23f64f7 100644
--- a/docs/python/_modules/pyarrow.html
+++ b/docs/python/_modules/pyarrow.html
@@ -87,71 +87,69 @@
<span class="k">pass</span>
-<span class="kn">import</span> <span class="nn">pyarrow._config</span>
-<span class="kn">from</span> <span class="nn">pyarrow._config</span> <span class="k">import</span> <span class="n">cpu_count</span><span class="p">,</span> <span class="n">set_cpu_count</span>
-
-<span class="kn">from</span> <span class="nn">pyarrow._array</span> <span class="k">import</span> <span class="p">(</span><span class="n">null</span><span class="p">,</span> <span class="n">bool_</span><span class="p">,</span>
- <span class="n">int8</span><span class="p">,</span> <span class="n">int16</span><span class="p">,</span> <span class="n">int32</span><span class="p">,</span> <span class="n">int64</span><span class="p">,</span>
- <span class="n">uint8</span><span class="p">,</span> <span class="n">uint16</span><span class="p">,</span> <span class="n">uint32</span><span class="p">,</span> <span class="n">uint64</span><span class="p">,</span>
- <span class="n">time32</span><span class="p">,</span> <span class="n">time64</span><span class="p">,</span> <span class="n">timestamp</span><span class="p">,</span> <span class="n">date32</span><span class="p">,</span> <span class="n">date64</span><span class="p">,</span>
- <span class="n">float16</span><span class="p">,</span> <span class="n">float32</span><span class="p">,</span> <span class="n">float64</span><span class="p">,</span>
- <span class="n">binary</span><span class="p">,</span> <span class="n">string</span><span class="p">,</span> <span class="n">decimal</span><span class="p">,</span>
- <span class="n">list_</span><span class="p">,</span> <span class="n">struct</span><span class="p">,</span> <span class="n">dictionary</span><span class="p">,</span> <span class="n">field</span><span class="p">,</span>
- <span class="n">DataType</span><span class="p">,</span>
- <span class="n">DecimalType</span><span class="p">,</span>
- <span class="n">DictionaryType</span><span class="p">,</span>
- <span class="n">FixedSizeBinaryType</span><span class="p">,</span>
- <span class="n">TimestampType</span><span class="p">,</span>
- <span class="n">Time32Type</span><span class="p">,</span>
- <span class="n">Time64Type</span><span class="p">,</span>
- <span class="n">Field</span><span class="p">,</span>
- <span class="n">Schema</span><span class="p">,</span>
- <span class="n">schema</span><span class="p">,</span>
- <span class="n">Array</span><span class="p">,</span> <span class="n">Tensor</span><span class="p">,</span>
- <span class="n">array</span><span class="p">,</span>
- <span class="n">from_numpy_dtype</span><span class="p">,</span>
- <span class="n">NullArray</span><span class="p">,</span>
- <span class="n">NumericArray</span><span class="p">,</span> <span class="n">IntegerArray</span><span class="p">,</span> <span class="n">FloatingPointArray</span><span class="p">,</span>
- <span class="n">BooleanArray</span><span class="p">,</span>
- <span class="n">Int8Array</span><span class="p">,</span> <span class="n">UInt8Array</span><span class="p">,</span>
- <span class="n">Int16Array</span><span class="p">,</span> <span class="n">UInt16Array</span><span class="p">,</span>
- <span class="n">Int32Array</span><span class="p">,</span> <span class="n">UInt32Array</span><span class="p">,</span>
- <span class="n">Int64Array</span><span class="p">,</span> <span class="n">UInt64Array</span><span class="p">,</span>
- <span class="n">ListArray</span><span class="p">,</span>
- <span class="n">BinaryArray</span><span class="p">,</span> <span class="n">StringArray</span><span class="p">,</span>
- <span class="n">FixedSizeBinaryArray</span><span class="p">,</span>
- <span class="n">DictionaryArray</span><span class="p">,</span>
- <span class="n">Date32Array</span><span class="p">,</span> <span class="n">Date64Array</span><span class="p">,</span>
- <span class="n">TimestampArray</span><span class="p">,</span> <span class="n">Time32Array</span><span class="p">,</span> <span class="n">Time64Array</span><span class="p">,</span>
- <span class="n">DecimalArray</span><span class="p">,</span>
- <span class="n">ArrayValue</span><span class="p">,</span> <span class="n">Scalar</span><span class="p">,</span> <span class="n">NA</span><span class="p">,</span> <span class="n">NAType</span><span class="p">,</span>
- <span class="n">BooleanValue</span><span class="p">,</span>
- <span class="n">Int8Value</span><span class="p">,</span> <span class="n">Int16Value</span><span class="p">,</span> <span class="n">Int32Value</span><span class="p">,</span> <span class="n">Int64Value</span><span class="p">,</span>
- <span class="n">UInt8Value</span><span class="p">,</span> <span class="n">UInt16Value</span><span class="p">,</span> <span class="n">UInt32Value</span><span class="p">,</span> <span class="n">UInt64Value</span><span class="p">,</span>
- <span class="n">FloatValue</span><span class="p">,</span> <span class="n">DoubleValue</span><span class="p">,</span> <span class="n">ListValue</span><span class="p">,</span>
- <span class="n">BinaryValue</span><span class="p">,</span> <span class="n">StringValue</span><span class="p">,</span> <span class="n">FixedSizeBinaryValue</span><span class="p">,</span>
- <span class="n">DecimalValue</span><span class="p">,</span>
- <span class="n">Date32Value</span><span class="p">,</span> <span class="n">Date64Value</span><span class="p">,</span> <span class="n">TimestampValue</span><span class="p">)</span>
-
-<span class="kn">from</span> <span class="nn">pyarrow._io</span> <span class="k">import</span> <span class="p">(</span><span class="n">HdfsFile</span><span class="p">,</span> <span class="n">NativeFile</span><span class="p">,</span> <span class="n">PythonFile</span><span class="p">,</span>
- <span class="n">Buffer</span><span class="p">,</span> <span class="n">BufferReader</span><span class="p">,</span> <span class="n">InMemoryOutputStream</span><span class="p">,</span>
+<span class="kn">from</span> <span class="nn">pyarrow.lib</span> <span class="k">import</span> <span class="n">cpu_count</span><span class="p">,</span> <span class="n">set_cpu_count</span>
+<span class="kn">from</span> <span class="nn">pyarrow.lib</span> <span class="k">import</span> <span class="p">(</span><span class="n">null</span><span class="p">,</span> <span class="n">bool_</span><span class="p">,</span>
+ <span class="n">int8</span><span class="p">,</span> <span class="n">int16</span><span class="p">,</span> <span class="n">int32</span><span class="p">,</span> <span class="n">int64</span><span class="p">,</span>
+ <span class="n">uint8</span><span class="p">,</span> <span class="n">uint16</span><span class="p">,</span> <span class="n">uint32</span><span class="p">,</span> <span class="n">uint64</span><span class="p">,</span>
+ <span class="n">time32</span><span class="p">,</span> <span class="n">time64</span><span class="p">,</span> <span class="n">timestamp</span><span class="p">,</span> <span class="n">date32</span><span class="p">,</span> <span class="n">date64</span><span class="p">,</span>
+ <span class="n">float16</span><span class="p">,</span> <span class="n">float32</span><span class="p">,</span> <span class="n">float64</span><span class="p">,</span>
+ <span class="n">binary</span><span class="p">,</span> <span class="n">string</span><span class="p">,</span> <span class="n">decimal</span><span class="p">,</span>
+ <span class="n">list_</span><span class="p">,</span> <span class="n">struct</span><span class="p">,</span> <span class="n">dictionary</span><span class="p">,</span> <span class="n">field</span><span class="p">,</span>
+ <span class="n">DataType</span><span class="p">,</span>
+ <span class="n">DecimalType</span><span class="p">,</span>
+ <span class="n">DictionaryType</span><span class="p">,</span>
+ <span class="n">FixedSizeBinaryType</span><span class="p">,</span>
+ <span class="n">TimestampType</span><span class="p">,</span>
+ <span class="n">Time32Type</span><span class="p">,</span>
+ <span class="n">Time64Type</span><span class="p">,</span>
+ <span class="n">Field</span><span class="p">,</span>
+ <span class="n">Schema</span><span class="p">,</span>
+ <span class="n">schema</span><span class="p">,</span>
+ <span class="n">Array</span><span class="p">,</span> <span class="n">Tensor</span><span class="p">,</span>
+ <span class="n">array</span><span class="p">,</span>
+ <span class="n">from_numpy_dtype</span><span class="p">,</span>
+ <span class="n">NullArray</span><span class="p">,</span>
+ <span class="n">NumericArray</span><span class="p">,</span> <span class="n">IntegerArray</span><span class="p">,</span> <span class="n">FloatingPointArray</span><span class="p">,</span>
+ <span class="n">BooleanArray</span><span class="p">,</span>
+ <span class="n">Int8Array</span><span class="p">,</span> <span class="n">UInt8Array</span><span class="p">,</span>
+ <span class="n">Int16Array</span><span class="p">,</span> <span class="n">UInt16Array</span><span class="p">,</span>
+ <span class="n">Int32Array</span><span class="p">,</span> <span class="n">UInt32Array</span><span class="p">,</span>
+ <span class="n">Int64Array</span><span class="p">,</span> <span class="n">UInt64Array</span><span class="p">,</span>
+ <span class="n">ListArray</span><span class="p">,</span>
+ <span class="n">BinaryArray</span><span class="p">,</span> <span class="n">StringArray</span><span class="p">,</span>
+ <span class="n">FixedSizeBinaryArray</span><span class="p">,</span>
+ <span class="n">DictionaryArray</span><span class="p">,</span>
+ <span class="n">Date32Array</span><span class="p">,</span> <span class="n">Date64Array</span><span class="p">,</span>
+ <span class="n">TimestampArray</span><span class="p">,</span> <span class="n">Time32Array</span><span class="p">,</span> <span class="n">Time64Array</span><span class="p">,</span>
+ <span class="n">DecimalArray</span><span class="p">,</span>
+ <span class="n">ArrayValue</span><span class="p">,</span> <span class="n">Scalar</span><span class="p">,</span> <span class="n">NA</span><span class="p">,</span> <span class="n">NAType</span><span class="p">,</span>
+ <span class="n">BooleanValue</span><span class="p">,</span>
+ <span class="n">Int8Value</span><span class="p">,</span> <span class="n">Int16Value</span><span class="p">,</span> <span class="n">Int32Value</span><span class="p">,</span> <span class="n">Int64Value</span><span class="p">,</span>
+ <span class="n">UInt8Value</span><span class="p">,</span> <span class="n">UInt16Value</span><span class="p">,</span> <span class="n">UInt32Value</span><span class="p">,</span> <span class="n">UInt64Value</span><span class="p">,</span>
+ <span class="n">FloatValue</span><span class="p">,</span> <span class="n">DoubleValue</span><span class="p">,</span> <span class="n">ListValue</span><span class="p">,</span>
+ <span class="n">BinaryValue</span><span class="p">,</span> <span class="n">StringValue</span><span class="p">,</span> <span class="n">FixedSizeBinaryValue</span><span class="p">,</span>
+ <span class="n">DecimalValue</span><span class="p">,</span>
+ <span class="n">Date32Value</span><span class="p">,</span> <span class="n">Date64Value</span><span class="p">,</span> <span class="n">TimestampValue</span><span class="p">)</span>
+
+<span class="kn">from</span> <span class="nn">pyarrow.lib</span> <span class="k">import</span> <span class="p">(</span><span class="n">HdfsFile</span><span class="p">,</span> <span class="n">NativeFile</span><span class="p">,</span> <span class="n">PythonFile</span><span class="p">,</span>
+ <span class="n">Buffer</span><span class="p">,</span> <span class="n">BufferReader</span><span class="p">,</span> <span class="n">BufferOutputStream</span><span class="p">,</span>
<span class="n">OSFile</span><span class="p">,</span> <span class="n">MemoryMappedFile</span><span class="p">,</span> <span class="n">memory_map</span><span class="p">,</span>
<span class="n">frombuffer</span><span class="p">,</span> <span class="n">read_tensor</span><span class="p">,</span> <span class="n">write_tensor</span><span class="p">,</span>
<span class="n">memory_map</span><span class="p">,</span> <span class="n">create_memory_map</span><span class="p">,</span>
<span class="n">get_record_batch_size</span><span class="p">,</span> <span class="n">get_tensor_size</span><span class="p">,</span>
<span class="n">have_libhdfs</span><span class="p">,</span> <span class="n">have_libhdfs3</span><span class="p">)</span>
-<span class="kn">from</span> <span class="nn">pyarrow._memory</span> <span class="k">import</span> <span class="p">(</span><span class="n">MemoryPool</span><span class="p">,</span> <span class="n">total_allocated_bytes</span><span class="p">,</span>
- <span class="n">set_memory_pool</span><span class="p">,</span> <span class="n">default_memory_pool</span><span class="p">)</span>
-<span class="kn">from</span> <span class="nn">pyarrow._table</span> <span class="k">import</span> <span class="p">(</span><span class="n">ChunkedArray</span><span class="p">,</span> <span class="n">Column</span><span class="p">,</span> <span class="n">RecordBatch</span><span class="p">,</span> <span class="n">Table</span><span class="p">,</span>
- <span class="n">concat_tables</span><span class="p">)</span>
-<span class="kn">from</span> <span class="nn">pyarrow._error</span> <span class="k">import</span> <span class="p">(</span><span class="n">ArrowException</span><span class="p">,</span>
- <span class="n">ArrowKeyError</span><span class="p">,</span>
- <span class="n">ArrowInvalid</span><span class="p">,</span>
- <span class="n">ArrowIOError</span><span class="p">,</span>
- <span class="n">ArrowMemoryError</span><span class="p">,</span>
- <span class="n">ArrowNotImplementedError</span><span class="p">,</span>
- <span class="n">ArrowTypeError</span><span class="p">)</span>
+<span class="kn">from</span> <span class="nn">pyarrow.lib</span> <span class="k">import</span> <span class="p">(</span><span class="n">MemoryPool</span><span class="p">,</span> <span class="n">total_allocated_bytes</span><span class="p">,</span>
+ <span class="n">set_memory_pool</span><span class="p">,</span> <span class="n">default_memory_pool</span><span class="p">)</span>
+<span class="kn">from</span> <span class="nn">pyarrow.lib</span> <span class="k">import</span> <span class="p">(</span><span class="n">ChunkedArray</span><span class="p">,</span> <span class="n">Column</span><span class="p">,</span> <span class="n">RecordBatch</span><span class="p">,</span> <span class="n">Table</span><span class="p">,</span>
+ <span class="n">concat_tables</span><span class="p">)</span>
+<span class="kn">from</span> <span class="nn">pyarrow.lib</span> <span class="k">import</span> <span class="p">(</span><span class="n">ArrowException</span><span class="p">,</span>
+ <span class="n">ArrowKeyError</span><span class="p">,</span>
+ <span class="n">ArrowInvalid</span><span class="p">,</span>
+ <span class="n">ArrowIOError</span><span class="p">,</span>
+ <span class="n">ArrowMemoryError</span><span class="p">,</span>
+ <span class="n">ArrowNotImplementedError</span><span class="p">,</span>
+ <span class="n">ArrowTypeError</span><span class="p">)</span>
<div class="viewcode-block" id="jemalloc_memory_pool"><a class="viewcode-back" href="../generated/pyarrow.jemalloc_memory_pool.html#pyarrow.jemalloc_memory_pool">[docs]</a><span class="k">def</span> <span class="nf">jemalloc_memory_pool</span><span class="p">():</span>
@@ -165,10 +163,49 @@
<span class="kn">from</span> <span class="nn">pyarrow.filesystem</span> <span class="k">import</span> <span class="n">Filesystem</span><span class="p">,</span> <span class="n">HdfsClient</span><span class="p">,</span> <span class="n">LocalFilesystem</span>
-<span class="kn">from</span> <span class="nn">pyarrow.ipc</span> <span class="k">import</span> <span class="n">FileReader</span><span class="p">,</span> <span class="n">FileWriter</span><span class="p">,</span> <span class="n">StreamReader</span><span class="p">,</span> <span class="n">StreamWriter</span>
+<span class="kn">from</span> <span class="nn">pyarrow.ipc</span> <span class="k">import</span> <span class="p">(</span><span class="n">RecordBatchFileReader</span><span class="p">,</span> <span class="n">RecordBatchFileWriter</span><span class="p">,</span>
+ <span class="n">RecordBatchStreamReader</span><span class="p">,</span> <span class="n">RecordBatchStreamWriter</span><span class="p">,</span>
+ <span class="n">open_stream</span><span class="p">,</span>
+ <span class="n">open_file</span><span class="p">,</span>
+ <span class="n">serialize_pandas</span><span class="p">,</span> <span class="n">deserialize_pandas</span><span class="p">)</span>
<span class="n">localfs</span> <span class="o">=</span> <span class="n">LocalFilesystem</span><span class="o">.</span><span class="n">get_instance</span><span class="p">()</span>
+
+
+<span class="c1"># ----------------------------------------------------------------------</span>
+<span class="c1"># 0.4.0 deprecations</span>
+
+<span class="kn">import</span> <span class="nn">warnings</span>
+
+<span class="k">def</span> <span class="nf">_deprecate_class</span><span class="p">(</span><span class="n">old_name</span><span class="p">,</span> <span class="n">new_name</span><span class="p">,</span> <span class="n">klass</span><span class="p">,</span> <span class="n">next_version</span><span class="o">=</span><span class="s1">'0.5.0'</span><span class="p">):</span>
+ <span class="n">msg</span> <span class="o">=</span> <span class="p">(</span><span class="s1">'pyarrow.</span><span class="si">{0}</span><span class="s1"> has been renamed to '</span>
+ <span class="s1">'</span><span class="si">{1}</span><span class="s1">, will be removed in </span><span class="si">{2}</span><span class="s1">'</span>
+ <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">old_name</span><span class="p">,</span> <span class="n">new_name</span><span class="p">,</span> <span class="n">next_version</span><span class="p">))</span>
+ <span class="k">def</span> <span class="nf">deprecated_factory</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
+ <span class="n">warnings</span><span class="o">.</span><span class="n">warn</span><span class="p">(</span><span class="n">msg</span><span class="p">,</span> <span class="ne">FutureWarning</span><span class="p">)</span>
+ <span class="k">return</span> <span class="n">klass</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
+ <span class="k">return</span> <span class="n">deprecated_factory</span>
+
+<span class="n">FileReader</span> <span class="o">=</span> <span class="n">_deprecate_class</span><span class="p">(</span><span class="s1">'FileReader'</span><span class="p">,</span>
+ <span class="s1">'RecordBatchFileReader'</span><span class="p">,</span>
+ <span class="n">RecordBatchFileReader</span><span class="p">,</span> <span class="s1">'0.5.0'</span><span class="p">)</span>
+
+<span class="n">FileWriter</span> <span class="o">=</span> <span class="n">_deprecate_class</span><span class="p">(</span><span class="s1">'FileWriter'</span><span class="p">,</span>
+ <span class="s1">'RecordBatchFileWriter'</span><span class="p">,</span>
+ <span class="n">RecordBatchFileWriter</span><span class="p">,</span> <span class="s1">'0.5.0'</span><span class="p">)</span>
+
+<span class="n">StreamReader</span> <span class="o">=</span> <span class="n">_deprecate_class</span><span class="p">(</span><span class="s1">'StreamReader'</span><span class="p">,</span>
+ <span class="s1">'RecordBatchStreamReader'</span><span class="p">,</span>
+ <span class="n">RecordBatchStreamReader</span><span class="p">,</span> <span class="s1">'0.5.0'</span><span class="p">)</span>
+
+<span class="n">StreamWriter</span> <span class="o">=</span> <span class="n">_deprecate_class</span><span class="p">(</span><span class="s1">'StreamWriter'</span><span class="p">,</span>
+ <span class="s1">'RecordBatchStreamWriter'</span><span class="p">,</span>
+ <span class="n">RecordBatchStreamWriter</span><span class="p">,</span> <span class="s1">'0.5.0'</span><span class="p">)</span>
+
+<span class="n">InMemoryOutputStream</span> <span class="o">=</span> <span class="n">_deprecate_class</span><span class="p">(</span><span class="s1">'InMemoryOutputStream'</span><span class="p">,</span>
+ <span class="s1">'BufferOutputStream'</span><span class="p">,</span>
+ <span class="n">BufferOutputStream</span><span class="p">,</span> <span class="s1">'0.5.0'</span><span class="p">)</span>
</pre></div>
</div>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_modules/pyarrow/ipc.html
----------------------------------------------------------------------
diff --git a/docs/python/_modules/pyarrow/ipc.html b/docs/python/_modules/pyarrow/ipc.html
index 750e2a3..aa2a3fb 100644
--- a/docs/python/_modules/pyarrow/ipc.html
+++ b/docs/python/_modules/pyarrow/ipc.html
@@ -80,10 +80,11 @@
<span class="c1"># Arrow file and stream reader/writer classes, and other messaging tools</span>
-<span class="kn">import</span> <span class="nn">pyarrow._io</span> <span class="k">as</span> <span class="nn">_io</span>
+<span class="kn">import</span> <span class="nn">pyarrow</span> <span class="k">as</span> <span class="nn">pa</span>
+<span class="kn">import</span> <span class="nn">pyarrow.lib</span> <span class="k">as</span> <span class="nn">lib</span>
-<div class="viewcode-block" id="StreamReader"><a class="viewcode-back" href="../../generated/pyarrow.StreamReader.html#pyarrow.StreamReader">[docs]</a><span class="k">class</span> <span class="nc">StreamReader</span><span class="p">(</span><span class="n">_io</span><span class="o">.</span><span class="n">_StreamReader</span><span class="p">):</span>
+<div class="viewcode-block" id="RecordBatchStreamReader"><a class="viewcode-back" href="../../generated/pyarrow.RecordBatchStreamReader.html#pyarrow.RecordBatchStreamReader">[docs]</a><span class="k">class</span> <span class="nc">RecordBatchStreamReader</span><span class="p">(</span><span class="n">lib</span><span class="o">.</span><span class="n">_RecordBatchReader</span><span class="p">):</span>
<span class="sd">"""</span>
<span class="sd"> Reader for the Arrow streaming binary format</span>
@@ -92,7 +93,7 @@
<span class="sd"> source : str, pyarrow.NativeFile, or file-like Python object</span>
<span class="sd"> Either a file path, or a readable file object</span>
<span class="sd"> """</span>
-<div class="viewcode-block" id="StreamReader.__init__"><a class="viewcode-back" href="../../generated/pyarrow.StreamReader.html#pyarrow.StreamReader.__init__">[docs]</a> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">source</span><span class="p">):</span>
+<div class="viewcode-block" id="RecordBatchStreamReader.__init__"><a class="viewcode-back" href="../../generated/pyarrow.RecordBatchStreamReader.html#pyarrow.RecordBatchStreamReader.__init__">[docs]</a> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">source</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_open</span><span class="p">(</span><span class="n">source</span><span class="p">)</span></div>
<span class="k">def</span> <span class="nf">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
@@ -100,7 +101,7 @@
<span class="k">yield</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_next_batch</span><span class="p">()</span></div>
-<div class="viewcode-block" id="StreamWriter"><a class="viewcode-back" href="../../generated/pyarrow.StreamWriter.html#pyarrow.StreamWriter">[docs]</a><span class="k">class</span> <span class="nc">StreamWriter</span><span class="p">(</span><span class="n">_io</span><span class="o">.</span><span class="n">_StreamWriter</span><span class="p">):</span>
+<div class="viewcode-block" id="RecordBatchStreamWriter"><a class="viewcode-back" href="../../generated/pyarrow.RecordBatchStreamWriter.html#pyarrow.RecordBatchStreamWriter">[docs]</a><span class="k">class</span> <span class="nc">RecordBatchStreamWriter</span><span class="p">(</span><span class="n">lib</span><span class="o">.</span><span class="n">_RecordBatchWriter</span><span class="p">):</span>
<span class="sd">"""</span>
<span class="sd"> Writer for the Arrow streaming binary format</span>
@@ -111,11 +112,11 @@
<span class="sd"> schema : pyarrow.Schema</span>
<span class="sd"> The Arrow schema for data to be written to the file</span>
<span class="sd"> """</span>
-<div class="viewcode-block" id="StreamWriter.__init__"><a class="viewcode-back" href="../../generated/pyarrow.StreamWriter.html#pyarrow.StreamWriter.__init__">[docs]</a> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sink</span><span class="p">,</span> <span class="n">schema</span><span class="p">):</span>
+<div class="viewcode-block" id="RecordBatchStreamWriter.__init__"><a class="viewcode-back" href="../../generated/pyarrow.RecordBatchStreamWriter.html#pyarrow.RecordBatchStreamWriter.__init__">[docs]</a> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sink</span><span class="p">,</span> <span class="n">schema</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_open</span><span class="p">(</span><span class="n">sink</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span></div></div>
-<div class="viewcode-block" id="FileReader"><a class="viewcode-back" href="../../generated/pyarrow.FileReader.html#pyarrow.FileReader">[docs]</a><span class="k">class</span> <span class="nc">FileReader</span><span class="p">(</span><span class="n">_io</span><span class="o">.</span><span class="n">_FileReader</span><span class="p">):</span>
+<div class="viewcode-block" id="RecordBatchFileReader"><a class="viewcode-back" href="../../generated/pyarrow.RecordBatchFileReader.html#pyarrow.RecordBatchFileReader">[docs]</a><span class="k">class</span> <span class="nc">RecordBatchFileReader</span><span class="p">(</span><span class="n">lib</span><span class="o">.</span><span class="n">_RecordBatchFileReader</span><span class="p">):</span>
<span class="sd">"""</span>
<span class="sd"> Class for reading Arrow record batch data from the Arrow binary file format</span>
@@ -127,11 +128,11 @@
<span class="sd"> If the file is embedded in some larger file, this is the byte offset to</span>
<span class="sd"> the very end of the file data</span>
<span class="sd"> """</span>
-<div class="viewcode-block" id="FileReader.__init__"><a class="viewcode-back" href="../../generated/pyarrow.FileReader.html#pyarrow.FileReader.__init__">[docs]</a> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">source</span><span class="p">,</span> <span class="n">footer_offset</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
+<div class="viewcode-block" id="RecordBatchFileReader.__init__"><a class="viewcode-back" href="../../generated/pyarrow.RecordBatchFileReader.html#pyarrow.RecordBatchFileReader.__init__">[docs]</a> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">source</span><span class="p">,</span> <span class="n">footer_offset</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_open</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">footer_offset</span><span class="o">=</span><span class="n">footer_offset</span><span class="p">)</span></div></div>
-<div class="viewcode-block" id="FileWriter"><a class="viewcode-back" href="../../generated/pyarrow.FileWriter.html#pyarrow.FileWriter">[docs]</a><span class="k">class</span> <span class="nc">FileWriter</span><span class="p">(</span><span class="n">_io</span><span class="o">.</span><span class="n">_FileWriter</span><span class="p">):</span>
+<div class="viewcode-block" id="RecordBatchFileWriter"><a class="viewcode-back" href="../../generated/pyarrow.RecordBatchFileWriter.html#pyarrow.RecordBatchFileWriter">[docs]</a><span class="k">class</span> <span class="nc">RecordBatchFileWriter</span><span class="p">(</span><span class="n">lib</span><span class="o">.</span><span class="n">_RecordBatchFileWriter</span><span class="p">):</span>
<span class="sd">"""</span>
<span class="sd"> Writer to create the Arrow binary file format</span>
@@ -142,8 +143,86 @@
<span class="sd"> schema : pyarrow.Schema</span>
<span class="sd"> The Arrow schema for data to be written to the file</span>
<span class="sd"> """</span>
-<div class="viewcode-block" id="FileWriter.__init__"><a class="viewcode-back" href="../../generated/pyarrow.FileWriter.html#pyarrow.FileWriter.__init__">[docs]</a> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sink</span><span class="p">,</span> <span class="n">schema</span><span class="p">):</span>
+<div class="viewcode-block" id="RecordBatchFileWriter.__init__"><a class="viewcode-back" href="../../generated/pyarrow.RecordBatchFileWriter.html#pyarrow.RecordBatchFileWriter.__init__">[docs]</a> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sink</span><span class="p">,</span> <span class="n">schema</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_open</span><span class="p">(</span><span class="n">sink</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span></div></div>
+
+
+<div class="viewcode-block" id="open_stream"><a class="viewcode-back" href="../../generated/pyarrow.open_stream.html#pyarrow.open_stream">[docs]</a><span class="k">def</span> <span class="nf">open_stream</span><span class="p">(</span><span class="n">source</span><span class="p">):</span>
+ <span class="sd">"""</span>
+<span class="sd"> Create reader for Arrow streaming format</span>
+
+<span class="sd"> Parameters</span>
+<span class="sd"> ----------</span>
+<span class="sd"> source : str, pyarrow.NativeFile, or file-like Python object</span>
+<span class="sd"> Either a file path, or a readable file object</span>
+<span class="sd"> footer_offset : int, default None</span>
+<span class="sd"> If the file is embedded in some larger file, this is the byte offset to</span>
+<span class="sd"> the very end of the file data</span>
+
+<span class="sd"> Returns</span>
+<span class="sd"> -------</span>
+<span class="sd"> reader : RecordBatchStreamReader</span>
+<span class="sd"> """</span>
+ <span class="k">return</span> <span class="n">RecordBatchStreamReader</span><span class="p">(</span><span class="n">source</span><span class="p">)</span></div>
+
+
+<div class="viewcode-block" id="open_file"><a class="viewcode-back" href="../../generated/pyarrow.open_file.html#pyarrow.open_file">[docs]</a><span class="k">def</span> <span class="nf">open_file</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">footer_offset</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
+ <span class="sd">"""</span>
+<span class="sd"> Create reader for Arrow file format</span>
+
+<span class="sd"> Parameters</span>
+<span class="sd"> ----------</span>
+<span class="sd"> source : str, pyarrow.NativeFile, or file-like Python object</span>
+<span class="sd"> Either a file path, or a readable file object</span>
+<span class="sd"> footer_offset : int, default None</span>
+<span class="sd"> If the file is embedded in some larger file, this is the byte offset to</span>
+<span class="sd"> the very end of the file data</span>
+
+<span class="sd"> Returns</span>
+<span class="sd"> -------</span>
+<span class="sd"> reader : RecordBatchFileReader</span>
+<span class="sd"> """</span>
+ <span class="k">return</span> <span class="n">RecordBatchFileReader</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">footer_offset</span><span class="o">=</span><span class="n">footer_offset</span><span class="p">)</span></div>
+
+
+<span class="k">def</span> <span class="nf">serialize_pandas</span><span class="p">(</span><span class="n">df</span><span class="p">):</span>
+ <span class="sd">"""Serialize a pandas DataFrame into a buffer protocol compatible object.</span>
+
+<span class="sd"> Parameters</span>
+<span class="sd"> ----------</span>
+<span class="sd"> df : pandas.DataFrame</span>
+
+<span class="sd"> Returns</span>
+<span class="sd"> -------</span>
+<span class="sd"> buf : buffer</span>
+<span class="sd"> An object compatible with the buffer protocol</span>
+<span class="sd"> """</span>
+ <span class="n">batch</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">RecordBatch</span><span class="o">.</span><span class="n">from_pandas</span><span class="p">(</span><span class="n">df</span><span class="p">)</span>
+ <span class="n">sink</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">InMemoryOutputStream</span><span class="p">()</span>
+ <span class="n">writer</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">RecordBatchFileWriter</span><span class="p">(</span><span class="n">sink</span><span class="p">,</span> <span class="n">batch</span><span class="o">.</span><span class="n">schema</span><span class="p">)</span>
+ <span class="n">writer</span><span class="o">.</span><span class="n">write_batch</span><span class="p">(</span><span class="n">batch</span><span class="p">)</span>
+ <span class="n">writer</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
+ <span class="k">return</span> <span class="n">sink</span><span class="o">.</span><span class="n">get_result</span><span class="p">()</span>
+
+
+<span class="k">def</span> <span class="nf">deserialize_pandas</span><span class="p">(</span><span class="n">buf</span><span class="p">,</span> <span class="n">nthreads</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
+ <span class="sd">"""Deserialize a buffer protocol compatible object into a pandas DataFrame.</span>
+
+<span class="sd"> Parameters</span>
+<span class="sd"> ----------</span>
+<span class="sd"> buf : buffer</span>
+<span class="sd"> An object compatible with the buffer protocol</span>
+<span class="sd"> nthreads : int, optional</span>
+<span class="sd"> The number of threads to use to convert the buffer to a DataFrame.</span>
+
+<span class="sd"> Returns</span>
+<span class="sd"> -------</span>
+<span class="sd"> df : pandas.DataFrame</span>
+<span class="sd"> """</span>
+ <span class="n">buffer_reader</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">BufferReader</span><span class="p">(</span><span class="n">buf</span><span class="p">)</span>
+ <span class="n">reader</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">RecordBatchFileReader</span><span class="p">(</span><span class="n">buffer_reader</span><span class="p">)</span>
+ <span class="n">table</span> <span class="o">=</span> <span class="n">reader</span><span class="o">.</span><span class="n">read_all</span><span class="p">()</span>
+ <span class="k">return</span> <span class="n">table</span><span class="o">.</span><span class="n">to_pandas</span><span class="p">(</span><span class="n">nthreads</span><span class="o">=</span><span class="n">nthreads</span><span class="p">)</span>
</pre></div>
</div>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_modules/pyarrow/parquet.html
----------------------------------------------------------------------
diff --git a/docs/python/_modules/pyarrow/parquet.html b/docs/python/_modules/pyarrow/parquet.html
index 7a386b9..e380981 100644
--- a/docs/python/_modules/pyarrow/parquet.html
+++ b/docs/python/_modules/pyarrow/parquet.html
@@ -78,6 +78,8 @@
<span class="c1"># specific language governing permissions and limitations</span>
<span class="c1"># under the License.</span>
+<span class="kn">import</span> <span class="nn">json</span>
+
<span class="kn">import</span> <span class="nn">six</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
@@ -87,8 +89,7 @@
<span class="n">RowGroupMetaData</span><span class="p">,</span> <span class="n">ParquetSchema</span><span class="p">,</span>
<span class="n">ParquetWriter</span><span class="p">)</span>
<span class="kn">import</span> <span class="nn">pyarrow._parquet</span> <span class="k">as</span> <span class="nn">_parquet</span> <span class="c1"># noqa</span>
-<span class="kn">import</span> <span class="nn">pyarrow._array</span> <span class="k">as</span> <span class="nn">_array</span>
-<span class="kn">import</span> <span class="nn">pyarrow._table</span> <span class="k">as</span> <span class="nn">_table</span>
+<span class="kn">import</span> <span class="nn">pyarrow.lib</span> <span class="k">as</span> <span class="nn">lib</span>
<span class="c1"># ----------------------------------------------------------------------</span>
@@ -165,14 +166,31 @@
<span class="n">column_indices</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_column_indices</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
<span class="k">if</span> <span class="n">nthreads</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">set_num_threads</span><span class="p">(</span><span class="n">nthreads</span><span class="p">)</span>
+
+ <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">read_all</span><span class="p">(</span><span class="n">column_indices</span><span class="o">=</span><span class="n">column_indices</span><span class="p">)</span>
+
+ <span class="k">def</span> <span class="nf">read_pandas</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">nthreads</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
+ <span class="n">column_indices</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_column_indices</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
+ <span class="n">custom_metadata</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">metadata</span><span class="o">.</span><span class="n">metadata</span>
+
+ <span class="k">if</span> <span class="n">custom_metadata</span> <span class="ow">and</span> <span class="sa">b</span><span class="s1">'pandas'</span> <span class="ow">in</span> <span class="n">custom_metadata</span><span class="p">:</span>
+ <span class="n">index_columns</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span>
+ <span class="n">custom_metadata</span><span class="p">[</span><span class="sa">b</span><span class="s1">'pandas'</span><span class="p">]</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s1">'utf8'</span><span class="p">)</span>
+ <span class="p">)[</span><span class="s1">'index_columns'</span><span class="p">]</span>
+ <span class="k">else</span><span class="p">:</span>
+ <span class="n">index_columns</span> <span class="o">=</span> <span class="p">[]</span>
+
+ <span class="k">if</span> <span class="n">column_indices</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="ow">and</span> <span class="n">index_columns</span><span class="p">:</span>
+ <span class="n">column_indices</span> <span class="o">+=</span> <span class="nb">map</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">column_name_idx</span><span class="p">,</span> <span class="n">index_columns</span><span class="p">)</span>
+
+ <span class="k">if</span> <span class="n">nthreads</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
+ <span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">set_num_threads</span><span class="p">(</span><span class="n">nthreads</span><span class="p">)</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">read_all</span><span class="p">(</span><span class="n">column_indices</span><span class="o">=</span><span class="n">column_indices</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">_get_column_indices</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column_names</span><span class="p">):</span>
<span class="k">if</span> <span class="n">column_names</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<span class="k">return</span> <span class="kc">None</span>
- <span class="k">else</span><span class="p">:</span>
- <span class="k">return</span> <span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">column_name_idx</span><span class="p">(</span><span class="n">column</span><span class="p">)</span>
- <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">column_names</span><span class="p">]</span></div>
+ <span class="k">return</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">column_name_idx</span><span class="p">,</span> <span class="n">column_names</span><span class="p">))</span></div>
<span class="c1"># ----------------------------------------------------------------------</span>
@@ -304,8 +322,8 @@
<span class="c1"># manifest, so ['a', 'b', 'c'] as in our example above.</span>
<span class="n">dictionary</span> <span class="o">=</span> <span class="n">partitions</span><span class="o">.</span><span class="n">levels</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">dictionary</span>
- <span class="n">arr</span> <span class="o">=</span> <span class="n">_array</span><span class="o">.</span><span class="n">DictionaryArray</span><span class="o">.</span><span class="n">from_arrays</span><span class="p">(</span><span class="n">indices</span><span class="p">,</span> <span class="n">dictionary</span><span class="p">)</span>
- <span class="n">col</span> <span class="o">=</span> <span class="n">_table</span><span class="o">.</span><span class="n">Column</span><span class="o">.</span><span class="n">from_array</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">arr</span><span class="p">)</span>
+ <span class="n">arr</span> <span class="o">=</span> <span class="n">lib</span><span class="o">.</span><span class="n">DictionaryArray</span><span class="o">.</span><span class="n">from_arrays</span><span class="p">(</span><span class="n">indices</span><span class="p">,</span> <span class="n">dictionary</span><span class="p">)</span>
+ <span class="n">col</span> <span class="o">=</span> <span class="n">lib</span><span class="o">.</span><span class="n">Column</span><span class="o">.</span><span class="n">from_array</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">arr</span><span class="p">)</span>
<span class="n">table</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">append_column</span><span class="p">(</span><span class="n">col</span><span class="p">)</span>
<span class="k">return</span> <span class="n">table</span>
@@ -361,9 +379,9 @@
<span class="c1"># Only integer and string partition types are supported right now</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">integer_keys</span> <span class="o">=</span> <span class="p">[</span><span class="nb">int</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">keys</span><span class="p">]</span>
- <span class="n">dictionary</span> <span class="o">=</span> <span class="n">_array</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="n">integer_keys</span><span class="p">)</span>
+ <span class="n">dictionary</span> <span class="o">=</span> <span class="n">lib</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="n">integer_keys</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
- <span class="n">dictionary</span> <span class="o">=</span> <span class="n">_array</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">keys</span><span class="p">)</span>
+ <span class="n">dictionary</span> <span class="o">=</span> <span class="n">lib</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">keys</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_dictionary</span> <span class="o">=</span> <span class="n">dictionary</span>
<span class="k">return</span> <span class="n">dictionary</span>
@@ -602,7 +620,7 @@
<span class="n">open_file_func</span><span class="o">=</span><span class="n">open_file</span><span class="p">)</span>
<span class="n">tables</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
- <span class="n">all_data</span> <span class="o">=</span> <span class="n">_table</span><span class="o">.</span><span class="n">concat_tables</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span>
+ <span class="n">all_data</span> <span class="o">=</span> <span class="n">lib</span><span class="o">.</span><span class="n">concat_tables</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span>
<span class="k">return</span> <span class="n">all_data</span>
<span class="k">def</span> <span class="nf">_get_open_file_func</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
@@ -626,7 +644,7 @@
<span class="k">if</span> <span class="n">is_string</span><span class="p">(</span><span class="n">path_or_paths</span><span class="p">)</span> <span class="ow">and</span> <span class="n">fs</span><span class="o">.</span><span class="n">isdir</span><span class="p">(</span><span class="n">path_or_paths</span><span class="p">):</span>
<span class="n">manifest</span> <span class="o">=</span> <span class="n">ParquetManifest</span><span class="p">(</span><span class="n">path_or_paths</span><span class="p">,</span> <span class="n">filesystem</span><span class="o">=</span><span class="n">fs</span><span class="p">,</span>
- <span class="n">pathsep</span><span class="o">=</span><span class="n">pathsep</span><span class="p">)</span>
+ <span class="n">pathsep</span><span class="o">=</span><span class="n">fs</span><span class="o">.</span><span class="n">pathsep</span><span class="p">)</span>
<span class="n">metadata_path</span> <span class="o">=</span> <span class="n">manifest</span><span class="o">.</span><span class="n">metadata_path</span>
<span class="n">pieces</span> <span class="o">=</span> <span class="n">manifest</span><span class="o">.</span><span class="n">pieces</span>
<span class="n">partitions</span> <span class="o">=</span> <span class="n">manifest</span><span class="o">.</span><span class="n">partitions</span>
@@ -682,6 +700,43 @@
<span class="k">return</span> <span class="n">pf</span><span class="o">.</span><span class="n">read</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="n">columns</span><span class="p">,</span> <span class="n">nthreads</span><span class="o">=</span><span class="n">nthreads</span><span class="p">)</span></div>
+<span class="k">def</span> <span class="nf">read_pandas</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">columns</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">nthreads</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">metadata</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
+ <span class="sd">"""</span>
+<span class="sd"> Read a Table from Parquet format, reconstructing the index values if</span>
+<span class="sd"> available.</span>
+
+<span class="sd"> Parameters</span>
+<span class="sd"> ----------</span>
+<span class="sd"> source: str or pyarrow.io.NativeFile</span>
+<span class="sd"> Location of Parquet dataset. If a string passed, can be a single file</span>
+<span class="sd"> name. For passing Python file objects or byte buffers,</span>
+<span class="sd"> see pyarrow.io.PythonFileInterface or pyarrow.io.BufferReader.</span>
+<span class="sd"> columns: list</span>
+<span class="sd"> If not None, only these columns will be read from the file.</span>
+<span class="sd"> nthreads : int, default 1</span>
+<span class="sd"> Number of columns to read in parallel. Requires that the underlying</span>
+<span class="sd"> file source is threadsafe</span>
+<span class="sd"> metadata : FileMetaData</span>
+<span class="sd"> If separately computed</span>
+
+<span class="sd"> Returns</span>
+<span class="sd"> -------</span>
+<span class="sd"> pyarrow.Table</span>
+<span class="sd"> Content of the file as a Table of Columns, including DataFrame indexes</span>
+<span class="sd"> as Columns.</span>
+<span class="sd"> """</span>
+ <span class="k">if</span> <span class="n">is_string</span><span class="p">(</span><span class="n">source</span><span class="p">):</span>
+ <span class="n">fs</span> <span class="o">=</span> <span class="n">LocalFilesystem</span><span class="o">.</span><span class="n">get_instance</span><span class="p">()</span>
+ <span class="k">if</span> <span class="n">fs</span><span class="o">.</span><span class="n">isdir</span><span class="p">(</span><span class="n">source</span><span class="p">):</span>
+ <span class="k">raise</span> <span class="ne">NotImplementedError</span><span class="p">(</span>
+ <span class="s1">'Reading a directory of Parquet files with DataFrame index '</span>
+ <span class="s1">'metadata is not yet supported'</span>
+ <span class="p">)</span>
+
+ <span class="n">pf</span> <span class="o">=</span> <span class="n">ParquetFile</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">metadata</span><span class="o">=</span><span class="n">metadata</span><span class="p">)</span>
+ <span class="k">return</span> <span class="n">pf</span><span class="o">.</span><span class="n">read_pandas</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="n">columns</span><span class="p">,</span> <span class="n">nthreads</span><span class="o">=</span><span class="n">nthreads</span><span class="p">)</span>
+
+
<div class="viewcode-block" id="write_table"><a class="viewcode-back" href="../../generated/pyarrow.parquet.write_table.html#pyarrow.parquet.write_table">[docs]</a><span class="k">def</span> <span class="nf">write_table</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="n">where</span><span class="p">,</span> <span class="n">row_group_size</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">version</span><span class="o">=</span><span class="s1">'1.0'</span><span class="p">,</span>
<span class="n">use_dictionary</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">compression</span><span class="o">=</span><span class="s1">'snappy'</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="sd">"""</span>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/api.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/api.rst.txt b/docs/python/_sources/api.rst.txt
index a8dd8c5..c145935 100644
--- a/docs/python/_sources/api.rst.txt
+++ b/docs/python/_sources/api.rst.txt
@@ -162,7 +162,7 @@ Input / Output and Shared Memory
Buffer
BufferReader
- InMemoryOutputStream
+ BufferOutputStream
NativeFile
MemoryMappedFile
memory_map
@@ -177,10 +177,12 @@ Interprocess Communication and Messaging
.. autosummary::
:toctree: generated/
- FileReader
- FileWriter
- StreamReader
- StreamWriter
+ RecordBatchFileReader
+ RecordBatchFileWriter
+ RecordBatchStreamReader
+ RecordBatchStreamWriter
+ open_file
+ open_stream
.. _api.memory_pool:
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/development.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/development.rst.txt b/docs/python/_sources/development.rst.txt
index 440c1c4..6ec563e 100644
--- a/docs/python/_sources/development.rst.txt
+++ b/docs/python/_sources/development.rst.txt
@@ -174,14 +174,37 @@ You should be able to run the unit tests with:
Windows
=======
-First, make sure you can `build the C++ library <https://github.com/apache/arrow/blob/master/cpp/doc/Windows.md>`_.
+First, we bootstrap a conda environment similar to the `C++ build instructions
+<https://github.com/apache/arrow/blob/master/cpp/doc/Windows.md>`_. This
+includes all the dependencies for Arrow and the Apache Parquet C++ libraries.
-Now, we need to build and install the C++ libraries someplace.
+First, starting from fresh clones of Apache Arrow and parquet-cpp:
+
+.. code-block:: shell
+
+ git clone https://github.com/apache/arrow.git
+ git clone https://github.com/apache/parquet-cpp.git
+
+.. code-block:: shell
+
+ conda create -n arrow-dev cmake git boost-cpp ^
+ flatbuffers snappy zlib brotli thrift-cpp rapidjson
+ activate arrow-dev
+
+As one git housekeeping item, we must run this command in our Arrow clone:
+
+.. code-block:: shell
+
+ cd arrow
+ git config core.symlinks true
+
+Now, we build and install Arrow C++ libraries
.. code-block:: shell
mkdir cpp\build
cd cpp\build
+ set ARROW_BUILD_TOOLCHAIN=%CONDA_PREFIX%\Library
set ARROW_HOME=C:\thirdparty
cmake -G "Visual Studio 14 2015 Win64" ^
-DCMAKE_INSTALL_PREFIX=%ARROW_HOME% ^
@@ -191,6 +214,22 @@ Now, we need to build and install the C++ libraries someplace.
cmake --build . --target INSTALL --config Release
cd ..\..
+Now, we build parquet-cpp and install the result in the same place:
+
+.. code-block:: shell
+
+ mkdir ..\parquet-cpp\build
+ pushd ..\parquet-cpp\build
+ set PARQUET_BUILD_TOOLCHAIN=%CONDA_PREFIX%\Library
+ set PARQUET_HOME=C:\thirdparty
+ cmake -G "Visual Studio 14 2015 Win64" ^
+ -DCMAKE_INSTALL_PREFIX=%PARQUET_HOME% ^
+ -DCMAKE_BUILD_TYPE=Release ^
+ -DPARQUET_ZLIB_VENDORED=off ^
+ -DPARQUET_BUILD_TESTS=off ..
+ cmake --build . --target INSTALL --config Release
+ popd
+
After that, we must put the install directory's bin path in our ``%PATH%``:
.. code-block:: shell
@@ -202,7 +241,13 @@ Now, we can build pyarrow:
.. code-block:: shell
cd python
- python setup.py build_ext --inplace
+ python setup.py build_ext --inplace --with-parquet
+
+Then run the unit tests with:
+
+.. code-block:: shell
+
+ py.test pyarrow -v
Running C++ unit tests with Python
----------------------------------
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/generated/pyarrow.BufferOutputStream.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/generated/pyarrow.BufferOutputStream.rst.txt b/docs/python/_sources/generated/pyarrow.BufferOutputStream.rst.txt
new file mode 100644
index 0000000..fda3757
--- /dev/null
+++ b/docs/python/_sources/generated/pyarrow.BufferOutputStream.rst.txt
@@ -0,0 +1,31 @@
+pyarrow\.BufferOutputStream
+===========================
+
+.. currentmodule:: pyarrow
+
+.. autoclass:: BufferOutputStream
+
+
+ .. automethod:: __init__
+
+
+ .. rubric:: Methods
+
+ .. autosummary::
+
+ ~BufferOutputStream.close
+ ~BufferOutputStream.download
+ ~BufferOutputStream.get_result
+ ~BufferOutputStream.read
+ ~BufferOutputStream.read_buffer
+ ~BufferOutputStream.seek
+ ~BufferOutputStream.size
+ ~BufferOutputStream.tell
+ ~BufferOutputStream.upload
+ ~BufferOutputStream.write
+
+
+
+
+
+
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/generated/pyarrow.RecordBatchFileReader.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/generated/pyarrow.RecordBatchFileReader.rst.txt b/docs/python/_sources/generated/pyarrow.RecordBatchFileReader.rst.txt
new file mode 100644
index 0000000..9ea6fb2
--- /dev/null
+++ b/docs/python/_sources/generated/pyarrow.RecordBatchFileReader.rst.txt
@@ -0,0 +1,31 @@
+pyarrow\.RecordBatchFileReader
+==============================
+
+.. currentmodule:: pyarrow
+
+.. autoclass:: RecordBatchFileReader
+
+
+ .. automethod:: __init__
+
+
+ .. rubric:: Methods
+
+ .. autosummary::
+
+ ~RecordBatchFileReader.__init__
+ ~RecordBatchFileReader.get_batch
+ ~RecordBatchFileReader.get_record_batch
+ ~RecordBatchFileReader.read_all
+
+
+
+
+
+ .. rubric:: Attributes
+
+ .. autosummary::
+
+ ~RecordBatchFileReader.num_record_batches
+
+
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/generated/pyarrow.RecordBatchFileWriter.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/generated/pyarrow.RecordBatchFileWriter.rst.txt b/docs/python/_sources/generated/pyarrow.RecordBatchFileWriter.rst.txt
new file mode 100644
index 0000000..ea23f14
--- /dev/null
+++ b/docs/python/_sources/generated/pyarrow.RecordBatchFileWriter.rst.txt
@@ -0,0 +1,24 @@
+pyarrow\.RecordBatchFileWriter
+==============================
+
+.. currentmodule:: pyarrow
+
+.. autoclass:: RecordBatchFileWriter
+
+
+ .. automethod:: __init__
+
+
+ .. rubric:: Methods
+
+ .. autosummary::
+
+ ~RecordBatchFileWriter.__init__
+ ~RecordBatchFileWriter.close
+ ~RecordBatchFileWriter.write_batch
+
+
+
+
+
+
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/generated/pyarrow.RecordBatchStreamReader.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/generated/pyarrow.RecordBatchStreamReader.rst.txt b/docs/python/_sources/generated/pyarrow.RecordBatchStreamReader.rst.txt
new file mode 100644
index 0000000..35765b4
--- /dev/null
+++ b/docs/python/_sources/generated/pyarrow.RecordBatchStreamReader.rst.txt
@@ -0,0 +1,30 @@
+pyarrow\.RecordBatchStreamReader
+================================
+
+.. currentmodule:: pyarrow
+
+.. autoclass:: RecordBatchStreamReader
+
+
+ .. automethod:: __init__
+
+
+ .. rubric:: Methods
+
+ .. autosummary::
+
+ ~RecordBatchStreamReader.__init__
+ ~RecordBatchStreamReader.get_next_batch
+ ~RecordBatchStreamReader.read_all
+
+
+
+
+
+ .. rubric:: Attributes
+
+ .. autosummary::
+
+ ~RecordBatchStreamReader.schema
+
+
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/generated/pyarrow.RecordBatchStreamWriter.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/generated/pyarrow.RecordBatchStreamWriter.rst.txt b/docs/python/_sources/generated/pyarrow.RecordBatchStreamWriter.rst.txt
new file mode 100644
index 0000000..1cf4cfe
--- /dev/null
+++ b/docs/python/_sources/generated/pyarrow.RecordBatchStreamWriter.rst.txt
@@ -0,0 +1,24 @@
+pyarrow\.RecordBatchStreamWriter
+================================
+
+.. currentmodule:: pyarrow
+
+.. autoclass:: RecordBatchStreamWriter
+
+
+ .. automethod:: __init__
+
+
+ .. rubric:: Methods
+
+ .. autosummary::
+
+ ~RecordBatchStreamWriter.__init__
+ ~RecordBatchStreamWriter.close
+ ~RecordBatchStreamWriter.write_batch
+
+
+
+
+
+
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/generated/pyarrow.Schema.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/generated/pyarrow.Schema.rst.txt b/docs/python/_sources/generated/pyarrow.Schema.rst.txt
index 280d4c1..436e84b 100644
--- a/docs/python/_sources/generated/pyarrow.Schema.rst.txt
+++ b/docs/python/_sources/generated/pyarrow.Schema.rst.txt
@@ -16,6 +16,7 @@ pyarrow\.Schema
~Schema.add_metadata
~Schema.equals
~Schema.field_by_name
+ ~Schema.get_field_index
~Schema.remove_metadata
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/generated/pyarrow.open_file.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/generated/pyarrow.open_file.rst.txt b/docs/python/_sources/generated/pyarrow.open_file.rst.txt
new file mode 100644
index 0000000..837dff6
--- /dev/null
+++ b/docs/python/_sources/generated/pyarrow.open_file.rst.txt
@@ -0,0 +1,6 @@
+pyarrow\.open\_file
+===================
+
+.. currentmodule:: pyarrow
+
+.. autofunction:: open_file
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/generated/pyarrow.open_stream.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/generated/pyarrow.open_stream.rst.txt b/docs/python/_sources/generated/pyarrow.open_stream.rst.txt
new file mode 100644
index 0000000..dd27862
--- /dev/null
+++ b/docs/python/_sources/generated/pyarrow.open_stream.rst.txt
@@ -0,0 +1,6 @@
+pyarrow\.open\_stream
+=====================
+
+.. currentmodule:: pyarrow
+
+.. autofunction:: open_stream
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/generated/pyarrow.parquet.ParquetFile.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/generated/pyarrow.parquet.ParquetFile.rst.txt b/docs/python/_sources/generated/pyarrow.parquet.ParquetFile.rst.txt
index b35af86..3dd4e20 100644
--- a/docs/python/_sources/generated/pyarrow.parquet.ParquetFile.rst.txt
+++ b/docs/python/_sources/generated/pyarrow.parquet.ParquetFile.rst.txt
@@ -15,6 +15,7 @@ pyarrow\.parquet\.ParquetFile
~ParquetFile.__init__
~ParquetFile.read
+ ~ParquetFile.read_pandas
~ParquetFile.read_row_group
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/ipc.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/ipc.rst.txt b/docs/python/_sources/ipc.rst.txt
index e63e745..f0844cd 100644
--- a/docs/python/_sources/ipc.rst.txt
+++ b/docs/python/_sources/ipc.rst.txt
@@ -55,13 +55,13 @@ First, let's create a small record batch:
batch.num_columns
Now, we can begin writing a stream containing some number of these batches. For
-this we use :class:`~pyarrow.StreamWriter`, which can write to a writeable
+this we use :class:`~pyarrow.RecordBatchStreamWriter`, which can write to a writeable
``NativeFile`` object or a writeable Python object:
.. ipython:: python
- sink = pa.InMemoryOutputStream()
- writer = pa.StreamWriter(sink, batch.schema)
+ sink = pa.BufferOutputStream()
+ writer = pa.RecordBatchStreamWriter(sink, batch.schema)
Here we used an in-memory Arrow buffer stream, but this could have been a
socket or some other IO sink.
@@ -80,11 +80,12 @@ particular stream. Now we can do:
buf.size
Now ``buf`` contains the complete stream as an in-memory byte buffer. We can
-read such a stream with :class:`~pyarrow.StreamReader`:
+read such a stream with :class:`~pyarrow.RecordBatchStreamReader` or the
+convenience function ``pyarrow.open_stream``:
.. ipython:: python
- reader = pa.StreamReader(buf)
+ reader = pa.open_stream(buf)
reader.schema
batches = [b for b in reader]
@@ -103,13 +104,13 @@ batches are also zero-copy and do not allocate any new memory on read.
Writing and Reading Random Access Files
---------------------------------------
-The :class:`~pyarrow.FileWriter` has the same API as
-:class:`~pyarrow.StreamWriter`:
+The :class:`~pyarrow.RecordBatchFileWriter` has the same API as
+:class:`~pyarrow.RecordBatchStreamWriter`:
.. ipython:: python
- sink = pa.InMemoryOutputStream()
- writer = pa.FileWriter(sink, batch.schema)
+ sink = pa.BufferOutputStream()
+ writer = pa.RecordBatchFileWriter(sink, batch.schema)
for i in range(10):
writer.write_batch(batch)
@@ -118,13 +119,14 @@ The :class:`~pyarrow.FileWriter` has the same API as
buf = sink.get_result()
buf.size
-The difference between :class:`~pyarrow.FileReader` and
-:class:`~pyarrow.StreamReader` is that the input source must have a ``seek``
-method for random access. The stream reader only requires read operations:
+The difference between :class:`~pyarrow.RecordBatchFileReader` and
+:class:`~pyarrow.RecordBatchStreamReader` is that the input source must have a
+``seek`` method for random access. The stream reader only requires read
+operations. We can also use the ``pyarrow.open_file`` method to open a file:
.. ipython:: python
- reader = pa.FileReader(buf)
+ reader = pa.open_file(buf)
Because we have access to the entire payload, we know the number of record
batches in the file, and can read any at random:
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/_sources/memory.rst.txt
----------------------------------------------------------------------
diff --git a/docs/python/_sources/memory.rst.txt b/docs/python/_sources/memory.rst.txt
index d1020da..ccc6298 100644
--- a/docs/python/_sources/memory.rst.txt
+++ b/docs/python/_sources/memory.rst.txt
@@ -104,8 +104,8 @@ There are several kinds of :class:`~pyarrow.NativeFile` options available:
memory maps
* :class:`~pyarrow.BufferReader`, for reading :class:`~pyarrow.Buffer` objects
as a file
-* :class:`~pyarrow.InMemoryOutputStream`, for writing data in-memory, producing
- a Buffer at the end
+* :class:`~pyarrow.BufferOutputStream`, for writing data in-memory, producing a
+ Buffer at the end
* :class:`~pyarrow.HdfsFile`, for reading and writing data to the Hadoop Filesystem
* :class:`~pyarrow.PythonFile`, for interfacing with Python file objects in C++
@@ -124,11 +124,11 @@ then precisely track amount of memory that has been allocated:
PyArrow uses a default built-in memory pool, but in the future there may be
additional memory pools (and subpools) to choose from. Let's consider an
-``InMemoryOutputStream``, which is like a ``BytesIO``:
+``BufferOutputStream``, which is like a ``BytesIO``:
.. ipython:: python
- stream = pa.InMemoryOutputStream()
+ stream = pa.BufferOutputStream()
stream.write(b'foo')
pa.total_allocated_bytes()
for i in range(1024): stream.write(b'foo')
@@ -150,7 +150,7 @@ pass in a custom memory pool:
my_pool = pa.jemalloc_memory_pool()
my_pool
my_pool.bytes_allocated()
- stream = pa.InMemoryOutputStream(my_pool)
+ stream = pa.BufferOutputStream(my_pool)
stream.write(b'foo')
my_pool.bytes_allocated()
@@ -215,7 +215,7 @@ file interfaces that can read and write to Arrow Buffers.
.. ipython:: python
- writer = pa.InMemoryOutputStream()
+ writer = pa.BufferOutputStream()
writer.write(b'hello, friends')
buf = writer.get_result()
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/api.html
----------------------------------------------------------------------
diff --git a/docs/python/api.html b/docs/python/api.html
index 9043e13..f98ee48 100644
--- a/docs/python/api.html
+++ b/docs/python/api.html
@@ -422,7 +422,7 @@
<tr class="row-even"><td><a class="reference internal" href="generated/pyarrow.BufferReader.html#pyarrow.BufferReader" title="pyarrow.BufferReader"><code class="xref py py-obj docutils literal"><span class="pre">BufferReader</span></code></a></td>
<td>Zero-copy reader from objects convertible to Arrow buffer</td>
</tr>
-<tr class="row-odd"><td><a class="reference internal" href="generated/pyarrow.InMemoryOutputStream.html#pyarrow.InMemoryOutputStream" title="pyarrow.InMemoryOutputStream"><code class="xref py py-obj docutils literal"><span class="pre">InMemoryOutputStream</span></code></a></td>
+<tr class="row-odd"><td><a class="reference internal" href="generated/pyarrow.BufferOutputStream.html#pyarrow.BufferOutputStream" title="pyarrow.BufferOutputStream"><code class="xref py py-obj docutils literal"><span class="pre">BufferOutputStream</span></code></a></td>
<td></td>
</tr>
<tr class="row-even"><td><a class="reference internal" href="generated/pyarrow.NativeFile.html#pyarrow.NativeFile" title="pyarrow.NativeFile"><code class="xref py py-obj docutils literal"><span class="pre">NativeFile</span></code></a></td>
@@ -451,18 +451,24 @@
<col width="90%" />
</colgroup>
<tbody valign="top">
-<tr class="row-odd"><td><a class="reference internal" href="generated/pyarrow.FileReader.html#pyarrow.FileReader" title="pyarrow.FileReader"><code class="xref py py-obj docutils literal"><span class="pre">FileReader</span></code></a>(source[, footer_offset])</td>
+<tr class="row-odd"><td><a class="reference internal" href="generated/pyarrow.RecordBatchFileReader.html#pyarrow.RecordBatchFileReader" title="pyarrow.RecordBatchFileReader"><code class="xref py py-obj docutils literal"><span class="pre">RecordBatchFileReader</span></code></a>(source[, footer_offset])</td>
<td>Class for reading Arrow record batch data from the Arrow binary file format</td>
</tr>
-<tr class="row-even"><td><a class="reference internal" href="generated/pyarrow.FileWriter.html#pyarrow.FileWriter" title="pyarrow.FileWriter"><code class="xref py py-obj docutils literal"><span class="pre">FileWriter</span></code></a>(sink, schema)</td>
+<tr class="row-even"><td><a class="reference internal" href="generated/pyarrow.RecordBatchFileWriter.html#pyarrow.RecordBatchFileWriter" title="pyarrow.RecordBatchFileWriter"><code class="xref py py-obj docutils literal"><span class="pre">RecordBatchFileWriter</span></code></a>(sink, schema)</td>
<td>Writer to create the Arrow binary file format</td>
</tr>
-<tr class="row-odd"><td><a class="reference internal" href="generated/pyarrow.StreamReader.html#pyarrow.StreamReader" title="pyarrow.StreamReader"><code class="xref py py-obj docutils literal"><span class="pre">StreamReader</span></code></a>(source)</td>
+<tr class="row-odd"><td><a class="reference internal" href="generated/pyarrow.RecordBatchStreamReader.html#pyarrow.RecordBatchStreamReader" title="pyarrow.RecordBatchStreamReader"><code class="xref py py-obj docutils literal"><span class="pre">RecordBatchStreamReader</span></code></a>(source)</td>
<td>Reader for the Arrow streaming binary format</td>
</tr>
-<tr class="row-even"><td><a class="reference internal" href="generated/pyarrow.StreamWriter.html#pyarrow.StreamWriter" title="pyarrow.StreamWriter"><code class="xref py py-obj docutils literal"><span class="pre">StreamWriter</span></code></a>(sink, schema)</td>
+<tr class="row-even"><td><a class="reference internal" href="generated/pyarrow.RecordBatchStreamWriter.html#pyarrow.RecordBatchStreamWriter" title="pyarrow.RecordBatchStreamWriter"><code class="xref py py-obj docutils literal"><span class="pre">RecordBatchStreamWriter</span></code></a>(sink, schema)</td>
<td>Writer for the Arrow streaming binary format</td>
</tr>
+<tr class="row-odd"><td><a class="reference internal" href="generated/pyarrow.open_file.html#pyarrow.open_file" title="pyarrow.open_file"><code class="xref py py-obj docutils literal"><span class="pre">open_file</span></code></a>(source[, footer_offset])</td>
+<td>Create reader for Arrow file format</td>
+</tr>
+<tr class="row-even"><td><a class="reference internal" href="generated/pyarrow.open_stream.html#pyarrow.open_stream" title="pyarrow.open_stream"><code class="xref py py-obj docutils literal"><span class="pre">open_stream</span></code></a>(source)</td>
+<td>Create reader for Arrow streaming format</td>
+</tr>
</tbody>
</table>
</div>
[3/4] arrow-site git commit: Update Python documentation
Posted by we...@apache.org.
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/data.html
----------------------------------------------------------------------
diff --git a/docs/python/data.html b/docs/python/data.html
index e16f145..99ed571 100644
--- a/docs/python/data.html
+++ b/docs/python/data.html
@@ -241,7 +241,7 @@ the <code class="docutils literal"><span class="pre">numpy.array</span></code> f
<span class="gp">In [24]: </span><span class="n">arr</span>
<span class="gh">Out[24]: </span><span class="go"></span>
-<span class="go"><pyarrow._array.Int64Array object at 0x2b0bcdd11318></span>
+<span class="go"><pyarrow.lib.Int64Array object at 0x7fce90483278></span>
<span class="go">[</span>
<span class="go"> 1,</span>
<span class="go"> 2,</span>
@@ -306,7 +306,7 @@ consider an example:</p>
<span class="gp">In [36]: </span><span class="n">dict_array</span>
<span class="gh">Out[36]: </span><span class="go"></span>
-<span class="go"><pyarrow._array.DictionaryArray object at 0x2b0bcdd19588></span>
+<span class="go"><pyarrow.lib.DictionaryArray object at 0x7fce90480768></span>
<span class="go">[</span>
<span class="go"> 'foo',</span>
<span class="go"> 'bar',</span>
@@ -325,7 +325,7 @@ consider an example:</p>
<span class="gp">In [38]: </span><span class="n">dict_array</span><span class="o">.</span><span class="n">indices</span>
<span class="go"> Out[38]: </span>
-<span class="go"><pyarrow._array.Int64Array object at 0x2b0bcdd29908></span>
+<span class="go"><pyarrow.lib.Int64Array object at 0x7fce904914f8></span>
<span class="go">[</span>
<span class="go"> 0,</span>
<span class="go"> 1,</span>
@@ -338,8 +338,8 @@ consider an example:</p>
<span class="go">]</span>
<span class="gp">In [39]: </span><span class="n">dict_array</span><span class="o">.</span><span class="n">dictionary</span>
-<span class="go"> Out[39]: </span>
-<span class="go"><pyarrow._array.StringArray object at 0x2b0bcdd29818></span>
+<span class="go"> Out[39]: </span>
+<span class="go"><pyarrow.lib.StringArray object at 0x7fce90491408></span>
<span class="go">[</span>
<span class="go"> 'foo',</span>
<span class="go"> 'bar',</span>
@@ -384,11 +384,10 @@ instances. Let’s consider a collection of arrays:</p>
<span class="go">f0: int64</span>
<span class="go">f1: string</span>
<span class="go">f2: bool</span>
-<span class="go">-- metadata --</span>
<span class="gp">In [46]: </span><span class="n">batch</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
-<span class="go"> Out[46]: </span>
-<span class="go"><pyarrow._array.StringArray object at 0x2b0bcdd36f98></span>
+<span class="go"> Out[46]: </span>
+<span class="go"><pyarrow.lib.StringArray object at 0x7fce904a6638></span>
<span class="go">[</span>
<span class="go"> 'foo',</span>
<span class="go"> 'bar',</span>
@@ -402,7 +401,7 @@ instances. Let’s consider a collection of arrays:</p>
<span class="gp">In [48]: </span><span class="n">batch2</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="gh">Out[48]: </span><span class="go"></span>
-<span class="go"><pyarrow._array.StringArray object at 0x2b0bcdd36ea8></span>
+<span class="go"><pyarrow.lib.StringArray object at 0x7fce904b9138></span>
<span class="go">[</span>
<span class="go"> 'bar',</span>
<span class="go"> 'baz',</span>
@@ -431,10 +430,9 @@ one or more copies of the batch using <code class="docutils literal"><span class
<span class="go">f0: int64</span>
<span class="go">f1: string</span>
<span class="go">f2: bool</span>
-<span class="go">-- metadata --</span>
<span class="gp">In [52]: </span><span class="n">table</span><span class="o">.</span><span class="n">num_rows</span>
-<span class="go"> Out[52]: 20</span>
+<span class="go"> Out[52]: 20</span>
</pre></div>
</div>
<p>The table’s columns are instances of <a class="reference internal" href="generated/pyarrow.Column.html#pyarrow.Column" title="pyarrow.Column"><code class="xref py py-class docutils literal"><span class="pre">Column</span></code></a>, which is a container
@@ -442,17 +440,17 @@ for one or more arrays of the same type.</p>
<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [53]: </span><span class="n">c</span> <span class="o">=</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="gp">In [54]: </span><span class="n">c</span>
-<span class="gh">Out[54]: </span><span class="go"><pyarrow._table.Column at 0x2b0bcda54f00></span>
+<span class="gh">Out[54]: </span><span class="go"><pyarrow.lib.Column at 0x7fce8f9e3e10></span>
<span class="gp">In [55]: </span><span class="n">c</span><span class="o">.</span><span class="n">data</span>
-<span class="go"> Out[55]: <pyarrow._table.ChunkedArray at 0x2b0bcda54ed0></span>
+<span class="go"> Out[55]: <pyarrow.lib.ChunkedArray at 0x7fce8f9e3c00></span>
<span class="gp">In [56]: </span><span class="n">c</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">num_chunks</span>
-<span class="go"> Out[56]: 5</span>
+<span class="go"> Out[56]: 5</span>
<span class="gp">In [57]: </span><span class="n">c</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">chunk</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
-<span class="go"> Out[57]: </span>
-<span class="go"><pyarrow._array.Int64Array object at 0x2b0bcdd4b3b8></span>
+<span class="go"> Out[57]: </span>
+<span class="go"><pyarrow.lib.Int64Array object at 0x7fce904b9818></span>
<span class="go">[</span>
<span class="go"> 1,</span>
<span class="go"> 2,</span>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/development.html
----------------------------------------------------------------------
diff --git a/docs/python/development.html b/docs/python/development.html
index bfc4110..2146146 100644
--- a/docs/python/development.html
+++ b/docs/python/development.html
@@ -226,10 +226,27 @@ pyarrow/tests/test_tensor.py ................
</div>
<div class="section" id="windows">
<h2>Windows<a class="headerlink" href="#windows" title="Permalink to this headline">¶</a></h2>
-<p>First, make sure you can <a class="reference external" href="https://github.com/apache/arrow/blob/master/cpp/doc/Windows.md">build the C++ library</a>.</p>
-<p>Now, we need to build and install the C++ libraries someplace.</p>
+<p>First, we bootstrap a conda environment similar to the <a class="reference external" href="https://github.com/apache/arrow/blob/master/cpp/doc/Windows.md">C++ build instructions</a>. This
+includes all the dependencies for Arrow and the Apache Parquet C++ libraries.</p>
+<p>First, starting from fresh clones of Apache Arrow and parquet-cpp:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>git clone https://github.com/apache/arrow.git
+git clone https://github.com/apache/parquet-cpp.git
+</pre></div>
+</div>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>conda create -n arrow-dev cmake git boost-cpp ^
+ flatbuffers snappy zlib brotli thrift-cpp rapidjson
+activate arrow-dev
+</pre></div>
+</div>
+<p>As one git housekeeping item, we must run this command in our Arrow clone:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="nb">cd</span> arrow
+git config core.symlinks <span class="nb">true</span>
+</pre></div>
+</div>
+<p>Now, we build and install Arrow C++ libraries</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span>mkdir cpp<span class="se">\b</span>uild
<span class="nb">cd</span> cpp<span class="se">\b</span>uild
+<span class="nb">set</span> <span class="nv">ARROW_BUILD_TOOLCHAIN</span><span class="o">=</span>%CONDA_PREFIX%<span class="se">\L</span>ibrary
<span class="nb">set</span> <span class="nv">ARROW_HOME</span><span class="o">=</span>C:<span class="se">\t</span>hirdparty
cmake -G <span class="s2">"Visual Studio 14 2015 Win64"</span> ^
-DCMAKE_INSTALL_PREFIX<span class="o">=</span>%ARROW_HOME% ^
@@ -240,13 +257,31 @@ cmake --build . --target INSTALL --config Release
<span class="nb">cd</span> ..<span class="se">\.</span>.
</pre></div>
</div>
+<p>Now, we build parquet-cpp and install the result in the same place:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>mkdir ..<span class="se">\p</span>arquet-cpp<span class="se">\b</span>uild
+<span class="nb">pushd</span> ..<span class="se">\p</span>arquet-cpp<span class="se">\b</span>uild
+<span class="nb">set</span> <span class="nv">PARQUET_BUILD_TOOLCHAIN</span><span class="o">=</span>%CONDA_PREFIX%<span class="se">\L</span>ibrary
+<span class="nb">set</span> <span class="nv">PARQUET_HOME</span><span class="o">=</span>C:<span class="se">\t</span>hirdparty
+cmake -G <span class="s2">"Visual Studio 14 2015 Win64"</span> ^
+ -DCMAKE_INSTALL_PREFIX<span class="o">=</span>%PARQUET_HOME% ^
+ -DCMAKE_BUILD_TYPE<span class="o">=</span>Release ^
+ -DPARQUET_ZLIB_VENDORED<span class="o">=</span>off ^
+ -DPARQUET_BUILD_TESTS<span class="o">=</span>off ..
+cmake --build . --target INSTALL --config Release
+<span class="nb">popd</span>
+</pre></div>
+</div>
<p>After that, we must put the install directory’s bin path in our <code class="docutils literal"><span class="pre">%PATH%</span></code>:</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="nb">set</span> <span class="nv">PATH</span><span class="o">=</span>%ARROW_HOME%<span class="se">\b</span>in<span class="p">;</span>%PATH%
</pre></div>
</div>
<p>Now, we can build pyarrow:</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="nb">cd</span> python
-python setup.py build_ext --inplace
+python setup.py build_ext --inplace --with-parquet
+</pre></div>
+</div>
+<p>Then run the unit tests with:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>py.test pyarrow -v
</pre></div>
</div>
<div class="section" id="running-c-unit-tests-with-python">
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.BufferOutputStream.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.BufferOutputStream.html b/docs/python/generated/pyarrow.BufferOutputStream.html
new file mode 100644
index 0000000..25f5794
--- /dev/null
+++ b/docs/python/generated/pyarrow.BufferOutputStream.html
@@ -0,0 +1,164 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+
+ <title>pyarrow.BufferOutputStream — pyarrow documentation</title>
+
+ <link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
+ <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT: '../',
+ VERSION: '',
+ COLLAPSE_INDEX: false,
+ FILE_SUFFIX: '.html',
+ HAS_SOURCE: true,
+ SOURCELINK_SUFFIX: '.txt'
+ };
+ </script>
+ <script type="text/javascript" src="../_static/jquery.js"></script>
+ <script type="text/javascript" src="../_static/underscore.js"></script>
+ <script type="text/javascript" src="../_static/doctools.js"></script>
+ <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <link rel="index" title="Index" href="../genindex.html" />
+ <link rel="search" title="Search" href="../search.html" />
+ <link rel="next" title="pyarrow.NativeFile" href="pyarrow.NativeFile.html" />
+ <link rel="prev" title="pyarrow.BufferReader" href="pyarrow.BufferReader.html" />
+ </head>
+ <body role="document">
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ accesskey="I">index</a></li>
+ <li class="right" >
+ <a href="pyarrow.NativeFile.html" title="pyarrow.NativeFile"
+ accesskey="N">next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.BufferReader.html" title="pyarrow.BufferReader"
+ accesskey="P">previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" accesskey="U">API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+ <div class="sphinxsidebarwrapper">
+ <h4>Previous topic</h4>
+ <p class="topless"><a href="pyarrow.BufferReader.html"
+ title="previous chapter">pyarrow.BufferReader</a></p>
+ <h4>Next topic</h4>
+ <p class="topless"><a href="pyarrow.NativeFile.html"
+ title="next chapter">pyarrow.NativeFile</a></p>
+ <div role="note" aria-label="source link">
+ <h3>This Page</h3>
+ <ul class="this-page-menu">
+ <li><a href="../_sources/generated/pyarrow.BufferOutputStream.rst.txt"
+ rel="nofollow">Show Source</a></li>
+ </ul>
+ </div>
+<div id="searchbox" style="display: none" role="search">
+ <h3>Quick search</h3>
+ <form class="search" action="../search.html" method="get">
+ <div><input type="text" name="q" /></div>
+ <div><input type="submit" value="Go" /></div>
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+ </div>
+ </div>
+
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body" role="main">
+
+ <div class="section" id="pyarrow-bufferoutputstream">
+<h1>pyarrow.BufferOutputStream<a class="headerlink" href="#pyarrow-bufferoutputstream" title="Permalink to this headline">¶</a></h1>
+<dl class="class">
+<dt id="pyarrow.BufferOutputStream">
+<em class="property">class </em><code class="descclassname">pyarrow.</code><code class="descname">BufferOutputStream</code><a class="headerlink" href="#pyarrow.BufferOutputStream" title="Permalink to this definition">¶</a></dt>
+<dd><dl class="method">
+<dt id="pyarrow.BufferOutputStream.__init__">
+<code class="descname">__init__</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyarrow.BufferOutputStream.__init__" title="Permalink to this definition">¶</a></dt>
+<dd><p>Initialize self. See help(type(self)) for accurate signature.</p>
+</dd></dl>
+
+<p class="rubric">Methods</p>
+<table border="1" class="longtable docutils">
+<colgroup>
+<col width="10%" />
+<col width="90%" />
+</colgroup>
+<tbody valign="top">
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">close</span></code>(self)</td>
+<td></td>
+</tr>
+<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">download</span></code>(self, stream_or_path[, buffer_size])</td>
+<td>Read file completely to local path (rather than reading completely into memory).</td>
+</tr>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">get_result</span></code>(self)</td>
+<td></td>
+</tr>
+<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">read</span></code>(self[, nbytes])</td>
+<td></td>
+</tr>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">read_buffer</span></code>(self[, nbytes])</td>
+<td></td>
+</tr>
+<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">seek</span></code>(self, int64_t position)</td>
+<td></td>
+</tr>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">size</span></code>(self)</td>
+<td></td>
+</tr>
+<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">tell</span></code>(self)</td>
+<td></td>
+</tr>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">upload</span></code>(self, stream[, buffer_size])</td>
+<td>Pipe file-like object to file</td>
+</tr>
+<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">write</span></code>(self, data)</td>
+<td>Write byte from any object implementing buffer protocol (bytes,</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</div>
+
+
+ </div>
+ </div>
+ </div>
+ <div class="clearer"></div>
+ </div>
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ >index</a></li>
+ <li class="right" >
+ <a href="pyarrow.NativeFile.html" title="pyarrow.NativeFile"
+ >next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.BufferReader.html" title="pyarrow.BufferReader"
+ >previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" >API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="footer" role="contentinfo">
+ © Copyright 2016-2017 Apache Software Foundation.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.
+ </div>
+ </body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.BufferReader.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.BufferReader.html b/docs/python/generated/pyarrow.BufferReader.html
index 197f273..e2b2615 100644
--- a/docs/python/generated/pyarrow.BufferReader.html
+++ b/docs/python/generated/pyarrow.BufferReader.html
@@ -27,7 +27,7 @@
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="pyarrow.InMemoryOutputStream" href="pyarrow.InMemoryOutputStream.html" />
+ <link rel="next" title="pyarrow.BufferOutputStream" href="pyarrow.BufferOutputStream.html" />
<link rel="prev" title="pyarrow.Buffer" href="pyarrow.Buffer.html" />
</head>
<body role="document">
@@ -38,7 +38,7 @@
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
- <a href="pyarrow.InMemoryOutputStream.html" title="pyarrow.InMemoryOutputStream"
+ <a href="pyarrow.BufferOutputStream.html" title="pyarrow.BufferOutputStream"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="pyarrow.Buffer.html" title="pyarrow.Buffer"
@@ -53,8 +53,8 @@
<p class="topless"><a href="pyarrow.Buffer.html"
title="previous chapter">pyarrow.Buffer</a></p>
<h4>Next topic</h4>
- <p class="topless"><a href="pyarrow.InMemoryOutputStream.html"
- title="next chapter">pyarrow.InMemoryOutputStream</a></p>
+ <p class="topless"><a href="pyarrow.BufferOutputStream.html"
+ title="next chapter">pyarrow.BufferOutputStream</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
@@ -90,7 +90,7 @@
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
-<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>obj</strong> (<em>Python bytes</em><em> or </em><em>pyarrow.io.Buffer</em>) – </td>
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>obj</strong> (<em>Python bytes</em><em> or </em><a class="reference internal" href="pyarrow.Buffer.html#pyarrow.Buffer" title="pyarrow.Buffer"><em>pyarrow.Buffer</em></a>) – </td>
</tr>
</tbody>
</table>
@@ -153,7 +153,7 @@
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
- <a href="pyarrow.InMemoryOutputStream.html" title="pyarrow.InMemoryOutputStream"
+ <a href="pyarrow.BufferOutputStream.html" title="pyarrow.BufferOutputStream"
>next</a> |</li>
<li class="right" >
<a href="pyarrow.Buffer.html" title="pyarrow.Buffer"
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.MemoryPool.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.MemoryPool.html b/docs/python/generated/pyarrow.MemoryPool.html
index 8cc0c7c..a43b5bb 100644
--- a/docs/python/generated/pyarrow.MemoryPool.html
+++ b/docs/python/generated/pyarrow.MemoryPool.html
@@ -28,7 +28,7 @@
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="pyarrow.default_memory_pool" href="pyarrow.default_memory_pool.html" />
- <link rel="prev" title="pyarrow.StreamWriter" href="pyarrow.StreamWriter.html" />
+ <link rel="prev" title="pyarrow.open_stream" href="pyarrow.open_stream.html" />
</head>
<body role="document">
<div class="related" role="navigation" aria-label="related navigation">
@@ -41,7 +41,7 @@
<a href="pyarrow.default_memory_pool.html" title="pyarrow.default_memory_pool"
accesskey="N">next</a> |</li>
<li class="right" >
- <a href="pyarrow.StreamWriter.html" title="pyarrow.StreamWriter"
+ <a href="pyarrow.open_stream.html" title="pyarrow.open_stream"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="../api.html" accesskey="U">API Reference</a> »</li>
@@ -50,8 +50,8 @@
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h4>Previous topic</h4>
- <p class="topless"><a href="pyarrow.StreamWriter.html"
- title="previous chapter">pyarrow.StreamWriter</a></p>
+ <p class="topless"><a href="pyarrow.open_stream.html"
+ title="previous chapter">pyarrow.open_stream</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="pyarrow.default_memory_pool.html"
title="next chapter">pyarrow.default_memory_pool</a></p>
@@ -123,7 +123,7 @@
<a href="pyarrow.default_memory_pool.html" title="pyarrow.default_memory_pool"
>next</a> |</li>
<li class="right" >
- <a href="pyarrow.StreamWriter.html" title="pyarrow.StreamWriter"
+ <a href="pyarrow.open_stream.html" title="pyarrow.open_stream"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="../api.html" >API Reference</a> »</li>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.NativeFile.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.NativeFile.html b/docs/python/generated/pyarrow.NativeFile.html
index 021ffa3..ef1209d 100644
--- a/docs/python/generated/pyarrow.NativeFile.html
+++ b/docs/python/generated/pyarrow.NativeFile.html
@@ -28,7 +28,7 @@
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="pyarrow.MemoryMappedFile" href="pyarrow.MemoryMappedFile.html" />
- <link rel="prev" title="pyarrow.InMemoryOutputStream" href="pyarrow.InMemoryOutputStream.html" />
+ <link rel="prev" title="pyarrow.BufferOutputStream" href="pyarrow.BufferOutputStream.html" />
</head>
<body role="document">
<div class="related" role="navigation" aria-label="related navigation">
@@ -41,7 +41,7 @@
<a href="pyarrow.MemoryMappedFile.html" title="pyarrow.MemoryMappedFile"
accesskey="N">next</a> |</li>
<li class="right" >
- <a href="pyarrow.InMemoryOutputStream.html" title="pyarrow.InMemoryOutputStream"
+ <a href="pyarrow.BufferOutputStream.html" title="pyarrow.BufferOutputStream"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="../api.html" accesskey="U">API Reference</a> »</li>
@@ -50,8 +50,8 @@
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h4>Previous topic</h4>
- <p class="topless"><a href="pyarrow.InMemoryOutputStream.html"
- title="previous chapter">pyarrow.InMemoryOutputStream</a></p>
+ <p class="topless"><a href="pyarrow.BufferOutputStream.html"
+ title="previous chapter">pyarrow.BufferOutputStream</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="pyarrow.MemoryMappedFile.html"
title="next chapter">pyarrow.MemoryMappedFile</a></p>
@@ -147,7 +147,7 @@
<a href="pyarrow.MemoryMappedFile.html" title="pyarrow.MemoryMappedFile"
>next</a> |</li>
<li class="right" >
- <a href="pyarrow.InMemoryOutputStream.html" title="pyarrow.InMemoryOutputStream"
+ <a href="pyarrow.BufferOutputStream.html" title="pyarrow.BufferOutputStream"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="../api.html" >API Reference</a> »</li>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.PythonFile.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.PythonFile.html b/docs/python/generated/pyarrow.PythonFile.html
index 683a695..b2d4ee0 100644
--- a/docs/python/generated/pyarrow.PythonFile.html
+++ b/docs/python/generated/pyarrow.PythonFile.html
@@ -27,7 +27,7 @@
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="pyarrow.FileReader" href="pyarrow.FileReader.html" />
+ <link rel="next" title="pyarrow.RecordBatchFileReader" href="pyarrow.RecordBatchFileReader.html" />
<link rel="prev" title="pyarrow.create_memory_map" href="pyarrow.create_memory_map.html" />
</head>
<body role="document">
@@ -38,7 +38,7 @@
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
- <a href="pyarrow.FileReader.html" title="pyarrow.FileReader"
+ <a href="pyarrow.RecordBatchFileReader.html" title="pyarrow.RecordBatchFileReader"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="pyarrow.create_memory_map.html" title="pyarrow.create_memory_map"
@@ -53,8 +53,8 @@
<p class="topless"><a href="pyarrow.create_memory_map.html"
title="previous chapter">pyarrow.create_memory_map</a></p>
<h4>Next topic</h4>
- <p class="topless"><a href="pyarrow.FileReader.html"
- title="next chapter">pyarrow.FileReader</a></p>
+ <p class="topless"><a href="pyarrow.RecordBatchFileReader.html"
+ title="next chapter">pyarrow.RecordBatchFileReader</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
@@ -144,7 +144,7 @@
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
- <a href="pyarrow.FileReader.html" title="pyarrow.FileReader"
+ <a href="pyarrow.RecordBatchFileReader.html" title="pyarrow.RecordBatchFileReader"
>next</a> |</li>
<li class="right" >
<a href="pyarrow.create_memory_map.html" title="pyarrow.create_memory_map"
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.RecordBatch.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.RecordBatch.html b/docs/python/generated/pyarrow.RecordBatch.html
index 1b29130..30bfb2a 100644
--- a/docs/python/generated/pyarrow.RecordBatch.html
+++ b/docs/python/generated/pyarrow.RecordBatch.html
@@ -110,7 +110,7 @@ methods instead.</p>
<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">from_arrays</span></code>(list arrays, list names, ...)</td>
<td>Construct a RecordBatch from multiple pyarrow.Arrays</td>
</tr>
-<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">from_pandas</span></code>(type cls, df[, schema])</td>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">from_pandas</span></code>(type cls, df, ...)</td>
<td>Convert pandas.DataFrame to an Arrow RecordBatch</td>
</tr>
<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">slice</span></code>(self[, offset, length])</td>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.RecordBatchFileReader.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.RecordBatchFileReader.html b/docs/python/generated/pyarrow.RecordBatchFileReader.html
new file mode 100644
index 0000000..35a0674
--- /dev/null
+++ b/docs/python/generated/pyarrow.RecordBatchFileReader.html
@@ -0,0 +1,171 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+
+ <title>pyarrow.RecordBatchFileReader — pyarrow documentation</title>
+
+ <link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
+ <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT: '../',
+ VERSION: '',
+ COLLAPSE_INDEX: false,
+ FILE_SUFFIX: '.html',
+ HAS_SOURCE: true,
+ SOURCELINK_SUFFIX: '.txt'
+ };
+ </script>
+ <script type="text/javascript" src="../_static/jquery.js"></script>
+ <script type="text/javascript" src="../_static/underscore.js"></script>
+ <script type="text/javascript" src="../_static/doctools.js"></script>
+ <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <link rel="index" title="Index" href="../genindex.html" />
+ <link rel="search" title="Search" href="../search.html" />
+ <link rel="next" title="pyarrow.RecordBatchFileWriter" href="pyarrow.RecordBatchFileWriter.html" />
+ <link rel="prev" title="pyarrow.PythonFile" href="pyarrow.PythonFile.html" />
+ </head>
+ <body role="document">
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ accesskey="I">index</a></li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchFileWriter.html" title="pyarrow.RecordBatchFileWriter"
+ accesskey="N">next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.PythonFile.html" title="pyarrow.PythonFile"
+ accesskey="P">previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" accesskey="U">API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+ <div class="sphinxsidebarwrapper">
+ <h4>Previous topic</h4>
+ <p class="topless"><a href="pyarrow.PythonFile.html"
+ title="previous chapter">pyarrow.PythonFile</a></p>
+ <h4>Next topic</h4>
+ <p class="topless"><a href="pyarrow.RecordBatchFileWriter.html"
+ title="next chapter">pyarrow.RecordBatchFileWriter</a></p>
+ <div role="note" aria-label="source link">
+ <h3>This Page</h3>
+ <ul class="this-page-menu">
+ <li><a href="../_sources/generated/pyarrow.RecordBatchFileReader.rst.txt"
+ rel="nofollow">Show Source</a></li>
+ </ul>
+ </div>
+<div id="searchbox" style="display: none" role="search">
+ <h3>Quick search</h3>
+ <form class="search" action="../search.html" method="get">
+ <div><input type="text" name="q" /></div>
+ <div><input type="submit" value="Go" /></div>
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+ </div>
+ </div>
+
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body" role="main">
+
+ <div class="section" id="pyarrow-recordbatchfilereader">
+<h1>pyarrow.RecordBatchFileReader<a class="headerlink" href="#pyarrow-recordbatchfilereader" title="Permalink to this headline">¶</a></h1>
+<dl class="class">
+<dt id="pyarrow.RecordBatchFileReader">
+<em class="property">class </em><code class="descclassname">pyarrow.</code><code class="descname">RecordBatchFileReader</code><span class="sig-paren">(</span><em>source</em>, <em>footer_offset=None</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/pyarrow/ipc.html#RecordBatchFileReader"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyarrow.RecordBatchFileReader" title="Permalink to this definition">¶</a></dt>
+<dd><p>Class for reading Arrow record batch data from the Arrow binary file format</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
+<li><strong>source</strong> (<em>str</em><em>, </em><a class="reference internal" href="pyarrow.NativeFile.html#pyarrow.NativeFile" title="pyarrow.NativeFile"><em>pyarrow.NativeFile</em></a><em>, or </em><em>file-like Python object</em>) – Either a file path, or a readable file object</li>
+<li><strong>footer_offset</strong> (<em>int</em><em>, </em><em>default None</em>) – If the file is embedded in some larger file, this is the byte offset to
+the very end of the file data</li>
+</ul>
+</td>
+</tr>
+</tbody>
+</table>
+<dl class="method">
+<dt id="pyarrow.RecordBatchFileReader.__init__">
+<code class="descname">__init__</code><span class="sig-paren">(</span><em>source</em>, <em>footer_offset=None</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/pyarrow/ipc.html#RecordBatchFileReader.__init__"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyarrow.RecordBatchFileReader.__init__" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<p class="rubric">Methods</p>
+<table border="1" class="longtable docutils">
+<colgroup>
+<col width="10%" />
+<col width="90%" />
+</colgroup>
+<tbody valign="top">
+<tr class="row-odd"><td><a class="reference internal" href="#pyarrow.RecordBatchFileReader.__init__" title="pyarrow.RecordBatchFileReader.__init__"><code class="xref py py-obj docutils literal"><span class="pre">__init__</span></code></a>(source[, footer_offset])</td>
+<td></td>
+</tr>
+<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">get_batch</span></code>(self, int i)</td>
+<td></td>
+</tr>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">get_record_batch</span></code></td>
+<td>_RecordBatchFileReader.get_batch(self, int i)</td>
+</tr>
+<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">read_all</span></code>(self)</td>
+<td>Read all record batches as a pyarrow.Table</td>
+</tr>
+</tbody>
+</table>
+<p class="rubric">Attributes</p>
+<table border="1" class="longtable docutils">
+<colgroup>
+<col width="10%" />
+<col width="90%" />
+</colgroup>
+<tbody valign="top">
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">num_record_batches</span></code></td>
+<td></td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</div>
+
+
+ </div>
+ </div>
+ </div>
+ <div class="clearer"></div>
+ </div>
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ >index</a></li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchFileWriter.html" title="pyarrow.RecordBatchFileWriter"
+ >next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.PythonFile.html" title="pyarrow.PythonFile"
+ >previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" >API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="footer" role="contentinfo">
+ © Copyright 2016-2017 Apache Software Foundation.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.
+ </div>
+ </body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.RecordBatchFileWriter.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.RecordBatchFileWriter.html b/docs/python/generated/pyarrow.RecordBatchFileWriter.html
new file mode 100644
index 0000000..14ff9cc
--- /dev/null
+++ b/docs/python/generated/pyarrow.RecordBatchFileWriter.html
@@ -0,0 +1,155 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+
+ <title>pyarrow.RecordBatchFileWriter — pyarrow documentation</title>
+
+ <link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
+ <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT: '../',
+ VERSION: '',
+ COLLAPSE_INDEX: false,
+ FILE_SUFFIX: '.html',
+ HAS_SOURCE: true,
+ SOURCELINK_SUFFIX: '.txt'
+ };
+ </script>
+ <script type="text/javascript" src="../_static/jquery.js"></script>
+ <script type="text/javascript" src="../_static/underscore.js"></script>
+ <script type="text/javascript" src="../_static/doctools.js"></script>
+ <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <link rel="index" title="Index" href="../genindex.html" />
+ <link rel="search" title="Search" href="../search.html" />
+ <link rel="next" title="pyarrow.RecordBatchStreamReader" href="pyarrow.RecordBatchStreamReader.html" />
+ <link rel="prev" title="pyarrow.RecordBatchFileReader" href="pyarrow.RecordBatchFileReader.html" />
+ </head>
+ <body role="document">
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ accesskey="I">index</a></li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchStreamReader.html" title="pyarrow.RecordBatchStreamReader"
+ accesskey="N">next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchFileReader.html" title="pyarrow.RecordBatchFileReader"
+ accesskey="P">previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" accesskey="U">API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+ <div class="sphinxsidebarwrapper">
+ <h4>Previous topic</h4>
+ <p class="topless"><a href="pyarrow.RecordBatchFileReader.html"
+ title="previous chapter">pyarrow.RecordBatchFileReader</a></p>
+ <h4>Next topic</h4>
+ <p class="topless"><a href="pyarrow.RecordBatchStreamReader.html"
+ title="next chapter">pyarrow.RecordBatchStreamReader</a></p>
+ <div role="note" aria-label="source link">
+ <h3>This Page</h3>
+ <ul class="this-page-menu">
+ <li><a href="../_sources/generated/pyarrow.RecordBatchFileWriter.rst.txt"
+ rel="nofollow">Show Source</a></li>
+ </ul>
+ </div>
+<div id="searchbox" style="display: none" role="search">
+ <h3>Quick search</h3>
+ <form class="search" action="../search.html" method="get">
+ <div><input type="text" name="q" /></div>
+ <div><input type="submit" value="Go" /></div>
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+ </div>
+ </div>
+
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body" role="main">
+
+ <div class="section" id="pyarrow-recordbatchfilewriter">
+<h1>pyarrow.RecordBatchFileWriter<a class="headerlink" href="#pyarrow-recordbatchfilewriter" title="Permalink to this headline">¶</a></h1>
+<dl class="class">
+<dt id="pyarrow.RecordBatchFileWriter">
+<em class="property">class </em><code class="descclassname">pyarrow.</code><code class="descname">RecordBatchFileWriter</code><span class="sig-paren">(</span><em>sink</em>, <em>schema</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/pyarrow/ipc.html#RecordBatchFileWriter"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyarrow.RecordBatchFileWriter" title="Permalink to this definition">¶</a></dt>
+<dd><p>Writer to create the Arrow binary file format</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
+<li><strong>sink</strong> (<em>str</em><em>, </em><a class="reference internal" href="pyarrow.NativeFile.html#pyarrow.NativeFile" title="pyarrow.NativeFile"><em>pyarrow.NativeFile</em></a><em>, or </em><em>file-like Python object</em>) – Either a file path, or a writeable file object</li>
+<li><strong>schema</strong> (<a class="reference internal" href="pyarrow.Schema.html#pyarrow.Schema" title="pyarrow.Schema"><em>pyarrow.Schema</em></a>) – The Arrow schema for data to be written to the file</li>
+</ul>
+</td>
+</tr>
+</tbody>
+</table>
+<dl class="method">
+<dt id="pyarrow.RecordBatchFileWriter.__init__">
+<code class="descname">__init__</code><span class="sig-paren">(</span><em>sink</em>, <em>schema</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/pyarrow/ipc.html#RecordBatchFileWriter.__init__"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyarrow.RecordBatchFileWriter.__init__" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<p class="rubric">Methods</p>
+<table border="1" class="longtable docutils">
+<colgroup>
+<col width="10%" />
+<col width="90%" />
+</colgroup>
+<tbody valign="top">
+<tr class="row-odd"><td><a class="reference internal" href="#pyarrow.RecordBatchFileWriter.__init__" title="pyarrow.RecordBatchFileWriter.__init__"><code class="xref py py-obj docutils literal"><span class="pre">__init__</span></code></a>(sink, schema)</td>
+<td></td>
+</tr>
+<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">close</span></code>(self)</td>
+<td></td>
+</tr>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">write_batch</span></code>(self, RecordBatch batch)</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</div>
+
+
+ </div>
+ </div>
+ </div>
+ <div class="clearer"></div>
+ </div>
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ >index</a></li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchStreamReader.html" title="pyarrow.RecordBatchStreamReader"
+ >next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchFileReader.html" title="pyarrow.RecordBatchFileReader"
+ >previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" >API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="footer" role="contentinfo">
+ © Copyright 2016-2017 Apache Software Foundation.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.
+ </div>
+ </body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.RecordBatchStreamReader.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.RecordBatchStreamReader.html b/docs/python/generated/pyarrow.RecordBatchStreamReader.html
new file mode 100644
index 0000000..249f2d2
--- /dev/null
+++ b/docs/python/generated/pyarrow.RecordBatchStreamReader.html
@@ -0,0 +1,163 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+
+ <title>pyarrow.RecordBatchStreamReader — pyarrow documentation</title>
+
+ <link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
+ <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT: '../',
+ VERSION: '',
+ COLLAPSE_INDEX: false,
+ FILE_SUFFIX: '.html',
+ HAS_SOURCE: true,
+ SOURCELINK_SUFFIX: '.txt'
+ };
+ </script>
+ <script type="text/javascript" src="../_static/jquery.js"></script>
+ <script type="text/javascript" src="../_static/underscore.js"></script>
+ <script type="text/javascript" src="../_static/doctools.js"></script>
+ <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <link rel="index" title="Index" href="../genindex.html" />
+ <link rel="search" title="Search" href="../search.html" />
+ <link rel="next" title="pyarrow.RecordBatchStreamWriter" href="pyarrow.RecordBatchStreamWriter.html" />
+ <link rel="prev" title="pyarrow.RecordBatchFileWriter" href="pyarrow.RecordBatchFileWriter.html" />
+ </head>
+ <body role="document">
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ accesskey="I">index</a></li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchStreamWriter.html" title="pyarrow.RecordBatchStreamWriter"
+ accesskey="N">next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchFileWriter.html" title="pyarrow.RecordBatchFileWriter"
+ accesskey="P">previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" accesskey="U">API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+ <div class="sphinxsidebarwrapper">
+ <h4>Previous topic</h4>
+ <p class="topless"><a href="pyarrow.RecordBatchFileWriter.html"
+ title="previous chapter">pyarrow.RecordBatchFileWriter</a></p>
+ <h4>Next topic</h4>
+ <p class="topless"><a href="pyarrow.RecordBatchStreamWriter.html"
+ title="next chapter">pyarrow.RecordBatchStreamWriter</a></p>
+ <div role="note" aria-label="source link">
+ <h3>This Page</h3>
+ <ul class="this-page-menu">
+ <li><a href="../_sources/generated/pyarrow.RecordBatchStreamReader.rst.txt"
+ rel="nofollow">Show Source</a></li>
+ </ul>
+ </div>
+<div id="searchbox" style="display: none" role="search">
+ <h3>Quick search</h3>
+ <form class="search" action="../search.html" method="get">
+ <div><input type="text" name="q" /></div>
+ <div><input type="submit" value="Go" /></div>
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+ </div>
+ </div>
+
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body" role="main">
+
+ <div class="section" id="pyarrow-recordbatchstreamreader">
+<h1>pyarrow.RecordBatchStreamReader<a class="headerlink" href="#pyarrow-recordbatchstreamreader" title="Permalink to this headline">¶</a></h1>
+<dl class="class">
+<dt id="pyarrow.RecordBatchStreamReader">
+<em class="property">class </em><code class="descclassname">pyarrow.</code><code class="descname">RecordBatchStreamReader</code><span class="sig-paren">(</span><em>source</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/pyarrow/ipc.html#RecordBatchStreamReader"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyarrow.RecordBatchStreamReader" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reader for the Arrow streaming binary format</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>source</strong> (<em>str</em><em>, </em><a class="reference internal" href="pyarrow.NativeFile.html#pyarrow.NativeFile" title="pyarrow.NativeFile"><em>pyarrow.NativeFile</em></a><em>, or </em><em>file-like Python object</em>) – Either a file path, or a readable file object</td>
+</tr>
+</tbody>
+</table>
+<dl class="method">
+<dt id="pyarrow.RecordBatchStreamReader.__init__">
+<code class="descname">__init__</code><span class="sig-paren">(</span><em>source</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/pyarrow/ipc.html#RecordBatchStreamReader.__init__"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyarrow.RecordBatchStreamReader.__init__" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<p class="rubric">Methods</p>
+<table border="1" class="longtable docutils">
+<colgroup>
+<col width="10%" />
+<col width="90%" />
+</colgroup>
+<tbody valign="top">
+<tr class="row-odd"><td><a class="reference internal" href="#pyarrow.RecordBatchStreamReader.__init__" title="pyarrow.RecordBatchStreamReader.__init__"><code class="xref py py-obj docutils literal"><span class="pre">__init__</span></code></a>(source)</td>
+<td></td>
+</tr>
+<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">get_next_batch</span></code>(self)</td>
+<td>Read next RecordBatch from the stream.</td>
+</tr>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">read_all</span></code>(self)</td>
+<td>Read all record batches as a pyarrow.Table</td>
+</tr>
+</tbody>
+</table>
+<p class="rubric">Attributes</p>
+<table border="1" class="longtable docutils">
+<colgroup>
+<col width="10%" />
+<col width="90%" />
+</colgroup>
+<tbody valign="top">
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">schema</span></code></td>
+<td></td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</div>
+
+
+ </div>
+ </div>
+ </div>
+ <div class="clearer"></div>
+ </div>
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ >index</a></li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchStreamWriter.html" title="pyarrow.RecordBatchStreamWriter"
+ >next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchFileWriter.html" title="pyarrow.RecordBatchFileWriter"
+ >previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" >API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="footer" role="contentinfo">
+ © Copyright 2016-2017 Apache Software Foundation.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.
+ </div>
+ </body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.RecordBatchStreamWriter.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.RecordBatchStreamWriter.html b/docs/python/generated/pyarrow.RecordBatchStreamWriter.html
new file mode 100644
index 0000000..bf5f1fe
--- /dev/null
+++ b/docs/python/generated/pyarrow.RecordBatchStreamWriter.html
@@ -0,0 +1,155 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+
+ <title>pyarrow.RecordBatchStreamWriter — pyarrow documentation</title>
+
+ <link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
+ <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT: '../',
+ VERSION: '',
+ COLLAPSE_INDEX: false,
+ FILE_SUFFIX: '.html',
+ HAS_SOURCE: true,
+ SOURCELINK_SUFFIX: '.txt'
+ };
+ </script>
+ <script type="text/javascript" src="../_static/jquery.js"></script>
+ <script type="text/javascript" src="../_static/underscore.js"></script>
+ <script type="text/javascript" src="../_static/doctools.js"></script>
+ <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <link rel="index" title="Index" href="../genindex.html" />
+ <link rel="search" title="Search" href="../search.html" />
+ <link rel="next" title="pyarrow.open_file" href="pyarrow.open_file.html" />
+ <link rel="prev" title="pyarrow.RecordBatchStreamReader" href="pyarrow.RecordBatchStreamReader.html" />
+ </head>
+ <body role="document">
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ accesskey="I">index</a></li>
+ <li class="right" >
+ <a href="pyarrow.open_file.html" title="pyarrow.open_file"
+ accesskey="N">next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchStreamReader.html" title="pyarrow.RecordBatchStreamReader"
+ accesskey="P">previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" accesskey="U">API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+ <div class="sphinxsidebarwrapper">
+ <h4>Previous topic</h4>
+ <p class="topless"><a href="pyarrow.RecordBatchStreamReader.html"
+ title="previous chapter">pyarrow.RecordBatchStreamReader</a></p>
+ <h4>Next topic</h4>
+ <p class="topless"><a href="pyarrow.open_file.html"
+ title="next chapter">pyarrow.open_file</a></p>
+ <div role="note" aria-label="source link">
+ <h3>This Page</h3>
+ <ul class="this-page-menu">
+ <li><a href="../_sources/generated/pyarrow.RecordBatchStreamWriter.rst.txt"
+ rel="nofollow">Show Source</a></li>
+ </ul>
+ </div>
+<div id="searchbox" style="display: none" role="search">
+ <h3>Quick search</h3>
+ <form class="search" action="../search.html" method="get">
+ <div><input type="text" name="q" /></div>
+ <div><input type="submit" value="Go" /></div>
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+ </div>
+ </div>
+
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body" role="main">
+
+ <div class="section" id="pyarrow-recordbatchstreamwriter">
+<h1>pyarrow.RecordBatchStreamWriter<a class="headerlink" href="#pyarrow-recordbatchstreamwriter" title="Permalink to this headline">¶</a></h1>
+<dl class="class">
+<dt id="pyarrow.RecordBatchStreamWriter">
+<em class="property">class </em><code class="descclassname">pyarrow.</code><code class="descname">RecordBatchStreamWriter</code><span class="sig-paren">(</span><em>sink</em>, <em>schema</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/pyarrow/ipc.html#RecordBatchStreamWriter"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyarrow.RecordBatchStreamWriter" title="Permalink to this definition">¶</a></dt>
+<dd><p>Writer for the Arrow streaming binary format</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
+<li><strong>sink</strong> (<em>str</em><em>, </em><a class="reference internal" href="pyarrow.NativeFile.html#pyarrow.NativeFile" title="pyarrow.NativeFile"><em>pyarrow.NativeFile</em></a><em>, or </em><em>file-like Python object</em>) – Either a file path, or a writeable file object</li>
+<li><strong>schema</strong> (<a class="reference internal" href="pyarrow.Schema.html#pyarrow.Schema" title="pyarrow.Schema"><em>pyarrow.Schema</em></a>) – The Arrow schema for data to be written to the file</li>
+</ul>
+</td>
+</tr>
+</tbody>
+</table>
+<dl class="method">
+<dt id="pyarrow.RecordBatchStreamWriter.__init__">
+<code class="descname">__init__</code><span class="sig-paren">(</span><em>sink</em>, <em>schema</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/pyarrow/ipc.html#RecordBatchStreamWriter.__init__"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyarrow.RecordBatchStreamWriter.__init__" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<p class="rubric">Methods</p>
+<table border="1" class="longtable docutils">
+<colgroup>
+<col width="10%" />
+<col width="90%" />
+</colgroup>
+<tbody valign="top">
+<tr class="row-odd"><td><a class="reference internal" href="#pyarrow.RecordBatchStreamWriter.__init__" title="pyarrow.RecordBatchStreamWriter.__init__"><code class="xref py py-obj docutils literal"><span class="pre">__init__</span></code></a>(sink, schema)</td>
+<td></td>
+</tr>
+<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">close</span></code>(self)</td>
+<td></td>
+</tr>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">write_batch</span></code>(self, RecordBatch batch)</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</div>
+
+
+ </div>
+ </div>
+ </div>
+ <div class="clearer"></div>
+ </div>
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ >index</a></li>
+ <li class="right" >
+ <a href="pyarrow.open_file.html" title="pyarrow.open_file"
+ >next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchStreamReader.html" title="pyarrow.RecordBatchStreamReader"
+ >previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" >API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="footer" role="contentinfo">
+ © Copyright 2016-2017 Apache Software Foundation.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.
+ </div>
+ </body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.Schema.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.Schema.html b/docs/python/generated/pyarrow.Schema.html
index b4be9d2..d954ce0 100644
--- a/docs/python/generated/pyarrow.Schema.html
+++ b/docs/python/generated/pyarrow.Schema.html
@@ -107,7 +107,10 @@
<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">field_by_name</span></code>(self, name)</td>
<td>Access a field by its name rather than the column index.</td>
</tr>
-<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">remove_metadata</span></code>(self)</td>
+<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">get_field_index</span></code>(self, name)</td>
+<td></td>
+</tr>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">remove_metadata</span></code>(self)</td>
<td>Create new schema without metadata, if any</td>
</tr>
</tbody>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.Table.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.Table.html b/docs/python/generated/pyarrow.Table.html
index f0f576b..6bbe379 100644
--- a/docs/python/generated/pyarrow.Table.html
+++ b/docs/python/generated/pyarrow.Table.html
@@ -110,7 +110,7 @@ methods instead.</p>
<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">append_column</span></code>(self, Column column)</td>
<td>Append column at end of columns.</td>
</tr>
-<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">column</span></code>(self, index)</td>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">column</span></code>(self, int64_t i)</td>
<td>Select a column by its numeric index.</td>
</tr>
<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">equals</span></code>(self, Table other)</td>
@@ -122,7 +122,7 @@ methods instead.</p>
<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">from_batches</span></code>(batches)</td>
<td>Construct a Table from a list of Arrow RecordBatches</td>
</tr>
-<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">from_pandas</span></code>(type cls, df[, ...])</td>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">from_pandas</span></code>(type cls, df, ...)</td>
<td>Convert pandas.DataFrame to an Arrow Table</td>
</tr>
<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">itercolumns</span></code>(self)</td>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.open_file.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.open_file.html b/docs/python/generated/pyarrow.open_file.html
new file mode 100644
index 0000000..2c4eeaf
--- /dev/null
+++ b/docs/python/generated/pyarrow.open_file.html
@@ -0,0 +1,136 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+
+ <title>pyarrow.open_file — pyarrow documentation</title>
+
+ <link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
+ <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT: '../',
+ VERSION: '',
+ COLLAPSE_INDEX: false,
+ FILE_SUFFIX: '.html',
+ HAS_SOURCE: true,
+ SOURCELINK_SUFFIX: '.txt'
+ };
+ </script>
+ <script type="text/javascript" src="../_static/jquery.js"></script>
+ <script type="text/javascript" src="../_static/underscore.js"></script>
+ <script type="text/javascript" src="../_static/doctools.js"></script>
+ <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <link rel="index" title="Index" href="../genindex.html" />
+ <link rel="search" title="Search" href="../search.html" />
+ <link rel="next" title="pyarrow.open_stream" href="pyarrow.open_stream.html" />
+ <link rel="prev" title="pyarrow.RecordBatchStreamWriter" href="pyarrow.RecordBatchStreamWriter.html" />
+ </head>
+ <body role="document">
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ accesskey="I">index</a></li>
+ <li class="right" >
+ <a href="pyarrow.open_stream.html" title="pyarrow.open_stream"
+ accesskey="N">next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchStreamWriter.html" title="pyarrow.RecordBatchStreamWriter"
+ accesskey="P">previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" accesskey="U">API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+ <div class="sphinxsidebarwrapper">
+ <h4>Previous topic</h4>
+ <p class="topless"><a href="pyarrow.RecordBatchStreamWriter.html"
+ title="previous chapter">pyarrow.RecordBatchStreamWriter</a></p>
+ <h4>Next topic</h4>
+ <p class="topless"><a href="pyarrow.open_stream.html"
+ title="next chapter">pyarrow.open_stream</a></p>
+ <div role="note" aria-label="source link">
+ <h3>This Page</h3>
+ <ul class="this-page-menu">
+ <li><a href="../_sources/generated/pyarrow.open_file.rst.txt"
+ rel="nofollow">Show Source</a></li>
+ </ul>
+ </div>
+<div id="searchbox" style="display: none" role="search">
+ <h3>Quick search</h3>
+ <form class="search" action="../search.html" method="get">
+ <div><input type="text" name="q" /></div>
+ <div><input type="submit" value="Go" /></div>
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+ </div>
+ </div>
+
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body" role="main">
+
+ <div class="section" id="pyarrow-open-file">
+<h1>pyarrow.open_file<a class="headerlink" href="#pyarrow-open-file" title="Permalink to this headline">¶</a></h1>
+<dl class="function">
+<dt id="pyarrow.open_file">
+<code class="descclassname">pyarrow.</code><code class="descname">open_file</code><span class="sig-paren">(</span><em>source</em>, <em>footer_offset=None</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/pyarrow/ipc.html#open_file"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyarrow.open_file" title="Permalink to this definition">¶</a></dt>
+<dd><p>Create reader for Arrow file format</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>source</strong> (<em>str</em><em>, </em><a class="reference internal" href="pyarrow.NativeFile.html#pyarrow.NativeFile" title="pyarrow.NativeFile"><em>pyarrow.NativeFile</em></a><em>, or </em><em>file-like Python object</em>) – Either a file path, or a readable file object</li>
+<li><strong>footer_offset</strong> (<em>int</em><em>, </em><em>default None</em>) – If the file is embedded in some larger file, this is the byte offset to
+the very end of the file data</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last"><strong>reader</strong> (<em>RecordBatchFileReader</em>)</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</div>
+
+
+ </div>
+ </div>
+ </div>
+ <div class="clearer"></div>
+ </div>
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ >index</a></li>
+ <li class="right" >
+ <a href="pyarrow.open_stream.html" title="pyarrow.open_stream"
+ >next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.RecordBatchStreamWriter.html" title="pyarrow.RecordBatchStreamWriter"
+ >previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" >API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="footer" role="contentinfo">
+ © Copyright 2016-2017 Apache Software Foundation.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.
+ </div>
+ </body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.open_stream.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.open_stream.html b/docs/python/generated/pyarrow.open_stream.html
new file mode 100644
index 0000000..1ad5700
--- /dev/null
+++ b/docs/python/generated/pyarrow.open_stream.html
@@ -0,0 +1,136 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+
+ <title>pyarrow.open_stream — pyarrow documentation</title>
+
+ <link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
+ <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT: '../',
+ VERSION: '',
+ COLLAPSE_INDEX: false,
+ FILE_SUFFIX: '.html',
+ HAS_SOURCE: true,
+ SOURCELINK_SUFFIX: '.txt'
+ };
+ </script>
+ <script type="text/javascript" src="../_static/jquery.js"></script>
+ <script type="text/javascript" src="../_static/underscore.js"></script>
+ <script type="text/javascript" src="../_static/doctools.js"></script>
+ <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <link rel="index" title="Index" href="../genindex.html" />
+ <link rel="search" title="Search" href="../search.html" />
+ <link rel="next" title="pyarrow.MemoryPool" href="pyarrow.MemoryPool.html" />
+ <link rel="prev" title="pyarrow.open_file" href="pyarrow.open_file.html" />
+ </head>
+ <body role="document">
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ accesskey="I">index</a></li>
+ <li class="right" >
+ <a href="pyarrow.MemoryPool.html" title="pyarrow.MemoryPool"
+ accesskey="N">next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.open_file.html" title="pyarrow.open_file"
+ accesskey="P">previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" accesskey="U">API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+ <div class="sphinxsidebarwrapper">
+ <h4>Previous topic</h4>
+ <p class="topless"><a href="pyarrow.open_file.html"
+ title="previous chapter">pyarrow.open_file</a></p>
+ <h4>Next topic</h4>
+ <p class="topless"><a href="pyarrow.MemoryPool.html"
+ title="next chapter">pyarrow.MemoryPool</a></p>
+ <div role="note" aria-label="source link">
+ <h3>This Page</h3>
+ <ul class="this-page-menu">
+ <li><a href="../_sources/generated/pyarrow.open_stream.rst.txt"
+ rel="nofollow">Show Source</a></li>
+ </ul>
+ </div>
+<div id="searchbox" style="display: none" role="search">
+ <h3>Quick search</h3>
+ <form class="search" action="../search.html" method="get">
+ <div><input type="text" name="q" /></div>
+ <div><input type="submit" value="Go" /></div>
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+ </div>
+ </div>
+
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body" role="main">
+
+ <div class="section" id="pyarrow-open-stream">
+<h1>pyarrow.open_stream<a class="headerlink" href="#pyarrow-open-stream" title="Permalink to this headline">¶</a></h1>
+<dl class="function">
+<dt id="pyarrow.open_stream">
+<code class="descclassname">pyarrow.</code><code class="descname">open_stream</code><span class="sig-paren">(</span><em>source</em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/pyarrow/ipc.html#open_stream"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyarrow.open_stream" title="Permalink to this definition">¶</a></dt>
+<dd><p>Create reader for Arrow streaming format</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>source</strong> (<em>str</em><em>, </em><a class="reference internal" href="pyarrow.NativeFile.html#pyarrow.NativeFile" title="pyarrow.NativeFile"><em>pyarrow.NativeFile</em></a><em>, or </em><em>file-like Python object</em>) – Either a file path, or a readable file object</li>
+<li><strong>footer_offset</strong> (<em>int</em><em>, </em><em>default None</em>) – If the file is embedded in some larger file, this is the byte offset to
+the very end of the file data</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last"><strong>reader</strong> (<em>RecordBatchStreamReader</em>)</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</div>
+
+
+ </div>
+ </div>
+ </div>
+ <div class="clearer"></div>
+ </div>
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ >index</a></li>
+ <li class="right" >
+ <a href="pyarrow.MemoryPool.html" title="pyarrow.MemoryPool"
+ >next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.open_file.html" title="pyarrow.open_file"
+ >previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" >API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="footer" role="contentinfo">
+ © Copyright 2016-2017 Apache Software Foundation.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.
+ </div>
+ </body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.parquet.ParquetFile.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.parquet.ParquetFile.html b/docs/python/generated/pyarrow.parquet.ParquetFile.html
index c2fa3c9..21129c1 100644
--- a/docs/python/generated/pyarrow.parquet.ParquetFile.html
+++ b/docs/python/generated/pyarrow.parquet.ParquetFile.html
@@ -117,7 +117,10 @@ see pyarrow.io.PythonFileInterface or pyarrow.io.BufferReader.</li>
<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">read</span></code>([columns, nthreads])</td>
<td>Read a Table from Parquet format</td>
</tr>
-<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">read_row_group</span></code>(i[, columns, nthreads])</td>
+<tr class="row-odd"><td><code class="xref py py-obj docutils literal"><span class="pre">read_pandas</span></code>([columns, nthreads])</td>
+<td></td>
+</tr>
+<tr class="row-even"><td><code class="xref py py-obj docutils literal"><span class="pre">read_row_group</span></code>(i[, columns, nthreads])</td>
<td>Read a single row group from a Parquet file</td>
</tr>
</tbody>
[2/4] arrow-site git commit: Update Python documentation
Posted by we...@apache.org.
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/genindex.html
----------------------------------------------------------------------
diff --git a/docs/python/genindex.html b/docs/python/genindex.html
index ea7903d..137f6fa 100644
--- a/docs/python/genindex.html
+++ b/docs/python/genindex.html
@@ -78,6 +78,7 @@
| <a href="#L"><strong>L</strong></a>
| <a href="#M"><strong>M</strong></a>
| <a href="#N"><strong>N</strong></a>
+ | <a href="#O"><strong>O</strong></a>
| <a href="#P"><strong>P</strong></a>
| <a href="#R"><strong>R</strong></a>
| <a href="#S"><strong>S</strong></a>
@@ -104,6 +105,8 @@
</li>
<li><a href="generated/pyarrow.Buffer.html#pyarrow.Buffer.__init__">(pyarrow.Buffer method)</a>
</li>
+ <li><a href="generated/pyarrow.BufferOutputStream.html#pyarrow.BufferOutputStream.__init__">(pyarrow.BufferOutputStream method)</a>
+</li>
<li><a href="generated/pyarrow.BufferReader.html#pyarrow.BufferReader.__init__">(pyarrow.BufferReader method)</a>
</li>
<li><a href="generated/pyarrow.ChunkedArray.html#pyarrow.ChunkedArray.__init__">(pyarrow.ChunkedArray method)</a>
@@ -134,10 +137,6 @@
</li>
<li><a href="generated/pyarrow.Field.html#pyarrow.Field.__init__">(pyarrow.Field method)</a>
</li>
- <li><a href="generated/pyarrow.FileReader.html#pyarrow.FileReader.__init__">(pyarrow.FileReader method)</a>
-</li>
- <li><a href="generated/pyarrow.FileWriter.html#pyarrow.FileWriter.__init__">(pyarrow.FileWriter method)</a>
-</li>
<li><a href="generated/pyarrow.FixedSizeBinaryArray.html#pyarrow.FixedSizeBinaryArray.__init__">(pyarrow.FixedSizeBinaryArray method)</a>
</li>
<li><a href="generated/pyarrow.FixedSizeBinaryType.html#pyarrow.FixedSizeBinaryType.__init__">(pyarrow.FixedSizeBinaryType method)</a>
@@ -148,8 +147,6 @@
</li>
<li><a href="generated/pyarrow.FloatingPointArray.html#pyarrow.FloatingPointArray.__init__">(pyarrow.FloatingPointArray method)</a>
</li>
- <li><a href="generated/pyarrow.InMemoryOutputStream.html#pyarrow.InMemoryOutputStream.__init__">(pyarrow.InMemoryOutputStream method)</a>
-</li>
<li><a href="generated/pyarrow.Int16Array.html#pyarrow.Int16Array.__init__">(pyarrow.Int16Array method)</a>
</li>
<li><a href="generated/pyarrow.Int16Value.html#pyarrow.Int16Value.__init__">(pyarrow.Int16Value method)</a>
@@ -188,13 +185,17 @@
</li>
<li><a href="generated/pyarrow.RecordBatch.html#pyarrow.RecordBatch.__init__">(pyarrow.RecordBatch method)</a>
</li>
- <li><a href="generated/pyarrow.Scalar.html#pyarrow.Scalar.__init__">(pyarrow.Scalar method)</a>
+ <li><a href="generated/pyarrow.RecordBatchFileReader.html#pyarrow.RecordBatchFileReader.__init__">(pyarrow.RecordBatchFileReader method)</a>
</li>
- <li><a href="generated/pyarrow.Schema.html#pyarrow.Schema.__init__">(pyarrow.Schema method)</a>
+ <li><a href="generated/pyarrow.RecordBatchFileWriter.html#pyarrow.RecordBatchFileWriter.__init__">(pyarrow.RecordBatchFileWriter method)</a>
+</li>
+ <li><a href="generated/pyarrow.RecordBatchStreamReader.html#pyarrow.RecordBatchStreamReader.__init__">(pyarrow.RecordBatchStreamReader method)</a>
</li>
- <li><a href="generated/pyarrow.StreamReader.html#pyarrow.StreamReader.__init__">(pyarrow.StreamReader method)</a>
+ <li><a href="generated/pyarrow.RecordBatchStreamWriter.html#pyarrow.RecordBatchStreamWriter.__init__">(pyarrow.RecordBatchStreamWriter method)</a>
</li>
- <li><a href="generated/pyarrow.StreamWriter.html#pyarrow.StreamWriter.__init__">(pyarrow.StreamWriter method)</a>
+ <li><a href="generated/pyarrow.Scalar.html#pyarrow.Scalar.__init__">(pyarrow.Scalar method)</a>
+</li>
+ <li><a href="generated/pyarrow.Schema.html#pyarrow.Schema.__init__">(pyarrow.Schema method)</a>
</li>
<li><a href="generated/pyarrow.StringArray.html#pyarrow.StringArray.__init__">(pyarrow.StringArray method)</a>
</li>
@@ -275,6 +276,8 @@
</li>
<li><a href="generated/pyarrow.Buffer.html#pyarrow.Buffer">Buffer (class in pyarrow)</a>
</li>
+ <li><a href="generated/pyarrow.BufferOutputStream.html#pyarrow.BufferOutputStream">BufferOutputStream (class in pyarrow)</a>
+</li>
<li><a href="generated/pyarrow.BufferReader.html#pyarrow.BufferReader">BufferReader (class in pyarrow)</a>
</li>
</ul></td>
@@ -341,18 +344,14 @@
</li>
<li><a href="generated/pyarrow.field.html#pyarrow.field">field() (in module pyarrow)</a>
</li>
- <li><a href="generated/pyarrow.FileReader.html#pyarrow.FileReader">FileReader (class in pyarrow)</a>
-</li>
- <li><a href="generated/pyarrow.FileWriter.html#pyarrow.FileWriter">FileWriter (class in pyarrow)</a>
-</li>
<li><a href="generated/pyarrow.FixedSizeBinaryArray.html#pyarrow.FixedSizeBinaryArray">FixedSizeBinaryArray (class in pyarrow)</a>
</li>
<li><a href="generated/pyarrow.FixedSizeBinaryType.html#pyarrow.FixedSizeBinaryType">FixedSizeBinaryType (class in pyarrow)</a>
</li>
- </ul></td>
- <td style="width: 33%; vertical-align: top;"><ul>
<li><a href="generated/pyarrow.FixedSizeBinaryValue.html#pyarrow.FixedSizeBinaryValue">FixedSizeBinaryValue (class in pyarrow)</a>
</li>
+ </ul></td>
+ <td style="width: 33%; vertical-align: top;"><ul>
<li><a href="generated/pyarrow.float16.html#pyarrow.float16">float16() (in module pyarrow)</a>
</li>
<li><a href="generated/pyarrow.float32.html#pyarrow.float32">float32() (in module pyarrow)</a>
@@ -383,8 +382,6 @@
<h2 id="I">I</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="generated/pyarrow.InMemoryOutputStream.html#pyarrow.InMemoryOutputStream">InMemoryOutputStream (class in pyarrow)</a>
-</li>
<li><a href="generated/pyarrow.int16.html#pyarrow.int16">int16() (in module pyarrow)</a>
</li>
<li><a href="generated/pyarrow.Int16Array.html#pyarrow.Int16Array">Int16Array (class in pyarrow)</a>
@@ -472,6 +469,18 @@
</ul></td>
</tr></table>
+<h2 id="O">O</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="generated/pyarrow.open_file.html#pyarrow.open_file">open_file() (in module pyarrow)</a>
+</li>
+ </ul></td>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="generated/pyarrow.open_stream.html#pyarrow.open_stream">open_stream() (in module pyarrow)</a>
+</li>
+ </ul></td>
+</tr></table>
+
<h2 id="P">P</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
@@ -491,13 +500,21 @@
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="generated/pyarrow.parquet.read_table.html#pyarrow.parquet.read_table">read_table() (in module pyarrow.parquet)</a>
</li>
- </ul></td>
- <td style="width: 33%; vertical-align: top;"><ul>
<li><a href="generated/pyarrow.read_tensor.html#pyarrow.read_tensor">read_tensor() (in module pyarrow)</a>
</li>
<li><a href="generated/pyarrow.RecordBatch.html#pyarrow.RecordBatch">RecordBatch (class in pyarrow)</a>
</li>
</ul></td>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="generated/pyarrow.RecordBatchFileReader.html#pyarrow.RecordBatchFileReader">RecordBatchFileReader (class in pyarrow)</a>
+</li>
+ <li><a href="generated/pyarrow.RecordBatchFileWriter.html#pyarrow.RecordBatchFileWriter">RecordBatchFileWriter (class in pyarrow)</a>
+</li>
+ <li><a href="generated/pyarrow.RecordBatchStreamReader.html#pyarrow.RecordBatchStreamReader">RecordBatchStreamReader (class in pyarrow)</a>
+</li>
+ <li><a href="generated/pyarrow.RecordBatchStreamWriter.html#pyarrow.RecordBatchStreamWriter">RecordBatchStreamWriter (class in pyarrow)</a>
+</li>
+ </ul></td>
</tr></table>
<h2 id="S">S</h2>
@@ -511,12 +528,8 @@
</li>
<li><a href="generated/pyarrow.set_memory_pool.html#pyarrow.set_memory_pool">set_memory_pool() (in module pyarrow)</a>
</li>
- <li><a href="generated/pyarrow.StreamReader.html#pyarrow.StreamReader">StreamReader (class in pyarrow)</a>
-</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="generated/pyarrow.StreamWriter.html#pyarrow.StreamWriter">StreamWriter (class in pyarrow)</a>
-</li>
<li><a href="generated/pyarrow.string.html#pyarrow.string">string() (in module pyarrow)</a>
</li>
<li><a href="generated/pyarrow.StringArray.html#pyarrow.StringArray">StringArray (class in pyarrow)</a>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/ipc.html
----------------------------------------------------------------------
diff --git a/docs/python/ipc.html b/docs/python/ipc.html
index 4fc1da8..3cb43e7 100644
Binary files a/docs/python/ipc.html and b/docs/python/ipc.html differ
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/memory.html
----------------------------------------------------------------------
diff --git a/docs/python/memory.html b/docs/python/memory.html
index f9f8cd2..b38f758 100644
Binary files a/docs/python/memory.html and b/docs/python/memory.html differ
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/objects.inv
----------------------------------------------------------------------
diff --git a/docs/python/objects.inv b/docs/python/objects.inv
index 5f1059a..a91c786 100644
Binary files a/docs/python/objects.inv and b/docs/python/objects.inv differ
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/parquet.html
----------------------------------------------------------------------
diff --git a/docs/python/parquet.html b/docs/python/parquet.html
index b5263c5..96e17c8 100644
--- a/docs/python/parquet.html
+++ b/docs/python/parquet.html
@@ -162,6 +162,7 @@ the whole file (due to the columnar layout):</p>
<span class="go">one: double</span>
<span class="go">three: bool</span>
<span class="go">-- metadata --</span>
+<span class="go">pandas: {"columns": [{"pandas_type": "float64", "numpy_type": "float64", "metadata": null, "name": "one"}, {"pandas_type": "boolean", "numpy_type": "bool", "metadata": null, "name": "three"}, {"pandas_type": "unicode", "numpy_type": "object", "metadata": null, "name": "two"}, {"pandas_type": "int64", "numpy_type": "int64", "metadata": null, "name": "__index_level_0__"}], "pandas_version": "0.20.1", "index_columns": ["__index_level_0__"]}</span>
</pre></div>
</div>
<p>We need not use a string to specify the origin of the file. It can be any of:</p>
@@ -181,20 +182,21 @@ maps) will perform the best.</p>
<span class="gp">In [13]: </span><span class="n">parquet_file</span><span class="o">.</span><span class="n">metadata</span>
<span class="gh">Out[13]: </span><span class="go"></span>
-<span class="go"><pyarrow._parquet.FileMetaData object at 0x2b0bce0dd9a8></span>
+<span class="go"><pyarrow._parquet.FileMetaData object at 0x7fce8cb28cc8></span>
<span class="go"> created_by: parquet-cpp version 1.0.0</span>
-<span class="go"> num_columns: 3</span>
+<span class="go"> num_columns: 4</span>
<span class="go"> num_rows: 3</span>
<span class="go"> num_row_groups: 1</span>
<span class="go"> format_version: 1.0</span>
-<span class="go"> serialized_size: 252</span>
+<span class="go"> serialized_size: 798</span>
<span class="gp">In [14]: </span><span class="n">parquet_file</span><span class="o">.</span><span class="n">schema</span>
<span class="go"> Out[14]: </span>
-<span class="go"><pyarrow._parquet.ParquetSchema object at 0x2b0bcdde94c8></span>
+<span class="go"><pyarrow._parquet.ParquetSchema object at 0x7fce8ede44c8></span>
<span class="go">one: DOUBLE</span>
<span class="go">three: BOOLEAN</span>
<span class="go">two: BYTE_ARRAY UTF8</span>
+<span class="go">__index_level_0__: INT64</span>
</pre></div>
</div>
<p>As you can learn more in the <a class="reference external" href="https://github.com/apache/parquet-format">Apache Parquet format</a>, a Parquet file consists of
@@ -210,7 +212,9 @@ concatenate them into a single table. You can read individual row groups with
<span class="go">one: double</span>
<span class="go">three: bool</span>
<span class="go">two: string</span>
+<span class="go">__index_level_0__: int64</span>
<span class="go">-- metadata --</span>
+<span class="go">pandas: {"columns": [{"pandas_type": "float64", "numpy_type": "float64", "metadata": null, "name": "one"}, {"pandas_type": "boolean", "numpy_type": "bool", "metadata": null, "name": "three"}, {"pandas_type": "unicode", "numpy_type": "object", "metadata": null, "name": "two"}, {"pandas_type": "int64", "numpy_type": "int64", "metadata": null, "name": "__index_level_0__"}], "pandas_version": "0.20.1", "index_columns": ["__index_level_0__"]}</span>
</pre></div>
</div>
<p>We can similarly write a Parquet file with multiple row groups by using