You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by we...@apache.org on 2017/05/23 21:22:26 UTC
[2/4] arrow-site git commit: Update Python documentation
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/genindex.html
----------------------------------------------------------------------
diff --git a/docs/python/genindex.html b/docs/python/genindex.html
index ea7903d..137f6fa 100644
--- a/docs/python/genindex.html
+++ b/docs/python/genindex.html
@@ -78,6 +78,7 @@
| <a href="#L"><strong>L</strong></a>
| <a href="#M"><strong>M</strong></a>
| <a href="#N"><strong>N</strong></a>
+ | <a href="#O"><strong>O</strong></a>
| <a href="#P"><strong>P</strong></a>
| <a href="#R"><strong>R</strong></a>
| <a href="#S"><strong>S</strong></a>
@@ -104,6 +105,8 @@
</li>
<li><a href="generated/pyarrow.Buffer.html#pyarrow.Buffer.__init__">(pyarrow.Buffer method)</a>
</li>
+ <li><a href="generated/pyarrow.BufferOutputStream.html#pyarrow.BufferOutputStream.__init__">(pyarrow.BufferOutputStream method)</a>
+</li>
<li><a href="generated/pyarrow.BufferReader.html#pyarrow.BufferReader.__init__">(pyarrow.BufferReader method)</a>
</li>
<li><a href="generated/pyarrow.ChunkedArray.html#pyarrow.ChunkedArray.__init__">(pyarrow.ChunkedArray method)</a>
@@ -134,10 +137,6 @@
</li>
<li><a href="generated/pyarrow.Field.html#pyarrow.Field.__init__">(pyarrow.Field method)</a>
</li>
- <li><a href="generated/pyarrow.FileReader.html#pyarrow.FileReader.__init__">(pyarrow.FileReader method)</a>
-</li>
- <li><a href="generated/pyarrow.FileWriter.html#pyarrow.FileWriter.__init__">(pyarrow.FileWriter method)</a>
-</li>
<li><a href="generated/pyarrow.FixedSizeBinaryArray.html#pyarrow.FixedSizeBinaryArray.__init__">(pyarrow.FixedSizeBinaryArray method)</a>
</li>
<li><a href="generated/pyarrow.FixedSizeBinaryType.html#pyarrow.FixedSizeBinaryType.__init__">(pyarrow.FixedSizeBinaryType method)</a>
@@ -148,8 +147,6 @@
</li>
<li><a href="generated/pyarrow.FloatingPointArray.html#pyarrow.FloatingPointArray.__init__">(pyarrow.FloatingPointArray method)</a>
</li>
- <li><a href="generated/pyarrow.InMemoryOutputStream.html#pyarrow.InMemoryOutputStream.__init__">(pyarrow.InMemoryOutputStream method)</a>
-</li>
<li><a href="generated/pyarrow.Int16Array.html#pyarrow.Int16Array.__init__">(pyarrow.Int16Array method)</a>
</li>
<li><a href="generated/pyarrow.Int16Value.html#pyarrow.Int16Value.__init__">(pyarrow.Int16Value method)</a>
@@ -188,13 +185,17 @@
</li>
<li><a href="generated/pyarrow.RecordBatch.html#pyarrow.RecordBatch.__init__">(pyarrow.RecordBatch method)</a>
</li>
- <li><a href="generated/pyarrow.Scalar.html#pyarrow.Scalar.__init__">(pyarrow.Scalar method)</a>
+ <li><a href="generated/pyarrow.RecordBatchFileReader.html#pyarrow.RecordBatchFileReader.__init__">(pyarrow.RecordBatchFileReader method)</a>
</li>
- <li><a href="generated/pyarrow.Schema.html#pyarrow.Schema.__init__">(pyarrow.Schema method)</a>
+ <li><a href="generated/pyarrow.RecordBatchFileWriter.html#pyarrow.RecordBatchFileWriter.__init__">(pyarrow.RecordBatchFileWriter method)</a>
+</li>
+ <li><a href="generated/pyarrow.RecordBatchStreamReader.html#pyarrow.RecordBatchStreamReader.__init__">(pyarrow.RecordBatchStreamReader method)</a>
</li>
- <li><a href="generated/pyarrow.StreamReader.html#pyarrow.StreamReader.__init__">(pyarrow.StreamReader method)</a>
+ <li><a href="generated/pyarrow.RecordBatchStreamWriter.html#pyarrow.RecordBatchStreamWriter.__init__">(pyarrow.RecordBatchStreamWriter method)</a>
</li>
- <li><a href="generated/pyarrow.StreamWriter.html#pyarrow.StreamWriter.__init__">(pyarrow.StreamWriter method)</a>
+ <li><a href="generated/pyarrow.Scalar.html#pyarrow.Scalar.__init__">(pyarrow.Scalar method)</a>
+</li>
+ <li><a href="generated/pyarrow.Schema.html#pyarrow.Schema.__init__">(pyarrow.Schema method)</a>
</li>
<li><a href="generated/pyarrow.StringArray.html#pyarrow.StringArray.__init__">(pyarrow.StringArray method)</a>
</li>
@@ -275,6 +276,8 @@
</li>
<li><a href="generated/pyarrow.Buffer.html#pyarrow.Buffer">Buffer (class in pyarrow)</a>
</li>
+ <li><a href="generated/pyarrow.BufferOutputStream.html#pyarrow.BufferOutputStream">BufferOutputStream (class in pyarrow)</a>
+</li>
<li><a href="generated/pyarrow.BufferReader.html#pyarrow.BufferReader">BufferReader (class in pyarrow)</a>
</li>
</ul></td>
@@ -341,18 +344,14 @@
</li>
<li><a href="generated/pyarrow.field.html#pyarrow.field">field() (in module pyarrow)</a>
</li>
- <li><a href="generated/pyarrow.FileReader.html#pyarrow.FileReader">FileReader (class in pyarrow)</a>
-</li>
- <li><a href="generated/pyarrow.FileWriter.html#pyarrow.FileWriter">FileWriter (class in pyarrow)</a>
-</li>
<li><a href="generated/pyarrow.FixedSizeBinaryArray.html#pyarrow.FixedSizeBinaryArray">FixedSizeBinaryArray (class in pyarrow)</a>
</li>
<li><a href="generated/pyarrow.FixedSizeBinaryType.html#pyarrow.FixedSizeBinaryType">FixedSizeBinaryType (class in pyarrow)</a>
</li>
- </ul></td>
- <td style="width: 33%; vertical-align: top;"><ul>
<li><a href="generated/pyarrow.FixedSizeBinaryValue.html#pyarrow.FixedSizeBinaryValue">FixedSizeBinaryValue (class in pyarrow)</a>
</li>
+ </ul></td>
+ <td style="width: 33%; vertical-align: top;"><ul>
<li><a href="generated/pyarrow.float16.html#pyarrow.float16">float16() (in module pyarrow)</a>
</li>
<li><a href="generated/pyarrow.float32.html#pyarrow.float32">float32() (in module pyarrow)</a>
@@ -383,8 +382,6 @@
<h2 id="I">I</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="generated/pyarrow.InMemoryOutputStream.html#pyarrow.InMemoryOutputStream">InMemoryOutputStream (class in pyarrow)</a>
-</li>
<li><a href="generated/pyarrow.int16.html#pyarrow.int16">int16() (in module pyarrow)</a>
</li>
<li><a href="generated/pyarrow.Int16Array.html#pyarrow.Int16Array">Int16Array (class in pyarrow)</a>
@@ -472,6 +469,18 @@
</ul></td>
</tr></table>
+<h2 id="O">O</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="generated/pyarrow.open_file.html#pyarrow.open_file">open_file() (in module pyarrow)</a>
+</li>
+ </ul></td>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="generated/pyarrow.open_stream.html#pyarrow.open_stream">open_stream() (in module pyarrow)</a>
+</li>
+ </ul></td>
+</tr></table>
+
<h2 id="P">P</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
@@ -491,13 +500,21 @@
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="generated/pyarrow.parquet.read_table.html#pyarrow.parquet.read_table">read_table() (in module pyarrow.parquet)</a>
</li>
- </ul></td>
- <td style="width: 33%; vertical-align: top;"><ul>
<li><a href="generated/pyarrow.read_tensor.html#pyarrow.read_tensor">read_tensor() (in module pyarrow)</a>
</li>
<li><a href="generated/pyarrow.RecordBatch.html#pyarrow.RecordBatch">RecordBatch (class in pyarrow)</a>
</li>
</ul></td>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="generated/pyarrow.RecordBatchFileReader.html#pyarrow.RecordBatchFileReader">RecordBatchFileReader (class in pyarrow)</a>
+</li>
+ <li><a href="generated/pyarrow.RecordBatchFileWriter.html#pyarrow.RecordBatchFileWriter">RecordBatchFileWriter (class in pyarrow)</a>
+</li>
+ <li><a href="generated/pyarrow.RecordBatchStreamReader.html#pyarrow.RecordBatchStreamReader">RecordBatchStreamReader (class in pyarrow)</a>
+</li>
+ <li><a href="generated/pyarrow.RecordBatchStreamWriter.html#pyarrow.RecordBatchStreamWriter">RecordBatchStreamWriter (class in pyarrow)</a>
+</li>
+ </ul></td>
</tr></table>
<h2 id="S">S</h2>
@@ -511,12 +528,8 @@
</li>
<li><a href="generated/pyarrow.set_memory_pool.html#pyarrow.set_memory_pool">set_memory_pool() (in module pyarrow)</a>
</li>
- <li><a href="generated/pyarrow.StreamReader.html#pyarrow.StreamReader">StreamReader (class in pyarrow)</a>
-</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="generated/pyarrow.StreamWriter.html#pyarrow.StreamWriter">StreamWriter (class in pyarrow)</a>
-</li>
<li><a href="generated/pyarrow.string.html#pyarrow.string">string() (in module pyarrow)</a>
</li>
<li><a href="generated/pyarrow.StringArray.html#pyarrow.StringArray">StringArray (class in pyarrow)</a>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/ipc.html
----------------------------------------------------------------------
diff --git a/docs/python/ipc.html b/docs/python/ipc.html
index 4fc1da8..3cb43e7 100644
Binary files a/docs/python/ipc.html and b/docs/python/ipc.html differ
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/memory.html
----------------------------------------------------------------------
diff --git a/docs/python/memory.html b/docs/python/memory.html
index f9f8cd2..b38f758 100644
Binary files a/docs/python/memory.html and b/docs/python/memory.html differ
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/objects.inv
----------------------------------------------------------------------
diff --git a/docs/python/objects.inv b/docs/python/objects.inv
index 5f1059a..a91c786 100644
Binary files a/docs/python/objects.inv and b/docs/python/objects.inv differ
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/parquet.html
----------------------------------------------------------------------
diff --git a/docs/python/parquet.html b/docs/python/parquet.html
index b5263c5..96e17c8 100644
--- a/docs/python/parquet.html
+++ b/docs/python/parquet.html
@@ -162,6 +162,7 @@ the whole file (due to the columnar layout):</p>
<span class="go">one: double</span>
<span class="go">three: bool</span>
<span class="go">-- metadata --</span>
+<span class="go">pandas: {"columns": [{"pandas_type": "float64", "numpy_type": "float64", "metadata": null, "name": "one"}, {"pandas_type": "boolean", "numpy_type": "bool", "metadata": null, "name": "three"}, {"pandas_type": "unicode", "numpy_type": "object", "metadata": null, "name": "two"}, {"pandas_type": "int64", "numpy_type": "int64", "metadata": null, "name": "__index_level_0__"}], "pandas_version": "0.20.1", "index_columns": ["__index_level_0__"]}</span>
</pre></div>
</div>
<p>We need not use a string to specify the origin of the file. It can be any of:</p>
@@ -181,20 +182,21 @@ maps) will perform the best.</p>
<span class="gp">In [13]: </span><span class="n">parquet_file</span><span class="o">.</span><span class="n">metadata</span>
<span class="gh">Out[13]: </span><span class="go"></span>
-<span class="go"><pyarrow._parquet.FileMetaData object at 0x2b0bce0dd9a8></span>
+<span class="go"><pyarrow._parquet.FileMetaData object at 0x7fce8cb28cc8></span>
<span class="go"> created_by: parquet-cpp version 1.0.0</span>
-<span class="go"> num_columns: 3</span>
+<span class="go"> num_columns: 4</span>
<span class="go"> num_rows: 3</span>
<span class="go"> num_row_groups: 1</span>
<span class="go"> format_version: 1.0</span>
-<span class="go"> serialized_size: 252</span>
+<span class="go"> serialized_size: 798</span>
<span class="gp">In [14]: </span><span class="n">parquet_file</span><span class="o">.</span><span class="n">schema</span>
<span class="go"> Out[14]: </span>
-<span class="go"><pyarrow._parquet.ParquetSchema object at 0x2b0bcdde94c8></span>
+<span class="go"><pyarrow._parquet.ParquetSchema object at 0x7fce8ede44c8></span>
<span class="go">one: DOUBLE</span>
<span class="go">three: BOOLEAN</span>
<span class="go">two: BYTE_ARRAY UTF8</span>
+<span class="go">__index_level_0__: INT64</span>
</pre></div>
</div>
<p>As you can learn more in the <a class="reference external" href="https://github.com/apache/parquet-format">Apache Parquet format</a>, a Parquet file consists of
@@ -210,7 +212,9 @@ concatenate them into a single table. You can read individual row groups with
<span class="go">one: double</span>
<span class="go">three: bool</span>
<span class="go">two: string</span>
+<span class="go">__index_level_0__: int64</span>
<span class="go">-- metadata --</span>
+<span class="go">pandas: {"columns": [{"pandas_type": "float64", "numpy_type": "float64", "metadata": null, "name": "one"}, {"pandas_type": "boolean", "numpy_type": "bool", "metadata": null, "name": "three"}, {"pandas_type": "unicode", "numpy_type": "object", "metadata": null, "name": "two"}, {"pandas_type": "int64", "numpy_type": "int64", "metadata": null, "name": "__index_level_0__"}], "pandas_version": "0.20.1", "index_columns": ["__index_level_0__"]}</span>
</pre></div>
</div>
<p>We can similarly write a Parquet file with multiple row groups by using