You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by we...@apache.org on 2017/07/17 13:06:15 UTC
[02/15] arrow-site git commit: Update pyarrow Python documentation
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/796ce23f/docs/python/parquet.html
----------------------------------------------------------------------
diff --git a/docs/python/parquet.html b/docs/python/parquet.html
index 7f82186..d3a08ee 100644
--- a/docs/python/parquet.html
+++ b/docs/python/parquet.html
@@ -1,5 +1,6 @@
<!DOCTYPE html>
+
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
@@ -37,7 +38,7 @@
<meta name="apple-mobile-web-app-capable" content="yes">
</head>
- <body>
+ <body role="document">
<div id="navbar" class="navbar navbar-default navbar-fixed-top">
<div class="container">
@@ -174,7 +175,7 @@ details.</p>
<h2>Reading and Writing Single Files<a class="headerlink" href="#reading-and-writing-single-files" title="Permalink to this headline">¶</a></h2>
<p>The functions <a class="reference internal" href="generated/pyarrow.parquet.read_table.html#pyarrow.parquet.read_table" title="pyarrow.parquet.read_table"><code class="xref py py-func docutils literal"><span class="pre">read_table()</span></code></a> and <a class="reference internal" href="generated/pyarrow.parquet.write_table.html#pyarrow.parquet.write_table" title="pyarrow.parquet.write_table"><code class="xref py py-func docutils literal"><span class="pre">write_table()</span></code></a>
read and write the <a class="reference internal" href="data.html#data-table"><span class="std std-ref">pyarrow.Table</span></a> objects, respectively.</p>
-<p>Let’s look at a simple table:</p>
+<p>Let’s look at a simple table:</p>
<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [2]: </span><span class="kn">import</span> <span class="nn">numpy</span> <span class="kn">as</span> <span class="nn">np</span>
<span class="gp">In [3]: </span><span class="kn">import</span> <span class="nn">pandas</span> <span class="kn">as</span> <span class="nn">pd</span>
@@ -216,7 +217,7 @@ the whole file (due to the columnar layout):</p>
<span class="go">one: double</span>
<span class="go">three: bool</span>
<span class="go">-- metadata --</span>
-<span class="go">pandas: {"pandas_version": "0.19.0", "index_columns": ["__index_level_0__"], "columns": [{"metadata": null, "name": "one", "numpy_type": "float64", "pandas_type": "float64"}, {"metadata": null, "name": "three", "numpy_type": "bool", "pandas_type": "boolean"}, {"metadata": null, "name": "two", "numpy_type": "object", "pandas_type": "bytes"}, {"metadata": null, "name": "__index_level_0__", "numpy_type": "int64", "pandas_type": "int64"}]}</span>
+<span class="go">pandas: {"pandas_version": "0.19.2", "index_columns": ["__index_level_0__"], "columns": [{"metadata": null, "numpy_type": "float64", "pandas_type": "float64", "name": "one"}, {"metadata": null, "numpy_type": "bool", "pandas_type": "bool", "name": "three"}, {"metadata": null, "numpy_type": "object", "pandas_type": "unicode", "name": "two"}, {"metadata": null, "numpy_type": "int64", "pandas_type": "int64", "name": "__index_level_0__"}]}</span>
</pre></div>
</div>
<p>We need not use a string to specify the origin of the file. It can be any of:</p>
@@ -236,20 +237,20 @@ maps) will perform the best.</p>
<span class="gp">In [13]: </span><span class="n">parquet_file</span><span class="o">.</span><span class="n">metadata</span>
<span class="gh">Out[13]: </span><span class="go"></span>
-<span class="go"><pyarrow._parquet.FileMetaData object at 0x7efc440f5940></span>
+<span class="go"><pyarrow._parquet.FileMetaData object at 0x2b9643b72cc8></span>
<span class="go"> created_by: parquet-cpp version 1.1.1-SNAPSHOT</span>
<span class="go"> num_columns: 4</span>
<span class="go"> num_rows: 3</span>
<span class="go"> num_row_groups: 1</span>
<span class="go"> format_version: 1.0</span>
-<span class="go"> serialized_size: 803</span>
+<span class="go"> serialized_size: 804</span>
<span class="gp">In [14]: </span><span class="n">parquet_file</span><span class="o">.</span><span class="n">schema</span>
-<span class="gh">Out[14]: </span><span class="go"></span>
-<span class="go"><pyarrow._parquet.ParquetSchema object at 0x7efc46c13fc8></span>
+<span class="go"> Out[14]: </span>
+<span class="go"><pyarrow._parquet.ParquetSchema object at 0x2b9642aecfc8></span>
<span class="go">one: DOUBLE</span>
<span class="go">three: BOOLEAN</span>
-<span class="go">two: BYTE_ARRAY</span>
+<span class="go">two: BYTE_ARRAY UTF8</span>
<span class="go">__index_level_0__: INT64</span>
</pre></div>
</div>
@@ -261,14 +262,14 @@ concatenate them into a single table. You can read individual row groups with
<span class="gh">Out[15]: </span><span class="go">1</span>
<span class="gp">In [16]: </span><span class="n">parquet_file</span><span class="o">.</span><span class="n">read_row_group</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
-<span class="gh">Out[16]: </span><span class="go"></span>
+<span class="go"> Out[16]: </span>
<span class="go">pyarrow.Table</span>
<span class="go">one: double</span>
<span class="go">three: bool</span>
-<span class="go">two: binary</span>
+<span class="go">two: string</span>
<span class="go">__index_level_0__: int64</span>
<span class="go">-- metadata --</span>
-<span class="go">pandas: {"pandas_version": "0.19.0", "index_columns": ["__index_level_0__"], "columns": [{"metadata": null, "name": "one", "numpy_type": "float64", "pandas_type": "float64"}, {"metadata": null, "name": "three", "numpy_type": "bool", "pandas_type": "boolean"}, {"metadata": null, "name": "two", "numpy_type": "object", "pandas_type": "bytes"}, {"metadata": null, "name": "__index_level_0__", "numpy_type": "int64", "pandas_type": "int64"}]}</span>
+<span class="go">pandas: {"pandas_version": "0.19.2", "index_columns": ["__index_level_0__"], "columns": [{"metadata": null, "numpy_type": "float64", "pandas_type": "float64", "name": "one"}, {"metadata": null, "numpy_type": "bool", "pandas_type": "bool", "name": "three"}, {"metadata": null, "numpy_type": "object", "pandas_type": "unicode", "name": "two"}, {"metadata": null, "numpy_type": "int64", "pandas_type": "int64", "name": "__index_level_0__"}]}</span>
</pre></div>
</div>
<p>We can similarly write a Parquet file with multiple row groups by using
@@ -291,7 +292,7 @@ concatenate them into a single table. You can read individual row groups with
<div class="section" id="compression-encoding-and-file-compatibility">
<h2>Compression, Encoding, and File Compatibility<a class="headerlink" href="#compression-encoding-and-file-compatibility" title="Permalink to this headline">¶</a></h2>
<p>The most commonly used Parquet implementations use dictionary encoding when
-writing files; if the dictionaries grow too large, then they “fall back” to
+writing files; if the dictionaries grow too large, then they “fall back” to
plain encoding. Whether dictionary encoding is used can be toggled using the
<code class="docutils literal"><span class="pre">use_dictionary</span></code> option:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">pq</span><span class="o">.</span><span class="n">write_table</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="n">where</span><span class="p">,</span> <span class="n">use_dictionary</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
@@ -376,7 +377,7 @@ throughput:</p>
</p>
<p>
© Copyright 2016-2017 Apache Software Foundation.<br/>
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.2.<br/>
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.<br/>
</p>
</div>
</footer>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/796ce23f/docs/python/search.html
----------------------------------------------------------------------
diff --git a/docs/python/search.html b/docs/python/search.html
index eddb979..cc410d7 100644
--- a/docs/python/search.html
+++ b/docs/python/search.html
@@ -1,5 +1,6 @@
<!DOCTYPE html>
+
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
@@ -43,7 +44,7 @@
</head>
- <body>
+ <body role="document">
<div id="navbar" class="navbar navbar-default navbar-fixed-top">
<div class="container">
@@ -164,7 +165,7 @@
</p>
<p>
© Copyright 2016-2017 Apache Software Foundation.<br/>
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.2.<br/>
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.<br/>
</p>
</div>
</footer>