You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by we...@apache.org on 2017/05/08 04:53:10 UTC
[25/27] arrow-site git commit: Update Python documentation
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/6360599f/docs/python/data.html
----------------------------------------------------------------------
diff --git a/docs/python/data.html b/docs/python/data.html
new file mode 100644
index 0000000..e16f145
--- /dev/null
+++ b/docs/python/data.html
@@ -0,0 +1,524 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+
+ <title>In-Memory Data Model — pyarrow documentation</title>
+
+ <link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
+ <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT: './',
+ VERSION: '',
+ COLLAPSE_INDEX: false,
+ FILE_SUFFIX: '.html',
+ HAS_SOURCE: true,
+ SOURCELINK_SUFFIX: '.txt'
+ };
+ </script>
+ <script type="text/javascript" src="_static/jquery.js"></script>
+ <script type="text/javascript" src="_static/underscore.js"></script>
+ <script type="text/javascript" src="_static/doctools.js"></script>
+ <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <link rel="index" title="Index" href="genindex.html" />
+ <link rel="search" title="Search" href="search.html" />
+ <link rel="next" title="IPC: Fast Streaming and Serialization" href="ipc.html" />
+ <link rel="prev" title="Memory and IO Interfaces" href="memory.html" />
+ </head>
+ <body role="document">
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="genindex.html" title="General Index"
+ accesskey="I">index</a></li>
+ <li class="right" >
+ <a href="ipc.html" title="IPC: Fast Streaming and Serialization"
+ accesskey="N">next</a> |</li>
+ <li class="right" >
+ <a href="memory.html" title="Memory and IO Interfaces"
+ accesskey="P">previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="index.html">pyarrow documentation</a> »</li>
+ </ul>
+ </div>
+ <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+ <div class="sphinxsidebarwrapper">
+ <h3><a href="index.html">Table Of Contents</a></h3>
+ <ul>
+<li><a class="reference internal" href="#">In-Memory Data Model</a><ul>
+<li><a class="reference internal" href="#type-metadata">Type Metadata</a></li>
+<li><a class="reference internal" href="#schemas">Schemas</a></li>
+<li><a class="reference internal" href="#arrays">Arrays</a><ul>
+<li><a class="reference internal" href="#dictionary-arrays">Dictionary Arrays</a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#record-batches">Record Batches</a></li>
+<li><a class="reference internal" href="#tables">Tables</a></li>
+<li><a class="reference internal" href="#custom-schema-and-field-metadata">Custom Schema and Field Metadata</a></li>
+</ul>
+</li>
+</ul>
+
+ <h4>Previous topic</h4>
+ <p class="topless"><a href="memory.html"
+ title="previous chapter">Memory and IO Interfaces</a></p>
+ <h4>Next topic</h4>
+ <p class="topless"><a href="ipc.html"
+ title="next chapter">IPC: Fast Streaming and Serialization</a></p>
+ <div role="note" aria-label="source link">
+ <h3>This Page</h3>
+ <ul class="this-page-menu">
+ <li><a href="_sources/data.rst.txt"
+ rel="nofollow">Show Source</a></li>
+ </ul>
+ </div>
+<div id="searchbox" style="display: none" role="search">
+ <h3>Quick search</h3>
+ <form class="search" action="search.html" method="get">
+ <div><input type="text" name="q" /></div>
+ <div><input type="submit" value="Go" /></div>
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+ </div>
+ </div>
+
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body" role="main">
+
+ <div class="section" id="in-memory-data-model">
+<span id="data"></span><h1>In-Memory Data Model<a class="headerlink" href="#in-memory-data-model" title="Permalink to this headline">¶</a></h1>
+<p>Apache Arrow defines columnar array data structures by composing type metadata
+with memory buffers, like the ones explained in the documentation on
+<a class="reference internal" href="memory.html#io"><span class="std std-ref">Memory and IO</span></a>. These data structures are exposed in Python through
+a series of interrelated classes:</p>
+<ul class="simple">
+<li><strong>Type Metadata</strong>: Instances of <code class="docutils literal"><span class="pre">pyarrow.DataType</span></code>, which describe a logical
+array type</li>
+<li><strong>Schemas</strong>: Instances of <code class="docutils literal"><span class="pre">pyarrow.Schema</span></code>, which describe a named
+collection of types. These can be thought of as the column types in a
+table-like object.</li>
+<li><strong>Arrays</strong>: Instances of <code class="docutils literal"><span class="pre">pyarrow.Array</span></code>, which are atomic, contiguous
+columnar data structures composed from Arrow Buffer objects</li>
+<li><strong>Record Batches</strong>: Instances of <code class="docutils literal"><span class="pre">pyarrow.RecordBatch</span></code>, which are a
+collection of Array objects with a particular Schema</li>
+<li><strong>Tables</strong>: Instances of <code class="docutils literal"><span class="pre">pyarrow.Table</span></code>, a logical table data structure in
+which each column consists of one or more <code class="docutils literal"><span class="pre">pyarrow.Array</span></code> objects of the
+same type.</li>
+</ul>
+<p>We will examine these in the sections below in a series of examples.</p>
+<div class="section" id="type-metadata">
+<span id="data-types"></span><h2>Type Metadata<a class="headerlink" href="#type-metadata" title="Permalink to this headline">¶</a></h2>
+<p>Apache Arrow defines language agnostic column-oriented data structures for
+array data. These include:</p>
+<ul class="simple">
+<li><strong>Fixed-length primitive types</strong>: numbers, booleans, date and times, fixed
+size binary, decimals, and other values that fit into a given number</li>
+<li><strong>Variable-length primitive types</strong>: binary, string</li>
+<li><strong>Nested types</strong>: list, struct, and union</li>
+<li><strong>Dictionary type</strong>: An encoded categorical type (more on this later)</li>
+</ul>
+<p>Each logical data type in Arrow has a corresponding factory function for
+creating an instance of that type object in Python:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [1]: </span><span class="kn">import</span> <span class="nn">pyarrow</span> <span class="kn">as</span> <span class="nn">pa</span>
+
+<span class="gp">In [2]: </span><span class="n">t1</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">int32</span><span class="p">()</span>
+
+<span class="gp">In [3]: </span><span class="n">t2</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">string</span><span class="p">()</span>
+
+<span class="gp">In [4]: </span><span class="n">t3</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">binary</span><span class="p">()</span>
+
+<span class="gp">In [5]: </span><span class="n">t4</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">binary</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
+
+<span class="gp">In [6]: </span><span class="n">t5</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">timestamp</span><span class="p">(</span><span class="s1">'ms'</span><span class="p">)</span>
+
+<span class="gp">In [7]: </span><span class="n">t1</span>
+<span class="gh">Out[7]: </span><span class="go">DataType(int32)</span>
+
+<span class="gp">In [8]: </span><span class="k">print</span><span class="p">(</span><span class="n">t1</span><span class="p">)</span>
+<span class="go"> int32</span>
+
+<span class="gp">In [9]: </span><span class="k">print</span><span class="p">(</span><span class="n">t4</span><span class="p">)</span>
+<span class="go"> fixed_size_binary[10]</span>
+
+<span class="gp">In [10]: </span><span class="k">print</span><span class="p">(</span><span class="n">t5</span><span class="p">)</span>
+<span class="go"> timestamp[ms]</span>
+</pre></div>
+</div>
+<p>We use the name <strong>logical type</strong> because the <strong>physical</strong> storage may be the
+same for one or more types. For example, <code class="docutils literal"><span class="pre">int64</span></code>, <code class="docutils literal"><span class="pre">float64</span></code>, and
+<code class="docutils literal"><span class="pre">timestamp[ms]</span></code> all occupy 64 bits per value.</p>
+<p>These objects are <cite>metadata</cite>; they are used for describing the data in arrays,
+schemas, and record batches. In Python, they can be used in functions where the
+input data (e.g. Python objects) may be coerced to more than one Arrow type.</p>
+<p>The <a class="reference internal" href="generated/pyarrow.Field.html#pyarrow.Field" title="pyarrow.Field"><code class="xref py py-class docutils literal"><span class="pre">Field</span></code></a> type is a type plus a name and optional
+user-defined metadata:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [11]: </span><span class="n">f0</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">field</span><span class="p">(</span><span class="s1">'int32_field'</span><span class="p">,</span> <span class="n">t1</span><span class="p">)</span>
+
+<span class="gp">In [12]: </span><span class="n">f0</span>
+<span class="gh">Out[12]: </span><span class="go">pyarrow.Field<int32_field: int32></span>
+
+<span class="gp">In [13]: </span><span class="n">f0</span><span class="o">.</span><span class="n">name</span>
+<span class="go"> Out[13]: 'int32_field'</span>
+
+<span class="gp">In [14]: </span><span class="n">f0</span><span class="o">.</span><span class="n">type</span>
+<span class="go"> Out[14]: DataType(int32)</span>
+</pre></div>
+</div>
+<p>Arrow supports <strong>nested value types</strong> like list, struct, and union. When
+creating these, you must pass types or fields to indicate the data types of the
+types’ children. For example, we can define a list of int32 values with:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [15]: </span><span class="n">t6</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">list_</span><span class="p">(</span><span class="n">t1</span><span class="p">)</span>
+
+<span class="gp">In [16]: </span><span class="n">t6</span>
+<span class="gh">Out[16]: </span><span class="go">DataType(list<item: int32>)</span>
+</pre></div>
+</div>
+<p>A <cite>struct</cite> is a collection of named fields:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [17]: </span><span class="n">fields</span> <span class="o">=</span> <span class="p">[</span>
+<span class="gp"> ....: </span> <span class="n">pa</span><span class="o">.</span><span class="n">field</span><span class="p">(</span><span class="s1">'s0'</span><span class="p">,</span> <span class="n">t1</span><span class="p">),</span>
+<span class="gp"> ....: </span> <span class="n">pa</span><span class="o">.</span><span class="n">field</span><span class="p">(</span><span class="s1">'s1'</span><span class="p">,</span> <span class="n">t2</span><span class="p">),</span>
+<span class="gp"> ....: </span> <span class="n">pa</span><span class="o">.</span><span class="n">field</span><span class="p">(</span><span class="s1">'s2'</span><span class="p">,</span> <span class="n">t4</span><span class="p">),</span>
+<span class="gp"> ....: </span> <span class="n">pa</span><span class="o">.</span><span class="n">field</span><span class="p">(</span><span class="s1">'s3'</span><span class="p">,</span> <span class="n">t6</span><span class="p">)</span>
+<span class="gp"> ....: </span><span class="p">]</span>
+<span class="gp"> ....: </span>
+
+<span class="gp">In [18]: </span><span class="n">t7</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">struct</span><span class="p">(</span><span class="n">fields</span><span class="p">)</span>
+
+<span class="gp">In [19]: </span><span class="k">print</span><span class="p">(</span><span class="n">t7</span><span class="p">)</span>
+<span class="go">struct<s0: int32, s1: string, s2: fixed_size_binary[10], s3: list<item: int32>></span>
+</pre></div>
+</div>
+<p>See <a class="reference internal" href="api.html#api-types"><span class="std std-ref">Data Types API</span></a> for a full listing of data type
+functions.</p>
+</div>
+<div class="section" id="schemas">
+<span id="data-schema"></span><h2>Schemas<a class="headerlink" href="#schemas" title="Permalink to this headline">¶</a></h2>
+<p>The <a class="reference internal" href="generated/pyarrow.Schema.html#pyarrow.Schema" title="pyarrow.Schema"><code class="xref py py-class docutils literal"><span class="pre">Schema</span></code></a> type is similar to the <code class="docutils literal"><span class="pre">struct</span></code> array type; it
+defines the column names and types in a record batch or table data
+structure. The <code class="docutils literal"><span class="pre">pyarrow.schema</span></code> factory function makes new Schema objects in
+Python:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [20]: </span><span class="n">fields</span> <span class="o">=</span> <span class="p">[</span>
+<span class="gp"> ....: </span> <span class="n">pa</span><span class="o">.</span><span class="n">field</span><span class="p">(</span><span class="s1">'s0'</span><span class="p">,</span> <span class="n">t1</span><span class="p">),</span>
+<span class="gp"> ....: </span> <span class="n">pa</span><span class="o">.</span><span class="n">field</span><span class="p">(</span><span class="s1">'s1'</span><span class="p">,</span> <span class="n">t2</span><span class="p">),</span>
+<span class="gp"> ....: </span> <span class="n">pa</span><span class="o">.</span><span class="n">field</span><span class="p">(</span><span class="s1">'s2'</span><span class="p">,</span> <span class="n">t4</span><span class="p">),</span>
+<span class="gp"> ....: </span> <span class="n">pa</span><span class="o">.</span><span class="n">field</span><span class="p">(</span><span class="s1">'s3'</span><span class="p">,</span> <span class="n">t6</span><span class="p">)</span>
+<span class="gp"> ....: </span><span class="p">]</span>
+<span class="gp"> ....: </span>
+
+<span class="gp">In [21]: </span><span class="n">my_schema</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">schema</span><span class="p">(</span><span class="n">fields</span><span class="p">)</span>
+
+<span class="gp">In [22]: </span><span class="n">my_schema</span>
+<span class="gh">Out[22]: </span><span class="go"></span>
+<span class="go">s0: int32</span>
+<span class="go">s1: string</span>
+<span class="go">s2: fixed_size_binary[10]</span>
+<span class="go">s3: list<item: int32></span>
+</pre></div>
+</div>
+<p>In some applications, you may not create schemas directly, only using the ones
+that are embedded in <a class="reference internal" href="ipc.html#ipc"><span class="std std-ref">IPC messages</span></a>.</p>
+</div>
+<div class="section" id="arrays">
+<span id="data-array"></span><h2>Arrays<a class="headerlink" href="#arrays" title="Permalink to this headline">¶</a></h2>
+<p>For each data type, there is an accompanying array data structure for holding
+memory buffers that define a single contiguous chunk of columnar array
+data. When you are using PyArrow, this data may come from IPC tools, though it
+can also be created from various types of Python sequences (lists, NumPy
+arrays, pandas data).</p>
+<p>A simple way to create arrays is with <code class="docutils literal"><span class="pre">pyarrow.array</span></code>, which is similar to
+the <code class="docutils literal"><span class="pre">numpy.array</span></code> function:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [23]: </span><span class="n">arr</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="bp">None</span><span class="p">,</span> <span class="mi">3</span><span class="p">])</span>
+
+<span class="gp">In [24]: </span><span class="n">arr</span>
+<span class="gh">Out[24]: </span><span class="go"></span>
+<span class="go"><pyarrow._array.Int64Array object at 0x2b0bcdd11318></span>
+<span class="go">[</span>
+<span class="go"> 1,</span>
+<span class="go"> 2,</span>
+<span class="go"> NA,</span>
+<span class="go"> 3</span>
+<span class="go">]</span>
+</pre></div>
+</div>
+<p>The array’s <code class="docutils literal"><span class="pre">type</span></code> attribute is the corresponding piece of type metadata:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [25]: </span><span class="n">arr</span><span class="o">.</span><span class="n">type</span>
+<span class="gh">Out[25]: </span><span class="go">DataType(int64)</span>
+</pre></div>
+</div>
+<p>Each in-memory array has a known length and null count (which will be 0 if
+there are no null values):</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [26]: </span><span class="nb">len</span><span class="p">(</span><span class="n">arr</span><span class="p">)</span>
+<span class="gh">Out[26]: </span><span class="go">4</span>
+
+<span class="gp">In [27]: </span><span class="n">arr</span><span class="o">.</span><span class="n">null_count</span>
+<span class="go"> Out[27]: 1</span>
+</pre></div>
+</div>
+<p>Scalar values can be selected with normal indexing. <code class="docutils literal"><span class="pre">pyarrow.array</span></code> converts
+<code class="docutils literal"><span class="pre">None</span></code> values to Arrow nulls; we return the special <code class="docutils literal"><span class="pre">pyarrow.NA</span></code> value for
+nulls:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [28]: </span><span class="n">arr</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
+<span class="gh">Out[28]: </span><span class="go">1</span>
+
+<span class="gp">In [29]: </span><span class="n">arr</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
+<span class="go"> Out[29]: NA</span>
+</pre></div>
+</div>
+<p>Arrow data is immutable, so values can be selected but not assigned.</p>
+<p>Arrays can be sliced without copying:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [30]: </span><span class="n">arr</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span>
+<span class="gh">Out[30]: </span><span class="go">3</span>
+</pre></div>
+</div>
+<p><code class="docutils literal"><span class="pre">pyarrow.array</span></code> can create simple nested data structures like lists:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [31]: </span><span class="n">nested_arr</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">array</span><span class="p">([[],</span> <span class="bp">None</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span> <span class="p">[</span><span class="bp">None</span><span class="p">,</span> <span class="mi">1</span><span class="p">]])</span>
+
+<span class="gp">In [32]: </span><span class="k">print</span><span class="p">(</span><span class="n">nested_arr</span><span class="o">.</span><span class="n">type</span><span class="p">)</span>
+<span class="go">list<item: int64></span>
+</pre></div>
+</div>
+<div class="section" id="dictionary-arrays">
+<h3>Dictionary Arrays<a class="headerlink" href="#dictionary-arrays" title="Permalink to this headline">¶</a></h3>
+<p>The <strong>Dictionary</strong> type in PyArrow is a special array type that is similar to a
+factor in R or a <code class="docutils literal"><span class="pre">pandas.Categorical</span></code>. It enables one or more record batches
+in a file or stream to transmit integer <em>indices</em> referencing a shared
+<strong>dictionary</strong> containing the distinct values in the logical array. This is
+particularly often used with strings to save memory and improve performance.</p>
+<p>The way that dictionaries are handled in the Apache Arrow format and the way
+they appear in C++ and Python is slightly different. We define a special
+<a class="reference internal" href="generated/pyarrow.DictionaryArray.html#pyarrow.DictionaryArray" title="pyarrow.DictionaryArray"><code class="xref py py-class docutils literal"><span class="pre">DictionaryArray</span></code></a> type with a corresponding dictionary type. Let’s
+consider an example:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [33]: </span><span class="n">indices</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="bp">None</span><span class="p">,</span> <span class="mi">2</span><span class="p">])</span>
+
+<span class="gp">In [34]: </span><span class="n">dictionary</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="s1">'foo'</span><span class="p">,</span> <span class="s1">'bar'</span><span class="p">,</span> <span class="s1">'baz'</span><span class="p">])</span>
+
+<span class="gp">In [35]: </span><span class="n">dict_array</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">DictionaryArray</span><span class="o">.</span><span class="n">from_arrays</span><span class="p">(</span><span class="n">indices</span><span class="p">,</span> <span class="n">dictionary</span><span class="p">)</span>
+
+<span class="gp">In [36]: </span><span class="n">dict_array</span>
+<span class="gh">Out[36]: </span><span class="go"></span>
+<span class="go"><pyarrow._array.DictionaryArray object at 0x2b0bcdd19588></span>
+<span class="go">[</span>
+<span class="go"> 'foo',</span>
+<span class="go"> 'bar',</span>
+<span class="go"> 'foo',</span>
+<span class="go"> 'bar',</span>
+<span class="go"> 'baz',</span>
+<span class="go"> 'foo',</span>
+<span class="go"> NA,</span>
+<span class="go"> 'baz'</span>
+<span class="go">]</span>
+</pre></div>
+</div>
+<p>Here we have:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [37]: </span><span class="k">print</span><span class="p">(</span><span class="n">dict_array</span><span class="o">.</span><span class="n">type</span><span class="p">)</span>
+<span class="go">dictionary<values=string, indices=int64></span>
+
+<span class="gp">In [38]: </span><span class="n">dict_array</span><span class="o">.</span><span class="n">indices</span>
+<span class="go"> Out[38]: </span>
+<span class="go"><pyarrow._array.Int64Array object at 0x2b0bcdd29908></span>
+<span class="go">[</span>
+<span class="go"> 0,</span>
+<span class="go"> 1,</span>
+<span class="go"> 0,</span>
+<span class="go"> 1,</span>
+<span class="go"> 2,</span>
+<span class="go"> 0,</span>
+<span class="go"> NA,</span>
+<span class="go"> 2</span>
+<span class="go">]</span>
+
+<span class="gp">In [39]: </span><span class="n">dict_array</span><span class="o">.</span><span class="n">dictionary</span>
+<span class="go"> Out[39]: </span>
+<span class="go"><pyarrow._array.StringArray object at 0x2b0bcdd29818></span>
+<span class="go">[</span>
+<span class="go"> 'foo',</span>
+<span class="go"> 'bar',</span>
+<span class="go"> 'baz'</span>
+<span class="go">]</span>
+</pre></div>
+</div>
+<p>When using <a class="reference internal" href="generated/pyarrow.DictionaryArray.html#pyarrow.DictionaryArray" title="pyarrow.DictionaryArray"><code class="xref py py-class docutils literal"><span class="pre">DictionaryArray</span></code></a> with pandas, the analogue is
+<code class="docutils literal"><span class="pre">pandas.Categorical</span></code> (more on this later):</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [40]: </span><span class="n">dict_array</span><span class="o">.</span><span class="n">to_pandas</span><span class="p">()</span>
+<span class="gh">Out[40]: </span><span class="go"></span>
+<span class="go">[foo, bar, foo, bar, baz, foo, NaN, baz]</span>
+<span class="go">Categories (3, object): [foo, bar, baz]</span>
+</pre></div>
+</div>
+</div>
+</div>
+<div class="section" id="record-batches">
+<span id="data-record-batch"></span><h2>Record Batches<a class="headerlink" href="#record-batches" title="Permalink to this headline">¶</a></h2>
+<p>A <strong>Record Batch</strong> in Apache Arrow is a collection of equal-length array
+instances. Let’s consider a collection of arrays:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [41]: </span><span class="n">data</span> <span class="o">=</span> <span class="p">[</span>
+<span class="gp"> ....: </span> <span class="n">pa</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]),</span>
+<span class="gp"> ....: </span> <span class="n">pa</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="s1">'foo'</span><span class="p">,</span> <span class="s1">'bar'</span><span class="p">,</span> <span class="s1">'baz'</span><span class="p">,</span> <span class="bp">None</span><span class="p">]),</span>
+<span class="gp"> ....: </span> <span class="n">pa</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="bp">True</span><span class="p">,</span> <span class="bp">None</span><span class="p">,</span> <span class="bp">False</span><span class="p">,</span> <span class="bp">True</span><span class="p">])</span>
+<span class="gp"> ....: </span><span class="p">]</span>
+<span class="gp"> ....: </span>
+</pre></div>
+</div>
+<p>A record batch can be created from this list of arrays using
+<code class="docutils literal"><span class="pre">RecordBatch.from_arrays</span></code>:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [42]: </span><span class="n">batch</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">RecordBatch</span><span class="o">.</span><span class="n">from_arrays</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="p">[</span><span class="s1">'f0'</span><span class="p">,</span> <span class="s1">'f1'</span><span class="p">,</span> <span class="s1">'f2'</span><span class="p">])</span>
+
+<span class="gp">In [43]: </span><span class="n">batch</span><span class="o">.</span><span class="n">num_columns</span>
+<span class="gh">Out[43]: </span><span class="go">3</span>
+
+<span class="gp">In [44]: </span><span class="n">batch</span><span class="o">.</span><span class="n">num_rows</span>
+<span class="go"> Out[44]: 4</span>
+
+<span class="gp">In [45]: </span><span class="n">batch</span><span class="o">.</span><span class="n">schema</span>
+<span class="go"> Out[45]: </span>
+<span class="go">f0: int64</span>
+<span class="go">f1: string</span>
+<span class="go">f2: bool</span>
+<span class="go">-- metadata --</span>
+
+<span class="gp">In [46]: </span><span class="n">batch</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
+<span class="go"> Out[46]: </span>
+<span class="go"><pyarrow._array.StringArray object at 0x2b0bcdd36f98></span>
+<span class="go">[</span>
+<span class="go"> 'foo',</span>
+<span class="go"> 'bar',</span>
+<span class="go"> 'baz',</span>
+<span class="go"> NA</span>
+<span class="go">]</span>
+</pre></div>
+</div>
+<p>A record batch can be sliced without copying memory like an array:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [47]: </span><span class="n">batch2</span> <span class="o">=</span> <span class="n">batch</span><span class="o">.</span><span class="n">slice</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
+
+<span class="gp">In [48]: </span><span class="n">batch2</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
+<span class="gh">Out[48]: </span><span class="go"></span>
+<span class="go"><pyarrow._array.StringArray object at 0x2b0bcdd36ea8></span>
+<span class="go">[</span>
+<span class="go"> 'bar',</span>
+<span class="go"> 'baz',</span>
+<span class="go"> NA</span>
+<span class="go">]</span>
+</pre></div>
+</div>
+</div>
+<div class="section" id="tables">
+<span id="data-table"></span><h2>Tables<a class="headerlink" href="#tables" title="Permalink to this headline">¶</a></h2>
+<p>The PyArrow <a class="reference internal" href="generated/pyarrow.Table.html#pyarrow.Table" title="pyarrow.Table"><code class="xref py py-class docutils literal"><span class="pre">Table</span></code></a> type is not part of the Apache Arrow
+specification, but is rather a tool to help with wrangling multiple record
+batches and array pieces as a single logical dataset. As a relevant example, we
+may receive multiple small record batches in a socket stream, then need to
+concatenate them into contiguous memory for use in NumPy or pandas. The Table
+object makes this efficient without requiring additional memory copying.</p>
+<p>Considering the record batch we created above, we can create a Table containing
+one or more copies of the batch using <code class="docutils literal"><span class="pre">Table.from_batches</span></code>:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [49]: </span><span class="n">batches</span> <span class="o">=</span> <span class="p">[</span><span class="n">batch</span><span class="p">]</span> <span class="o">*</span> <span class="mi">5</span>
+
+<span class="gp">In [50]: </span><span class="n">table</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">Table</span><span class="o">.</span><span class="n">from_batches</span><span class="p">(</span><span class="n">batches</span><span class="p">)</span>
+
+<span class="gp">In [51]: </span><span class="n">table</span>
+<span class="gh">Out[51]: </span><span class="go"></span>
+<span class="go">pyarrow.Table</span>
+<span class="go">f0: int64</span>
+<span class="go">f1: string</span>
+<span class="go">f2: bool</span>
+<span class="go">-- metadata --</span>
+
+<span class="gp">In [52]: </span><span class="n">table</span><span class="o">.</span><span class="n">num_rows</span>
+<span class="go"> Out[52]: 20</span>
+</pre></div>
+</div>
+<p>The table’s columns are instances of <a class="reference internal" href="generated/pyarrow.Column.html#pyarrow.Column" title="pyarrow.Column"><code class="xref py py-class docutils literal"><span class="pre">Column</span></code></a>, which is a container
+for one or more arrays of the same type.</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [53]: </span><span class="n">c</span> <span class="o">=</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
+
+<span class="gp">In [54]: </span><span class="n">c</span>
+<span class="gh">Out[54]: </span><span class="go"><pyarrow._table.Column at 0x2b0bcda54f00></span>
+
+<span class="gp">In [55]: </span><span class="n">c</span><span class="o">.</span><span class="n">data</span>
+<span class="go"> Out[55]: <pyarrow._table.ChunkedArray at 0x2b0bcda54ed0></span>
+
+<span class="gp">In [56]: </span><span class="n">c</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">num_chunks</span>
+<span class="go"> Out[56]: 5</span>
+
+<span class="gp">In [57]: </span><span class="n">c</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">chunk</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
+<span class="go"> Out[57]: </span>
+<span class="go"><pyarrow._array.Int64Array object at 0x2b0bcdd4b3b8></span>
+<span class="go">[</span>
+<span class="go"> 1,</span>
+<span class="go"> 2,</span>
+<span class="go"> 3,</span>
+<span class="go"> 4</span>
+<span class="go">]</span>
+</pre></div>
+</div>
+<p>As you’ll see in the <span class="xref std std-ref">pandas section</span>, we can convert thee
+objects to contiguous NumPy arrays for use in pandas:</p>
+<div class="highlight-ipython"><div class="highlight"><pre><span></span><span class="gp">In [58]: </span><span class="n">c</span><span class="o">.</span><span class="n">to_pandas</span><span class="p">()</span>
+<span class="gh">Out[58]: </span><span class="go"></span>
+<span class="go">0 1</span>
+<span class="go">1 2</span>
+<span class="go">2 3</span>
+<span class="go">3 4</span>
+<span class="go">4 1</span>
+<span class="go">5 2</span>
+<span class="go">6 3</span>
+<span class="go">7 4</span>
+<span class="go">8 1</span>
+<span class="go">9 2</span>
+<span class="go">10 3</span>
+<span class="go">11 4</span>
+<span class="go">12 1</span>
+<span class="go">13 2</span>
+<span class="go">14 3</span>
+<span class="go">15 4</span>
+<span class="go">16 1</span>
+<span class="go">17 2</span>
+<span class="go">18 3</span>
+<span class="go">19 4</span>
+<span class="go">Name: f0, dtype: int64</span>
+</pre></div>
+</div>
+</div>
+<div class="section" id="custom-schema-and-field-metadata">
+<h2>Custom Schema and Field Metadata<a class="headerlink" href="#custom-schema-and-field-metadata" title="Permalink to this headline">¶</a></h2>
+<p>TODO</p>
+</div>
+</div>
+
+
+ </div>
+ </div>
+ </div>
+ <div class="clearer"></div>
+ </div>
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="genindex.html" title="General Index"
+ >index</a></li>
+ <li class="right" >
+ <a href="ipc.html" title="IPC: Fast Streaming and Serialization"
+ >next</a> |</li>
+ <li class="right" >
+ <a href="memory.html" title="Memory and IO Interfaces"
+ >previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="index.html">pyarrow documentation</a> »</li>
+ </ul>
+ </div>
+ <div class="footer" role="contentinfo">
+ © Copyright 2016-2017 Apache Software Foundation.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.
+ </div>
+ </body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/6360599f/docs/python/development.html
----------------------------------------------------------------------
diff --git a/docs/python/development.html b/docs/python/development.html
new file mode 100644
index 0000000..bfc4110
--- /dev/null
+++ b/docs/python/development.html
@@ -0,0 +1,291 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+
+ <title>Development — pyarrow documentation</title>
+
+ <link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
+ <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT: './',
+ VERSION: '',
+ COLLAPSE_INDEX: false,
+ FILE_SUFFIX: '.html',
+ HAS_SOURCE: true,
+ SOURCELINK_SUFFIX: '.txt'
+ };
+ </script>
+ <script type="text/javascript" src="_static/jquery.js"></script>
+ <script type="text/javascript" src="_static/underscore.js"></script>
+ <script type="text/javascript" src="_static/doctools.js"></script>
+ <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <link rel="index" title="Index" href="genindex.html" />
+ <link rel="search" title="Search" href="search.html" />
+ <link rel="next" title="Memory and IO Interfaces" href="memory.html" />
+ <link rel="prev" title="Install PyArrow" href="install.html" />
+ </head>
+ <body role="document">
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="genindex.html" title="General Index"
+ accesskey="I">index</a></li>
+ <li class="right" >
+ <a href="memory.html" title="Memory and IO Interfaces"
+ accesskey="N">next</a> |</li>
+ <li class="right" >
+ <a href="install.html" title="Install PyArrow"
+ accesskey="P">previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="index.html">pyarrow documentation</a> »</li>
+ </ul>
+ </div>
+ <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+ <div class="sphinxsidebarwrapper">
+ <h3><a href="index.html">Table Of Contents</a></h3>
+ <ul>
+<li><a class="reference internal" href="#">Development</a><ul>
+<li><a class="reference internal" href="#developing-with-conda">Developing with conda</a><ul>
+<li><a class="reference internal" href="#linux-and-macos">Linux and macOS</a><ul>
+<li><a class="reference internal" href="#system-requirements">System Requirements</a></li>
+<li><a class="reference internal" href="#environment-setup-and-build">Environment Setup and Build</a></li>
+</ul>
+</li>
+</ul>
+</li>
+<li><a class="reference internal" href="#windows">Windows</a><ul>
+<li><a class="reference internal" href="#running-c-unit-tests-with-python">Running C++ unit tests with Python</a></li>
+</ul>
+</li>
+</ul>
+</li>
+</ul>
+
+ <h4>Previous topic</h4>
+ <p class="topless"><a href="install.html"
+ title="previous chapter">Install PyArrow</a></p>
+ <h4>Next topic</h4>
+ <p class="topless"><a href="memory.html"
+ title="next chapter">Memory and IO Interfaces</a></p>
+ <div role="note" aria-label="source link">
+ <h3>This Page</h3>
+ <ul class="this-page-menu">
+ <li><a href="_sources/development.rst.txt"
+ rel="nofollow">Show Source</a></li>
+ </ul>
+ </div>
+<div id="searchbox" style="display: none" role="search">
+ <h3>Quick search</h3>
+ <form class="search" action="search.html" method="get">
+ <div><input type="text" name="q" /></div>
+ <div><input type="submit" value="Go" /></div>
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+ </div>
+ </div>
+
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body" role="main">
+
+ <div class="section" id="development">
+<span id="id1"></span><h1>Development<a class="headerlink" href="#development" title="Permalink to this headline">¶</a></h1>
+<div class="section" id="developing-with-conda">
+<h2>Developing with conda<a class="headerlink" href="#developing-with-conda" title="Permalink to this headline">¶</a></h2>
+<div class="section" id="linux-and-macos">
+<h3>Linux and macOS<a class="headerlink" href="#linux-and-macos" title="Permalink to this headline">¶</a></h3>
+<div class="section" id="system-requirements">
+<h4>System Requirements<a class="headerlink" href="#system-requirements" title="Permalink to this headline">¶</a></h4>
+<p>On macOS, any modern XCode (6.4 or higher; the current version is 8.3.1) is
+sufficient.</p>
+<p>On Linux, for this guide, we recommend using gcc 4.8 or 4.9, or clang 3.7 or
+higher. You can check your version by running</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>$ gcc --version
+</pre></div>
+</div>
+<p>On Ubuntu 16.04 and higher, you can obtain gcc 4.9 with:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>$ sudo apt-get install g++-4.9
+</pre></div>
+</div>
+<p>Finally, set gcc 4.9 as the active compiler using:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="nb">export</span> <span class="nv">CC</span><span class="o">=</span>gcc-4.9
+<span class="nb">export</span> <span class="nv">CXX</span><span class="o">=</span>g++-4.9
+</pre></div>
+</div>
+</div>
+<div class="section" id="environment-setup-and-build">
+<h4>Environment Setup and Build<a class="headerlink" href="#environment-setup-and-build" title="Permalink to this headline">¶</a></h4>
+<p>First, let’s create a conda environment with all the C++ build and Python
+dependencies from conda-forge:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>conda create -y -q -n pyarrow-dev <span class="se">\</span>
+ <span class="nv">python</span><span class="o">=</span><span class="m">3</span>.6 numpy six setuptools cython pandas pytest <span class="se">\</span>
+ cmake flatbuffers rapidjson boost-cpp thrift-cpp snappy zlib <span class="se">\</span>
+ brotli jemalloc -c conda-forge
+<span class="nb">source</span> activate pyarrow-dev
+</pre></div>
+</div>
+<p>Now, let’s clone the Arrow and Parquet git repositories:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>mkdir repos
+<span class="nb">cd</span> repos
+git clone https://github.com/apache/arrow.git
+git clone https://github.com/apache/parquet-cpp.git
+</pre></div>
+</div>
+<p>You should now see</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>$ ls -l
+total <span class="m">8</span>
+drwxrwxr-x <span class="m">12</span> wesm wesm <span class="m">4096</span> Apr <span class="m">15</span> <span class="m">19</span>:19 arrow/
+drwxrwxr-x <span class="m">12</span> wesm wesm <span class="m">4096</span> Apr <span class="m">15</span> <span class="m">19</span>:19 parquet-cpp/
+</pre></div>
+</div>
+<p>We need to set some environment variables to let Arrow’s build system know
+about our build toolchain:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="nb">export</span> <span class="nv">ARROW_BUILD_TYPE</span><span class="o">=</span>release
+
+<span class="nb">export</span> <span class="nv">ARROW_BUILD_TOOLCHAIN</span><span class="o">=</span><span class="nv">$CONDA_PREFIX</span>
+<span class="nb">export</span> <span class="nv">PARQUET_BUILD_TOOLCHAIN</span><span class="o">=</span><span class="nv">$CONDA_PREFIX</span>
+<span class="nb">export</span> <span class="nv">ARROW_HOME</span><span class="o">=</span><span class="nv">$CONDA_PREFIX</span>
+<span class="nb">export</span> <span class="nv">PARQUET_HOME</span><span class="o">=</span><span class="nv">$CONDA_PREFIX</span>
+</pre></div>
+</div>
+<p>Now build and install the Arrow C++ libraries:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>mkdir arrow/cpp/build
+<span class="nb">pushd</span> arrow/cpp/build
+
+cmake -DCMAKE_BUILD_TYPE<span class="o">=</span><span class="nv">$ARROW_BUILD_TYPE</span> <span class="se">\</span>
+ -DCMAKE_INSTALL_PREFIX<span class="o">=</span><span class="nv">$ARROW_HOME</span> <span class="se">\</span>
+ -DARROW_PYTHON<span class="o">=</span>on <span class="se">\</span>
+ -DARROW_BUILD_TESTS<span class="o">=</span>OFF <span class="se">\</span>
+ ..
+make -j4
+make install
+<span class="nb">popd</span>
+</pre></div>
+</div>
+<p>Now, optionally build and install the Apache Parquet libraries in your
+toolchain:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>mkdir parquet-cpp/build
+<span class="nb">pushd</span> parquet-cpp/build
+
+cmake -DCMAKE_BUILD_TYPE<span class="o">=</span><span class="nv">$ARROW_BUILD_TYPE</span> <span class="se">\</span>
+ -DCMAKE_INSTALL_PREFIX<span class="o">=</span><span class="nv">$PARQUET_HOME</span> <span class="se">\</span>
+ -DPARQUET_BUILD_BENCHMARKS<span class="o">=</span>off <span class="se">\</span>
+ -DPARQUET_BUILD_EXECUTABLES<span class="o">=</span>off <span class="se">\</span>
+ -DPARQUET_ZLIB_VENDORED<span class="o">=</span>off <span class="se">\</span>
+ -DPARQUET_BUILD_TESTS<span class="o">=</span>off <span class="se">\</span>
+ ..
+
+make -j4
+make install
+<span class="nb">popd</span>
+</pre></div>
+</div>
+<p>Now, build pyarrow:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="nb">cd</span> arrow/python
+python setup.py build_ext --build-type<span class="o">=</span><span class="nv">$ARROW_BUILD_TYPE</span> <span class="se">\</span>
+ --with-parquet --with-jemalloc --inplace
+</pre></div>
+</div>
+<p>If you did not build parquet-cpp, you can omit <code class="docutils literal"><span class="pre">--with-parquet</span></code>.</p>
+<p>You should be able to run the unit tests with:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>$ py.test <span class="nv">pyarrow</span>
+<span class="o">================================</span> <span class="nb">test</span> session <span class="nv">starts</span> <span class="o">====================</span>
+platform linux -- Python <span class="m">3</span>.6.1, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
+rootdir: /home/wesm/arrow-clone/python, inifile:
+collected <span class="m">198</span> items
+
+pyarrow/tests/test_array.py ...........
+pyarrow/tests/test_convert_builtin.py .....................
+pyarrow/tests/test_convert_pandas.py .............................
+pyarrow/tests/test_feather.py ..........................
+pyarrow/tests/test_hdfs.py sssssssssssssss
+pyarrow/tests/test_io.py ..................
+pyarrow/tests/test_ipc.py ........
+pyarrow/tests/test_jemalloc.py ss
+pyarrow/tests/test_parquet.py ....................
+pyarrow/tests/test_scalars.py ..........
+pyarrow/tests/test_schema.py .........
+pyarrow/tests/test_table.py .............
+pyarrow/tests/test_tensor.py ................
+
+<span class="o">======================</span> <span class="m">181</span> passed, <span class="m">17</span> skipped in <span class="m">0</span>.98 <span class="nv">seconds</span> <span class="o">===========</span>
+</pre></div>
+</div>
+</div>
+</div>
+</div>
+<div class="section" id="windows">
+<h2>Windows<a class="headerlink" href="#windows" title="Permalink to this headline">¶</a></h2>
+<p>First, make sure you can <a class="reference external" href="https://github.com/apache/arrow/blob/master/cpp/doc/Windows.md">build the C++ library</a>.</p>
+<p>Now, we need to build and install the C++ libraries someplace.</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span>mkdir cpp<span class="se">\b</span>uild
+<span class="nb">cd</span> cpp<span class="se">\b</span>uild
+<span class="nb">set</span> <span class="nv">ARROW_HOME</span><span class="o">=</span>C:<span class="se">\t</span>hirdparty
+cmake -G <span class="s2">"Visual Studio 14 2015 Win64"</span> ^
+ -DCMAKE_INSTALL_PREFIX<span class="o">=</span>%ARROW_HOME% ^
+ -DCMAKE_BUILD_TYPE<span class="o">=</span>Release ^
+ -DARROW_BUILD_TESTS<span class="o">=</span>off ^
+ -DARROW_PYTHON<span class="o">=</span>on ..
+cmake --build . --target INSTALL --config Release
+<span class="nb">cd</span> ..<span class="se">\.</span>.
+</pre></div>
+</div>
+<p>After that, we must put the install directory’s bin path in our <code class="docutils literal"><span class="pre">%PATH%</span></code>:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="nb">set</span> <span class="nv">PATH</span><span class="o">=</span>%ARROW_HOME%<span class="se">\b</span>in<span class="p">;</span>%PATH%
+</pre></div>
+</div>
+<p>Now, we can build pyarrow:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="nb">cd</span> python
+python setup.py build_ext --inplace
+</pre></div>
+</div>
+<div class="section" id="running-c-unit-tests-with-python">
+<h3>Running C++ unit tests with Python<a class="headerlink" href="#running-c-unit-tests-with-python" title="Permalink to this headline">¶</a></h3>
+<p>Getting <code class="docutils literal"><span class="pre">python-test.exe</span></code> to run is a bit tricky because your
+<code class="docutils literal"><span class="pre">%PYTHONPATH%</span></code> must be configured given the active conda environment:</p>
+<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="nb">set</span> <span class="nv">CONDA_ENV</span><span class="o">=</span>C:<span class="se">\U</span>sers<span class="se">\w</span>esm<span class="se">\M</span>iniconda<span class="se">\e</span>nvs<span class="se">\a</span>rrow-test
+<span class="nb">set</span> <span class="nv">PYTHONPATH</span><span class="o">=</span>%CONDA_ENV%<span class="se">\L</span>ib<span class="p">;</span>%CONDA_ENV%<span class="se">\L</span>ib<span class="se">\s</span>ite-packages<span class="p">;</span>%CONDA_ENV%<span class="se">\p</span>ython35.zip<span class="p">;</span>%CONDA_ENV%<span class="se">\D</span>LLs<span class="p">;</span>%CONDA_ENV%
+</pre></div>
+</div>
+<p>Now <code class="docutils literal"><span class="pre">python-test.exe</span></code> or simply <code class="docutils literal"><span class="pre">ctest</span></code> (to run all tests) should work.</p>
+</div>
+</div>
+</div>
+
+
+ </div>
+ </div>
+ </div>
+ <div class="clearer"></div>
+ </div>
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="genindex.html" title="General Index"
+ >index</a></li>
+ <li class="right" >
+ <a href="memory.html" title="Memory and IO Interfaces"
+ >next</a> |</li>
+ <li class="right" >
+ <a href="install.html" title="Install PyArrow"
+ >previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="index.html">pyarrow documentation</a> »</li>
+ </ul>
+ </div>
+ <div class="footer" role="contentinfo">
+ © Copyright 2016-2017 Apache Software Foundation.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.
+ </div>
+ </body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/6360599f/docs/python/filesystems.html
----------------------------------------------------------------------
diff --git a/docs/python/filesystems.html b/docs/python/filesystems.html
index c86da9e..be34885 100644
--- a/docs/python/filesystems.html
+++ b/docs/python/filesystems.html
@@ -1,174 +1,97 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!DOCTYPE html>
-<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
-<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
-<head>
- <meta charset="utf-8">
-
- <meta name="viewport" content="width=device-width, initial-scale=1.0">
-
- <title>File interfaces and Memory Maps — pyarrow documentation</title>
-
-
-
-
-
-
-
-
-
-
-
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-
-
-
-
-
- <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
-
-
-
-
-
- <link rel="index" title="Index"
- href="genindex.html"/>
- <link rel="search" title="Search" href="search.html"/>
- <link rel="top" title="pyarrow documentation" href="index.html"/>
- <link rel="next" title="Reading/Writing Parquet files" href="parquet.html"/>
- <link rel="prev" title="Pandas Interface" href="pandas.html"/>
-
-
- <script src="_static/js/modernizr.min.js"></script>
-
-</head>
-
-<body class="wy-body-for-nav" role="document">
-
-
- <div class="wy-grid-for-nav">
-
+ <title>Filesystem Interfaces — pyarrow documentation</title>
- <nav data-toggle="wy-nav-shift" class="wy-nav-side">
- <div class="wy-side-scroll">
- <div class="wy-side-nav-search">
-
-
-
- <a href="index.html" class="icon icon-home"> pyarrow
-
-
-
- </a>
-
-
-
-
-
-
-
-<div role="search">
- <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
- <input type="text" name="q" placeholder="Search docs" />
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
- </form>
-</div>
-
-
- </div>
-
- <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
-
-
-
-
-
-
- <p class="caption"><span class="caption-text">Getting Started</span></p>
-<ul class="current">
-<li class="toctree-l1"><a class="reference internal" href="install.html">Install PyArrow</a></li>
-<li class="toctree-l1"><a class="reference internal" href="pandas.html">Pandas Interface</a></li>
-<li class="toctree-l1 current"><a class="current reference internal" href="#">File interfaces and Memory Maps</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="#hadoop-file-system-hdfs">Hadoop File System (HDFS)</a></li>
+ <link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
+ <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT: './',
+ VERSION: '',
+ COLLAPSE_INDEX: false,
+ FILE_SUFFIX: '.html',
+ HAS_SOURCE: true,
+ SOURCELINK_SUFFIX: '.txt'
+ };
+ </script>
+ <script type="text/javascript" src="_static/jquery.js"></script>
+ <script type="text/javascript" src="_static/underscore.js"></script>
+ <script type="text/javascript" src="_static/doctools.js"></script>
+ <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <link rel="index" title="Index" href="genindex.html" />
+ <link rel="search" title="Search" href="search.html" />
+ <link rel="next" title="Using PyArrow with pandas" href="pandas.html" />
+ <link rel="prev" title="IPC: Fast Streaming and Serialization" href="ipc.html" />
+ </head>
+ <body role="document">
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="genindex.html" title="General Index"
+ accesskey="I">index</a></li>
+ <li class="right" >
+ <a href="pandas.html" title="Using PyArrow with pandas"
+ accesskey="N">next</a> |</li>
+ <li class="right" >
+ <a href="ipc.html" title="IPC: Fast Streaming and Serialization"
+ accesskey="P">previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="index.html">pyarrow documentation</a> »</li>
+ </ul>
+ </div>
+ <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+ <div class="sphinxsidebarwrapper">
+ <h3><a href="index.html">Table Of Contents</a></h3>
+ <ul>
+<li><a class="reference internal" href="#">Filesystem Interfaces</a><ul>
+<li><a class="reference internal" href="#hadoop-file-system-hdfs">Hadoop File System (HDFS)</a></li>
</ul>
</li>
-<li class="toctree-l1"><a class="reference internal" href="parquet.html">Reading/Writing Parquet files</a></li>
-<li class="toctree-l1"><a class="reference internal" href="api.html">API Reference</a></li>
-<li class="toctree-l1"><a class="reference internal" href="getting_involved.html">Getting Involved</a></li>
-</ul>
-<p class="caption"><span class="caption-text">Additional Features</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="jemalloc.html">jemalloc MemoryPool</a></li>
</ul>
-
-
+ <h4>Previous topic</h4>
+ <p class="topless"><a href="ipc.html"
+ title="previous chapter">IPC: Fast Streaming and Serialization</a></p>
+ <h4>Next topic</h4>
+ <p class="topless"><a href="pandas.html"
+ title="next chapter">Using PyArrow with pandas</a></p>
+ <div role="note" aria-label="source link">
+ <h3>This Page</h3>
+ <ul class="this-page-menu">
+ <li><a href="_sources/filesystems.rst.txt"
+ rel="nofollow">Show Source</a></li>
+ </ul>
+ </div>
+<div id="searchbox" style="display: none" role="search">
+ <h3>Quick search</h3>
+ <form class="search" action="search.html" method="get">
+ <div><input type="text" name="q" /></div>
+ <div><input type="submit" value="Go" /></div>
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
- </nav>
-
- <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
-
-
- <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
-
- <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
- <a href="index.html">pyarrow</a>
-
- </nav>
-
-
-
- <div class="wy-nav-content">
- <div class="rst-content">
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-<div role="navigation" aria-label="breadcrumbs navigation">
-
- <ul class="wy-breadcrumbs">
-
- <li><a href="index.html">Docs</a> »</li>
-
- <li>File interfaces and Memory Maps</li>
-
-
- <li class="wy-breadcrumbs-aside">
-
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body" role="main">
- <a href="_sources/filesystems.rst.txt" rel="nofollow"> View page source</a>
-
-
- </li>
-
- </ul>
-
-
- <hr/>
-</div>
- <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
- <div itemprop="articleBody">
-
- <div class="section" id="file-interfaces-and-memory-maps">
-<h1>File interfaces and Memory Maps<a class="headerlink" href="#file-interfaces-and-memory-maps" title="Permalink to this headline">¶</a></h1>
-<p>PyArrow features a number of file-like interfaces</p>
+ <div class="section" id="filesystem-interfaces">
+<h1>Filesystem Interfaces<a class="headerlink" href="#filesystem-interfaces" title="Permalink to this headline">¶</a></h1>
+<p>In this section, we discuss filesystem-like interfaces in PyArrow.</p>
<div class="section" id="hadoop-file-system-hdfs">
-<h2>Hadoop File System (HDFS)<a class="headerlink" href="#hadoop-file-system-hdfs" title="Permalink to this headline">¶</a></h2>
+<span id="hdfs"></span><h2>Hadoop File System (HDFS)<a class="headerlink" href="#hadoop-file-system-hdfs" title="Permalink to this headline">¶</a></h2>
<p>PyArrow comes with bindings to a C++-based interface to the Hadoop File
System. You connect like so:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pyarrow</span> <span class="kn">as</span> <span class="nn">pa</span>
@@ -187,7 +110,7 @@ relies on some environment variables.</p>
installed somewhere other than <code class="docutils literal"><span class="pre">$HADOOP_HOME/lib/native</span></code>.</li>
<li><code class="docutils literal"><span class="pre">CLASSPATH</span></code>: must contain the Hadoop jars. You can set these using:</li>
</ul>
-<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="nb">export</span> <span class="nv">CLASSPATH</span><span class="o">=</span><span class="sb">`</span>$HADOOP_HOME/bin/hdfs classpath --glob<span class="sb">`</span>
+<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="nb">export</span> <span class="nv">CLASSPATH</span><span class="o">=</span><span class="sb">`</span><span class="nv">$HADOOP_HOME</span>/bin/hdfs classpath --glob<span class="sb">`</span>
</pre></div>
</div>
<p>You can also use libhdfs3, a thirdparty C++ library for HDFS from Pivotal Labs:</p>
@@ -199,76 +122,29 @@ installed somewhere other than <code class="docutils literal"><span class="pre">
</div>
- </div>
- <div class="articleComments">
-
- </div>
</div>
- <footer>
-
- <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
-
- <a href="parquet.html" class="btn btn-neutral float-right" title="Reading/Writing Parquet files" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
-
-
- <a href="pandas.html" class="btn btn-neutral" title="Pandas Interface" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
-
- </div>
-
-
- <hr/>
-
- <div role="contentinfo">
- <p>
- © Copyright 2016 Apache Software Foundation.
-
- </p>
- </div>
- Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
-
-</footer>
-
</div>
</div>
-
- </section>
-
- </div>
-
-
-
-
-
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT:'./',
- VERSION:'',
- COLLAPSE_INDEX:false,
- FILE_SUFFIX:'.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
- <script type="text/javascript" src="_static/jquery.js"></script>
- <script type="text/javascript" src="_static/underscore.js"></script>
- <script type="text/javascript" src="_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
-
-
-
-
-
- <script type="text/javascript" src="_static/js/theme.js"></script>
-
-
-
-
- <script type="text/javascript">
- jQuery(function () {
- SphinxRtdTheme.StickyNav.enable();
- });
- </script>
-
-
-</body>
+ <div class="clearer"></div>
+ </div>
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="genindex.html" title="General Index"
+ >index</a></li>
+ <li class="right" >
+ <a href="pandas.html" title="Using PyArrow with pandas"
+ >next</a> |</li>
+ <li class="right" >
+ <a href="ipc.html" title="IPC: Fast Streaming and Serialization"
+ >previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="index.html">pyarrow documentation</a> »</li>
+ </ul>
+ </div>
+ <div class="footer" role="contentinfo">
+ © Copyright 2016-2017 Apache Software Foundation.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.
+ </div>
+ </body>
</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/6360599f/docs/python/generated/pyarrow.Array.html
----------------------------------------------------------------------
diff --git a/docs/python/generated/pyarrow.Array.html b/docs/python/generated/pyarrow.Array.html
index c115b5e..2e7e0ac 100644
--- a/docs/python/generated/pyarrow.Array.html
+++ b/docs/python/generated/pyarrow.Array.html
@@ -1,208 +1,84 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!DOCTYPE html>
-<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
-<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
-<head>
- <meta charset="utf-8">
-
- <meta name="viewport" content="width=device-width, initial-scale=1.0">
-
- <title>pyarrow.Array — pyarrow documentation</title>
-
-
-
-
-
-
-
-
-
-
-
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-
-
-
-
-
- <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
-
-
-
-
-
- <link rel="index" title="Index"
- href="../genindex.html"/>
- <link rel="search" title="Search" href="../search.html"/>
- <link rel="top" title="pyarrow documentation" href="../index.html"/>
- <link rel="up" title="API Reference" href="../api.html"/>
- <link rel="next" title="pyarrow.BooleanArray" href="pyarrow.BooleanArray.html"/>
- <link rel="prev" title="pyarrow.array" href="pyarrow.array.html"/>
-
-
- <script src="../_static/js/modernizr.min.js"></script>
-
-</head>
-
-<body class="wy-body-for-nav" role="document">
-
-
- <div class="wy-grid-for-nav">
-
+ <title>pyarrow.Array — pyarrow documentation</title>
- <nav data-toggle="wy-nav-shift" class="wy-nav-side">
- <div class="wy-side-scroll">
- <div class="wy-side-nav-search">
-
-
-
- <a href="../index.html" class="icon icon-home"> pyarrow
-
-
-
- </a>
-
-
-
-
-
-
-
-<div role="search">
- <form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
- <input type="text" name="q" placeholder="Search docs" />
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
- </form>
+ <link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
+ <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT: '../',
+ VERSION: '',
+ COLLAPSE_INDEX: false,
+ FILE_SUFFIX: '.html',
+ HAS_SOURCE: true,
+ SOURCELINK_SUFFIX: '.txt'
+ };
+ </script>
+ <script type="text/javascript" src="../_static/jquery.js"></script>
+ <script type="text/javascript" src="../_static/underscore.js"></script>
+ <script type="text/javascript" src="../_static/doctools.js"></script>
+ <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <link rel="index" title="Index" href="../genindex.html" />
+ <link rel="search" title="Search" href="../search.html" />
+ <link rel="next" title="pyarrow.BooleanArray" href="pyarrow.BooleanArray.html" />
+ <link rel="prev" title="pyarrow.array" href="pyarrow.array.html" />
+ </head>
+ <body role="document">
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ accesskey="I">index</a></li>
+ <li class="right" >
+ <a href="pyarrow.BooleanArray.html" title="pyarrow.BooleanArray"
+ accesskey="N">next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.array.html" title="pyarrow.array"
+ accesskey="P">previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" accesskey="U">API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+ <div class="sphinxsidebarwrapper">
+ <h4>Previous topic</h4>
+ <p class="topless"><a href="pyarrow.array.html"
+ title="previous chapter">pyarrow.array</a></p>
+ <h4>Next topic</h4>
+ <p class="topless"><a href="pyarrow.BooleanArray.html"
+ title="next chapter">pyarrow.BooleanArray</a></p>
+ <div role="note" aria-label="source link">
+ <h3>This Page</h3>
+ <ul class="this-page-menu">
+ <li><a href="../_sources/generated/pyarrow.Array.rst.txt"
+ rel="nofollow">Show Source</a></li>
+ </ul>
+ </div>
+<div id="searchbox" style="display: none" role="search">
+ <h3>Quick search</h3>
+ <form class="search" action="../search.html" method="get">
+ <div><input type="text" name="q" /></div>
+ <div><input type="submit" value="Go" /></div>
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
</div>
-
-
- </div>
-
- <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
-
-
-
-
-
-
- <p class="caption"><span class="caption-text">Getting Started</span></p>
-<ul class="current">
-<li class="toctree-l1"><a class="reference internal" href="../install.html">Install PyArrow</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../pandas.html">Pandas Interface</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../filesystems.html">File interfaces and Memory Maps</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../parquet.html">Reading/Writing Parquet files</a></li>
-<li class="toctree-l1 current"><a class="reference internal" href="../api.html">API Reference</a><ul class="current">
-<li class="toctree-l2"><a class="reference internal" href="../api.html#type-and-schema-factory-functions">Type and Schema Factory Functions</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../api.html#scalar-value-types">Scalar Value Types</a></li>
-<li class="toctree-l2 current"><a class="reference internal" href="../api.html#array-types-and-constructors">Array Types and Constructors</a><ul class="current">
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.array.html">pyarrow.array</a></li>
-<li class="toctree-l3 current"><a class="current reference internal" href="#">pyarrow.Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.BooleanArray.html">pyarrow.BooleanArray</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.DictionaryArray.html">pyarrow.DictionaryArray</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.FloatingPointArray.html">pyarrow.FloatingPointArray</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.IntegerArray.html">pyarrow.IntegerArray</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.Int8Array.html">pyarrow.Int8Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.Int16Array.html">pyarrow.Int16Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.Int32Array.html">pyarrow.Int32Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.Int64Array.html">pyarrow.Int64Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.NullArray.html">pyarrow.NullArray</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.NumericArray.html">pyarrow.NumericArray</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.UInt8Array.html">pyarrow.UInt8Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.UInt16Array.html">pyarrow.UInt16Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.UInt32Array.html">pyarrow.UInt32Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.UInt64Array.html">pyarrow.UInt64Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.BinaryArray.html">pyarrow.BinaryArray</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.FixedSizeBinaryArray.html">pyarrow.FixedSizeBinaryArray</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.StringArray.html">pyarrow.StringArray</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.Time32Array.html">pyarrow.Time32Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.Time64Array.html">pyarrow.Time64Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.Date32Array.html">pyarrow.Date32Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.Date64Array.html">pyarrow.Date64Array</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.TimestampArray.html">pyarrow.TimestampArray</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.DecimalArray.html">pyarrow.DecimalArray</a></li>
-<li class="toctree-l3"><a class="reference internal" href="pyarrow.ListArray.html">pyarrow.ListArray</a></li>
-</ul>
-</li>
-<li class="toctree-l2"><a class="reference internal" href="../api.html#tables-and-record-batches">Tables and Record Batches</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../api.html#tensor-type-and-functions">Tensor type and Functions</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../api.html#input-output-and-shared-memory">Input / Output and Shared Memory</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../api.html#interprocess-communication-and-messaging">Interprocess Communication and Messaging</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../api.html#memory-pools">Memory Pools</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../api.html#type-classes">Type Classes</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../api.html#apache-parquet">Apache Parquet</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="../getting_involved.html">Getting Involved</a></li>
-</ul>
-<p class="caption"><span class="caption-text">Additional Features</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../jemalloc.html">jemalloc MemoryPool</a></li>
-</ul>
-
-
-
+<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
- </nav>
-
- <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
-
-
- <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
-
- <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
- <a href="../index.html">pyarrow</a>
-
- </nav>
-
-
-
- <div class="wy-nav-content">
- <div class="rst-content">
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-<div role="navigation" aria-label="breadcrumbs navigation">
-
- <ul class="wy-breadcrumbs">
-
- <li><a href="../index.html">Docs</a> »</li>
-
- <li><a href="../api.html">API Reference</a> »</li>
-
- <li>pyarrow.Array</li>
-
-
- <li class="wy-breadcrumbs-aside">
-
-
- <a href="../_sources/generated/pyarrow.Array.rst.txt" rel="nofollow"> View page source</a>
-
-
- </li>
-
- </ul>
-
-
- <hr/>
-</div>
- <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
- <div itemprop="articleBody">
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body" role="main">
<div class="section" id="pyarrow-array">
<h1>pyarrow.Array<a class="headerlink" href="#pyarrow-array" title="Permalink to this headline">¶</a></h1>
@@ -212,7 +88,7 @@
<dd><dl class="method">
<dt id="pyarrow.Array.__init__">
<code class="descname">__init__</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyarrow.Array.__init__" title="Permalink to this definition">¶</a></dt>
-<dd><p>x.__init__(...) initializes x; see help(type(x)) for signature</p>
+<dd><p>Initialize self. See help(type(self)) for accurate signature.</p>
</dd></dl>
<p class="rubric">Methods</p>
@@ -262,76 +138,30 @@
</div>
- </div>
- <div class="articleComments">
-
- </div>
</div>
- <footer>
-
- <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
-
- <a href="pyarrow.BooleanArray.html" class="btn btn-neutral float-right" title="pyarrow.BooleanArray" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
-
-
- <a href="pyarrow.array.html" class="btn btn-neutral" title="pyarrow.array" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
-
- </div>
-
-
- <hr/>
-
- <div role="contentinfo">
- <p>
- © Copyright 2016 Apache Software Foundation.
-
- </p>
- </div>
- Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
-
-</footer>
-
</div>
</div>
-
- </section>
-
- </div>
-
-
-
-
-
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT:'../',
- VERSION:'',
- COLLAPSE_INDEX:false,
- FILE_SUFFIX:'.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
- <script type="text/javascript" src="../_static/jquery.js"></script>
- <script type="text/javascript" src="../_static/underscore.js"></script>
- <script type="text/javascript" src="../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
-
-
-
-
-
- <script type="text/javascript" src="../_static/js/theme.js"></script>
-
-
-
-
- <script type="text/javascript">
- jQuery(function () {
- SphinxRtdTheme.StickyNav.enable();
- });
- </script>
-
-
-</body>
+ <div class="clearer"></div>
+ </div>
+ <div class="related" role="navigation" aria-label="related navigation">
+ <h3>Navigation</h3>
+ <ul>
+ <li class="right" style="margin-right: 10px">
+ <a href="../genindex.html" title="General Index"
+ >index</a></li>
+ <li class="right" >
+ <a href="pyarrow.BooleanArray.html" title="pyarrow.BooleanArray"
+ >next</a> |</li>
+ <li class="right" >
+ <a href="pyarrow.array.html" title="pyarrow.array"
+ >previous</a> |</li>
+ <li class="nav-item nav-item-0"><a href="../index.html">pyarrow documentation</a> »</li>
+ <li class="nav-item nav-item-1"><a href="../api.html" >API Reference</a> »</li>
+ </ul>
+ </div>
+ <div class="footer" role="contentinfo">
+ © Copyright 2016-2017 Apache Software Foundation.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.5.
+ </div>
+ </body>
</html>
\ No newline at end of file