You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@calcite.apache.org by jh...@apache.org on 2016/02/21 19:03:15 UTC
svn commit: r1731544 - in /calcite/site: docs/stream.html img/pie-chart.png
img/window-types.png
Author: jhyde
Date: Sun Feb 21 18:03:15 2016
New Revision: 1731544
URL: http://svn.apache.org/viewvc?rev=1731544&view=rev
Log:
[CALCITE-1090] Revise Streaming SQL specification
Added:
calcite/site/img/pie-chart.png (with props)
calcite/site/img/window-types.png (with props)
Modified:
calcite/site/docs/stream.html
Modified: calcite/site/docs/stream.html
URL: http://svn.apache.org/viewvc/calcite/site/docs/stream.html?rev=1731544&r1=1731543&r2=1731544&view=diff
==============================================================================
--- calcite/site/docs/stream.html (original)
+++ calcite/site/docs/stream.html Sun Feb 21 18:03:15 2016
@@ -708,6 +708,40 @@
<p>Calcite has extended SQL and relational algebra in order to support
streaming queries.</p>
+<ul id="markdown-toc">
+ <li><a href="#introduction">Introduction</a></li>
+ <li><a href="#an-example-schema">An example schema</a></li>
+ <li><a href="#a-simple-query">A simple query</a></li>
+ <li><a href="#filtering-rows">Filtering rows</a></li>
+ <li><a href="#projecting-expressions">Projecting expressions</a></li>
+ <li><a href="#tumbling-windows">Tumbling windows</a></li>
+ <li><a href="#tumbling-windows-improved">Tumbling windows, improved</a></li>
+ <li><a href="#hopping-windows">Hopping windows</a></li>
+ <li><a href="#grouping-sets">GROUPING SETS</a></li>
+ <li><a href="#filtering-after-aggregation">Filtering after aggregation</a></li>
+ <li><a href="#sub-queries-views-and-sqls-closure-property">Sub-queries, views and SQLâs closure property</a></li>
+ <li><a href="#converting-between-streams-and-relations">Converting between streams and relations</a></li>
+ <li><a href="#the-pie-chart-problem-relational-queries-on-streams">The âpie chartâ problem: Relational queries on streams</a></li>
+ <li><a href="#sorting">Sorting</a></li>
+ <li><a href="#table-constructor">Table constructor</a></li>
+ <li><a href="#sliding-windows">Sliding windows</a></li>
+ <li><a href="#cascading-windows">Cascading windows</a></li>
+ <li><a href="#joining-streams-to-tables">Joining streams to tables</a></li>
+ <li><a href="#joining-streams-to-streams">Joining streams to streams</a></li>
+ <li><a href="#dml">DML</a></li>
+ <li><a href="#punctuation">Punctuation</a></li>
+ <li><a href="#state-of-the-stream">State of the stream</a> <ul>
+ <li><a href="#implemented">Implemented</a></li>
+ <li><a href="#not-implemented">Not implemented</a></li>
+ <li><a href="#to-do-in-this-document">To do in this document</a></li>
+ </ul>
+ </li>
+ <li><a href="#functions">Functions</a></li>
+ <li><a href="#references">References</a></li>
+</ul>
+
+<h2 id="introduction">Introduction</h2>
+
<p>Streams are collections to records that flow continuously, and forever.
Unlike tables, they are not typically stored on disk, but flow over the
network and are held for short periods of time in memory.</p>
@@ -792,10 +826,12 @@ SQL semantics.</p>
a streaming query on a table, or a relational query on a stream, Calcite gives
an error:</p>
-<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="o">></span> <span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">Shipments</span><span class="p">;</span>
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">Shipments</span><span class="p">;</span>
+
<span class="n">ERROR</span><span class="p">:</span> <span class="n">Cannot</span> <span class="k">convert</span> <span class="n">stream</span> <span class="s1">'SHIPMENTS'</span> <span class="k">to</span> <span class="n">a</span> <span class="k">table</span>
-<span class="o">></span> <span class="k">SELECT</span> <span class="n">STREAM</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">Products</span><span class="p">;</span>
+<span class="k">SELECT</span> <span class="n">STREAM</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">Products</span><span class="p">;</span>
+
<span class="n">ERROR</span><span class="p">:</span> <span class="n">Cannot</span> <span class="k">convert</span> <span class="k">table</span> <span class="s1">'PRODUCTS'</span> <span class="k">to</span> <span class="n">a</span> <span class="n">stream</span></code></pre></div>
<h1 id="filtering-rows">Filtering rows</h1>
@@ -852,32 +888,46 @@ differences are:</p>
<li>Is the result a stream or a relation?</li>
</ul>
+<p>There are various window types:</p>
+
+<ul>
+ <li>tumbling window (GROUP BY)</li>
+ <li>hopping window (multi GROUP BY)</li>
+ <li>sliding window (window functions)</li>
+ <li>cascading window (window functions)</li>
+</ul>
+
+<p>and the following diagram shows the kinds of query in which to use them:</p>
+
+<p><img src="/img/window-types.png" alt="Window types" /></p>
+
<p>First weâll look a <em>tumbling window</em>, which is defined by a streaming
<code>GROUP BY</code>. Here is an example:</p>
-<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">FLOOR</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">HOUR</span><span class="p">)</span> <span class="k">AS</span> <span class="n">rowtime</span><span class="p">,</span>
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">CEIL</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">HOUR</span><span class="p">)</span> <span class="k">AS</span> <span class="n">rowtime</span><span class="p">,</span>
<span class="n">productId</span><span class="p">,</span>
<span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="k">AS</span> <span class="k">c</span><span class="p">,</span>
<span class="k">SUM</span><span class="p">(</span><span class="n">units</span><span class="p">)</span> <span class="k">AS</span> <span class="n">units</span>
<span class="k">FROM</span> <span class="n">Orders</span>
-<span class="k">GROUP</span> <span class="k">BY</span> <span class="n">FLOOR</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">HOUR</span><span class="p">),</span> <span class="n">productId</span><span class="p">;</span>
+<span class="k">GROUP</span> <span class="k">BY</span> <span class="n">CEIL</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">HOUR</span><span class="p">),</span> <span class="n">productId</span><span class="p">;</span>
<span class="n">rowtime</span> <span class="o">|</span> <span class="n">productId</span> <span class="o">|</span> <span class="k">c</span> <span class="o">|</span> <span class="n">units</span>
<span class="c1">----------+-----------+---------+-------</span>
- <span class="mi">10</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="mi">24</span>
- <span class="mi">10</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">1</span>
- <span class="mi">10</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">7</span>
- <span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">3</span> <span class="o">|</span> <span class="mi">11</span>
- <span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">40</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">12</span></code></pre></div>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="mi">24</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">1</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">7</span>
+ <span class="mi">12</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">3</span> <span class="o">|</span> <span class="mi">11</span>
+ <span class="mi">12</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">40</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">12</span></code></pre></div>
<p>The result is a stream. At 11 oâclock, Calcite emits a sub-total for every
-<code>productId</code> that had an order since 10 oâclock. At 12 oâclock, it will emit
+<code>productId</code> that had an order since 10 oâclock, timestamped 11 oâclock.
+At 12 oâclock, it will emit
the orders that occurred between 11:00 and 12:00. Each input row contributes to
only one output row.</p>
<p>How did Calcite know that the 10:00:00 sub-totals were complete at 11:00:00,
so that it could emit them? It knows that <code>rowtime</code> is increasing, and it knows
-that <code>FLOOR(rowtime TO HOUR)</code> is also increasing. So, once it has seen a row
+that <code>CEIL(rowtime TO HOUR)</code> is also increasing. So, once it has seen a row
at or after 11:00:00, it will never see a row that will contribute to a 10:00:00
total.</p>
@@ -893,28 +943,166 @@ the column or expression is said to be <
Calcite is
not able to make progress, and it will not allow the query:</p>
-<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="o">></span> <span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">productId</span><span class="p">,</span>
-<span class="o">></span> <span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="k">AS</span> <span class="k">c</span><span class="p">,</span>
-<span class="o">></span> <span class="k">SUM</span><span class="p">(</span><span class="n">units</span><span class="p">)</span> <span class="k">AS</span> <span class="n">units</span>
-<span class="o">></span> <span class="k">FROM</span> <span class="n">Orders</span>
-<span class="o">></span> <span class="k">GROUP</span> <span class="k">BY</span> <span class="n">productId</span><span class="p">;</span>
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">productId</span><span class="p">,</span>
+ <span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="k">AS</span> <span class="k">c</span><span class="p">,</span>
+ <span class="k">SUM</span><span class="p">(</span><span class="n">units</span><span class="p">)</span> <span class="k">AS</span> <span class="n">units</span>
+<span class="k">FROM</span> <span class="n">Orders</span>
+<span class="k">GROUP</span> <span class="k">BY</span> <span class="n">productId</span><span class="p">;</span>
+
<span class="n">ERROR</span><span class="p">:</span> <span class="n">Streaming</span> <span class="n">aggregation</span> <span class="n">requires</span> <span class="k">at</span> <span class="n">least</span> <span class="n">one</span> <span class="n">monotonic</span> <span class="n">expression</span> <span class="k">in</span> <span class="k">GROUP</span> <span class="k">BY</span> <span class="n">clause</span></code></pre></div>
<p>Monotonic and quasi-monotonic columns need to be declared in the schema.
The monotonicity is
enforced when records enter the stream and assumed by queries that read from
that stream. We recommend that you give each stream a timestamp column called
-<code>rowtime</code>, but you can declare others, <code>orderId</code>, for example.</p>
+<code>rowtime</code>, but you can declare others to be monotonic, <code>orderId</code>, for example.</p>
+
+<p>We discuss punctuation, watermarks, and other ways of making progress
+<a href="#punctuation">below</a>.</p>
+
+<h1 id="tumbling-windows-improved">Tumbling windows, improved</h1>
+
+<p>The previous example of tumbling windows was easy to write because the window
+was one hour. For intervals that are not a whole time unit, say 2 hours or
+2 hours and 17 minutes, you cannot use <code>CEIL</code>, and the expression gets more
+complicated.</p>
+
+<p>Calcite supports an alternative syntax for tumbling windows:</p>
+
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">TUMBLE_END</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">)</span> <span class="k">AS</span> <span class="n">rowtime</span><span class="p">,</span>
+ <span class="n">productId</span><span class="p">,</span>
+ <span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="k">AS</span> <span class="k">c</span><span class="p">,</span>
+ <span class="k">SUM</span><span class="p">(</span><span class="n">units</span><span class="p">)</span> <span class="k">AS</span> <span class="n">units</span>
+<span class="k">FROM</span> <span class="n">Orders</span>
+<span class="k">GROUP</span> <span class="k">BY</span> <span class="n">TUMBLE</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">),</span> <span class="n">productId</span><span class="p">;</span>
+
+ <span class="n">rowtime</span> <span class="o">|</span> <span class="n">productId</span> <span class="o">|</span> <span class="k">c</span> <span class="o">|</span> <span class="n">units</span>
+<span class="c1">----------+-----------+---------+-------</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="mi">24</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">1</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">7</span>
+ <span class="mi">12</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">3</span> <span class="o">|</span> <span class="mi">11</span>
+ <span class="mi">12</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">40</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">12</span></code></pre></div>
+
+<p>As you can see, it returns the same results as the previous query. The <code>TUMBLE</code>
+function returns a grouping key that is the same for all the rows that will end
+up in a given summary row; the <code>TUMBLE_END</code> function takes the same arguments
+and returns the time at which that window ends;
+there is also a <code>TUMBLE_START</code> function.</p>
+
+<p><code>TUMBLE</code> has an optional parameter to align the window.
+In the following example,
+we use a 30 minute interval and 0:12 as the alignment time,
+so the query emits summaries at 12 and 42 minutes past each hour:</p>
+
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span>
+ <span class="n">TUMBLE_END</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'30'</span> <span class="k">MINUTE</span><span class="p">,</span> <span class="n">TIME</span> <span class="s1">'0:12'</span><span class="p">)</span> <span class="k">AS</span> <span class="n">rowtime</span><span class="p">,</span>
+ <span class="n">productId</span><span class="p">,</span>
+ <span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="k">AS</span> <span class="k">c</span><span class="p">,</span>
+ <span class="k">SUM</span><span class="p">(</span><span class="n">units</span><span class="p">)</span> <span class="k">AS</span> <span class="n">units</span>
+<span class="k">FROM</span> <span class="n">Orders</span>
+<span class="k">GROUP</span> <span class="k">BY</span> <span class="n">TUMBLE</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'30'</span> <span class="k">MINUTE</span><span class="p">,</span> <span class="n">TIME</span> <span class="s1">'0:12'</span><span class="p">),</span>
+ <span class="n">productId</span><span class="p">;</span>
+
+ <span class="n">rowtime</span> <span class="o">|</span> <span class="n">productId</span> <span class="o">|</span> <span class="k">c</span> <span class="o">|</span> <span class="n">units</span>
+<span class="c1">----------+-----------+---------+-------</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">42</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="mi">24</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">42</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">1</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">42</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">7</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">12</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="mi">7</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">12</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">40</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">12</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">42</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">4</span></code></pre></div>
+
+<h1 id="hopping-windows">Hopping windows</h1>
+
+<p>Hopping windows are a generalization of tumbling windows that allow data to
+be kept in a window for a longer than the emit interval.</p>
+
+<p>For example, the following query emits a row timestamped 11:00 containing data
+from 08:00 to 11:00 (or 10:59.9 if weâre being pedantic),
+and a row timestamped 12:00 containing data from 09:00
+to 12:00.</p>
+
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span>
+ <span class="n">HOP_END</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'3'</span> <span class="n">HOUR</span><span class="p">)</span> <span class="k">AS</span> <span class="n">rowtime</span><span class="p">,</span>
+ <span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="k">AS</span> <span class="k">c</span><span class="p">,</span>
+ <span class="k">SUM</span><span class="p">(</span><span class="n">units</span><span class="p">)</span> <span class="k">AS</span> <span class="n">units</span>
+<span class="k">FROM</span> <span class="n">Orders</span>
+<span class="k">GROUP</span> <span class="k">BY</span> <span class="n">HOP</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'3'</span> <span class="n">HOUR</span><span class="p">);</span>
+
+ <span class="n">rowtime</span> <span class="o">|</span> <span class="k">c</span> <span class="o">|</span> <span class="n">units</span>
+<span class="c1">----------+----------+-------</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">4</span> <span class="o">|</span> <span class="mi">27</span>
+ <span class="mi">12</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">8</span> <span class="o">|</span> <span class="mi">50</span></code></pre></div>
+
+<p>In this query, because the retain period is 3 times the emit period, every input
+row contributes to exactly 3 output rows. Imagine that the <code>HOP</code> function
+generates a collection of group keys for incoming row, and places its values
+in the accumulators of each of those group keys. For example,
+<code>HOP(10:18:00, INTERVAL '1' HOUR, INTERVAL '3')</code> generates 3 periods</p>
+
+<p><code>[08:00, 09:00)
+[09:00, 10:00)
+[10:00, 11:00)
+</code></p>
+
+<p>This raises the possibility of allowing user-defined partitioning functions
+for users who are not happy with the built-in functions <code>HOP</code> and <code>TUMBLE</code>.</p>
+
+<p>We can build complex complex expressions such as an exponentially decaying
+moving average:</p>
+
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">HOP_END</span><span class="p">(</span><span class="n">rowtime</span><span class="p">),</span>
+ <span class="n">productId</span><span class="p">,</span>
+ <span class="k">SUM</span><span class="p">(</span><span class="n">unitPrice</span> <span class="o">*</span> <span class="n">EXP</span><span class="p">((</span><span class="n">rowtime</span> <span class="o">-</span> <span class="n">HOP_START</span><span class="p">(</span><span class="n">rowtime</span><span class="p">))</span> <span class="k">SECOND</span> <span class="o">/</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">))</span>
+ <span class="o">/</span> <span class="k">SUM</span><span class="p">(</span><span class="n">EXP</span><span class="p">((</span><span class="n">rowtime</span> <span class="o">-</span> <span class="n">HOP_START</span><span class="p">(</span><span class="n">rowtime</span><span class="p">))</span> <span class="k">SECOND</span> <span class="o">/</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">))</span>
+<span class="k">FROM</span> <span class="n">Orders</span>
+<span class="k">GROUP</span> <span class="k">BY</span> <span class="n">HOP</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="k">SECOND</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">),</span>
+ <span class="n">productId</span></code></pre></div>
+
+<p>Emits:</p>
+
+<ul>
+ <li>a row at <code>11:00:00</code> containing rows in <code>[10:00:00, 11:00:00)</code>;</li>
+ <li>a row at <code>11:00:01</code> containing rows in <code>[10:00:01, 11:00:01)</code>.</li>
+</ul>
+
+<p>The expression weighs recent orders more heavily than older orders.
+Extending the window from 1 hour to 2 hours or 1 year would have
+virtually no effect on the accuracy of the result (but use more memory
+and compute).</p>
+
+<p>Note that we use <code>HOP_START</code> inside an aggregate function (<code>SUM</code>) because it
+is a value that is constant for all rows within a sub-total. This
+would not be allowed for typical aggregate functions (<code>SUM</code>, <code>COUNT</code>
+etc.).</p>
+
+<p>If you are familiar with <code>GROUPING SETS</code>, you may notice that partitioning
+functions can be seen as a generalization of <code>GROUPING SETS</code>, in that they
+allow an input row to contribute to multiple sub-totals.
+The auxiliary functions for <code>GROUPING SETS</code>,
+such as <code>GROUPING()</code> and <code>GROUP_ID</code>,
+can be used inside aggregate functions, so it is not surprising that
+<code>HOP_START</code> and <code>HOP_END</code> can be used in the same way.</p>
+
+<h1 id="grouping-sets">GROUPING SETS</h1>
+
+<p><code>GROUPING SETS</code> is valid for a streaming query provided that every
+grouping set contains a monotonic or quasi-monotonic expression.</p>
+
+<p><code>CUBE</code> and <code>ROLLUP</code> are not valid for streaming query, because they will
+produce at least one grouping set that aggregates everything (like
+<code>GROUP BY ()</code>).</p>
<h1 id="filtering-after-aggregation">Filtering after aggregation</h1>
<p>As in standard SQL, you can apply a <code>HAVING</code> clause to filter rows emitted by
a streaming <code>GROUP BY</code>:</p>
-<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">FLOOR</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">HOUR</span><span class="p">)</span> <span class="k">AS</span> <span class="n">rowtime</span><span class="p">,</span>
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">TUMBLE_END</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">)</span> <span class="k">AS</span> <span class="n">rowtime</span><span class="p">,</span>
<span class="n">productId</span>
<span class="k">FROM</span> <span class="n">Orders</span>
-<span class="k">GROUP</span> <span class="k">BY</span> <span class="n">FLOOR</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">HOUR</span><span class="p">),</span> <span class="n">productId</span>
+<span class="k">GROUP</span> <span class="k">BY</span> <span class="n">TUMBLE</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">),</span> <span class="n">productId</span>
<span class="k">HAVING</span> <span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="o">></span> <span class="mi">2</span> <span class="k">OR</span> <span class="k">SUM</span><span class="p">(</span><span class="n">units</span><span class="p">)</span> <span class="o">></span> <span class="mi">10</span><span class="p">;</span>
<span class="n">rowtime</span> <span class="o">|</span> <span class="n">productId</span>
@@ -930,12 +1118,12 @@ sub-query:</p>
<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">rowtime</span><span class="p">,</span> <span class="n">productId</span>
<span class="k">FROM</span> <span class="p">(</span>
- <span class="k">SELECT</span> <span class="n">FLOOR</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">HOUR</span><span class="p">)</span> <span class="k">AS</span> <span class="n">rowtime</span><span class="p">,</span>
+ <span class="k">SELECT</span> <span class="n">TUMBLE_END</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">)</span> <span class="k">AS</span> <span class="n">rowtime</span><span class="p">,</span>
<span class="n">productId</span><span class="p">,</span>
<span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="k">AS</span> <span class="k">c</span><span class="p">,</span>
<span class="k">SUM</span><span class="p">(</span><span class="n">units</span><span class="p">)</span> <span class="k">AS</span> <span class="n">su</span>
<span class="k">FROM</span> <span class="n">Orders</span>
- <span class="k">GROUP</span> <span class="k">BY</span> <span class="n">FLOOR</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">HOUR</span><span class="p">),</span> <span class="n">productId</span><span class="p">)</span>
+ <span class="k">GROUP</span> <span class="k">BY</span> <span class="n">TUMBLE</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">),</span> <span class="n">productId</span><span class="p">)</span>
<span class="k">WHERE</span> <span class="k">c</span> <span class="o">></span> <span class="mi">2</span> <span class="k">OR</span> <span class="n">su</span> <span class="o">></span> <span class="mi">10</span><span class="p">;</span>
<span class="n">rowtime</span> <span class="o">|</span> <span class="n">productId</span>
@@ -956,12 +1144,12 @@ any operation you can perform on a table
possible:</p>
<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">CREATE</span> <span class="k">VIEW</span> <span class="n">HourlyOrderTotals</span> <span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="n">productId</span><span class="p">,</span> <span class="k">c</span><span class="p">,</span> <span class="n">su</span><span class="p">)</span> <span class="k">AS</span>
- <span class="k">SELECT</span> <span class="n">FLOOR</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">HOUR</span><span class="p">),</span>
+ <span class="k">SELECT</span> <span class="n">TUMBLE_END</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">),</span>
<span class="n">productId</span><span class="p">,</span>
<span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">),</span>
<span class="k">SUM</span><span class="p">(</span><span class="n">units</span><span class="p">)</span>
<span class="k">FROM</span> <span class="n">Orders</span>
- <span class="k">GROUP</span> <span class="k">BY</span> <span class="n">FLOOR</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">HOUR</span><span class="p">),</span> <span class="n">productId</span><span class="p">;</span>
+ <span class="k">GROUP</span> <span class="k">BY</span> <span class="n">TUMBLE</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">),</span> <span class="n">productId</span><span class="p">;</span>
<span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">rowtime</span><span class="p">,</span> <span class="n">productId</span>
<span class="k">FROM</span> <span class="n">HourlyOrderTotals</span>
@@ -987,12 +1175,12 @@ Nested queries and views help to express
a view:</p>
<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">WITH</span> <span class="n">HourlyOrderTotals</span> <span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="n">productId</span><span class="p">,</span> <span class="k">c</span><span class="p">,</span> <span class="n">su</span><span class="p">)</span> <span class="k">AS</span> <span class="p">(</span>
- <span class="k">SELECT</span> <span class="n">FLOOR</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">HOUR</span><span class="p">),</span>
+ <span class="k">SELECT</span> <span class="n">TUMBLE_END</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">),</span>
<span class="n">productId</span><span class="p">,</span>
<span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">),</span>
<span class="k">SUM</span><span class="p">(</span><span class="n">units</span><span class="p">)</span>
<span class="k">FROM</span> <span class="n">Orders</span>
- <span class="k">GROUP</span> <span class="k">BY</span> <span class="n">FLOOR</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">HOUR</span><span class="p">),</span> <span class="n">productId</span><span class="p">)</span>
+ <span class="k">GROUP</span> <span class="k">BY</span> <span class="n">TUMBLE</span><span class="p">(</span><span class="n">rowtime</span><span class="p">,</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">),</span> <span class="n">productId</span><span class="p">)</span>
<span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">rowtime</span><span class="p">,</span> <span class="n">productId</span>
<span class="k">FROM</span> <span class="n">HourlyOrderTotals</span>
<span class="k">WHERE</span> <span class="k">c</span> <span class="o">></span> <span class="mi">2</span> <span class="k">OR</span> <span class="n">su</span> <span class="o">></span> <span class="mi">10</span><span class="p">;</span>
@@ -1003,7 +1191,7 @@ a view:</p>
<span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span>
<span class="mi">11</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">40</span></code></pre></div>
-<h2 id="converting-between-streams-and-relations">Converting between streams and relations</h2>
+<h1 id="converting-between-streams-and-relations">Converting between streams and relations</h1>
<p>Look back at the definition of the <code>HourlyOrderTotals</code> view.
Is the view a stream or a relation?</p>
@@ -1041,70 +1229,39 @@ data in an Apache Kafka [<a href="#ref2"
but not all. At run time, Calcite figures out whether there is sufficient
history to run the query, and if not, gives an error.</p>
-<h2 id="hopping-windows">Hopping windows</h2>
+<h1 id="the-pie-chart-problem-relational-queries-on-streams">The âpie chartâ problem: Relational queries on streams</h1>
-<p>Previously we saw how to define a tumbling window using a <code>GROUP BY</code> clause.
-Each record contributed to a single sub-total record, the one containing its
-hour and product id.</p>
-
-<p>But suppose we want to emit, every hour, the number of each product ordered over
-the past three hours. To do this, we use <code>SELECT ... OVER</code> and a sliding window
-to combine multiple tumbling windows.</p>
+<p>One particular case where you need to convert a stream to a relation
+occurs in what I call the âpie chart problemâ. Imagine that you need to
+write a web page with a chart, like the following, that summarizes the
+number of orders for each product over the last hour.</p>
-<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">rowtime</span><span class="p">,</span>
- <span class="n">productId</span><span class="p">,</span>
- <span class="k">SUM</span><span class="p">(</span><span class="n">su</span><span class="p">)</span> <span class="n">OVER</span> <span class="n">w</span> <span class="k">AS</span> <span class="n">su</span><span class="p">,</span>
- <span class="k">SUM</span><span class="p">(</span><span class="k">c</span><span class="p">)</span> <span class="n">OVER</span> <span class="n">w</span> <span class="k">AS</span> <span class="k">c</span>
-<span class="k">FROM</span> <span class="n">HourlyTotals</span>
-<span class="n">WINDOW</span> <span class="n">w</span> <span class="k">AS</span> <span class="p">(</span>
- <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">rowtime</span>
- <span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">productId</span>
- <span class="n">RANGE</span> <span class="nb">INTERVAL</span> <span class="s1">'2'</span> <span class="n">HOUR</span> <span class="n">PRECEDING</span><span class="p">)</span></code></pre></div>
+<p><img src="/img/pie-chart.png" alt="Pie chart" /></p>
-<p>This query uses the <code>HourlyOrderTotals</code> view defined previously.
-The 2 hour interval combines the totals timestamped 09:00:00, 10:00:00 and
-11:00:00 for a particular product into a single total timestamped 11:00:00 and
-summarizing orders for that product between 09:00:00 and 12:00:00.</p>
-
-<h2 id="limitations-of-tumbling-and-hopping-windows">Limitations of tumbling and hopping windows</h2>
-
-<p>In the present syntax, we acknowledge that it is not easy to create certain
-kinds of windows.</p>
-
-<p>First, letâs consider tumbling windows over complex periods.</p>
-
-<p>The <code>FLOOR</code> and <code>CEIL</code> functions make is easy to create a tumbling window that
-emits on a whole time unit (say every hour, or every minute) but less easy to
-emit, say, every 15 minutes. One could imagine an extension to the <code>FLOOR</code>
-function that emits unique values on just about any periodic basis (say in 11
-minute intervals starting from midnight of the current day).</p>
-
-<p>Next, letâs consider hopping windows whose retention period is not a multiple
-of its emission period. Say we want to output, at the top of each hour, the
-orders for the previous 7,007 seconds. If we were to simulate this hopping
-window using a sliding window over a tumbling window, as before, we would have
-to sum lots of 1-second windows (because 3,600 and 7,007 are co-prime).
-This is a lot of effort for both the system and the person writing the query.</p>
-
-<p>Calcite could perhaps solve this generalizing <code>GROUP BY</code> syntax, but we would
-be destroying the principle that an input row into a <code>GROUP BY</code> appears in
-precisely one output row.</p>
-
-<p>Calciteâs SQL extensions for streaming queries are evolving. As we learn more
-about how people wish to query streams, we plan to make the language more
-expressive while remaining compatible with standard SQL and consistent with
-its principles, look and feel.</p>
+<p>But the <code>Orders</code> stream only contains a few records, not an hourâs summary.
+We need to run a relational query on the history of the stream:</p>
-<h2 id="sorting">Sorting</h2>
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">productId</span><span class="p">,</span> <span class="k">count</span><span class="p">(</span><span class="o">*</span><span class="p">)</span>
+<span class="k">FROM</span> <span class="n">Orders</span>
+<span class="k">WHERE</span> <span class="n">rowtime</span> <span class="k">BETWEEN</span> <span class="k">current_timestamp</span> <span class="o">-</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span>
+ <span class="k">AND</span> <span class="k">current_timestamp</span><span class="p">;</span></code></pre></div>
+
+<p>If the history of the <code>Orders</code> stream is being spooled to the <code>Orders</code> table,
+we can answer the query, albeit at a high cost. Better, if we can tell the
+system to materialize one hour summary into a table,
+maintain it continuously as the stream flows,
+and automatically rewrite queries to use the table.</p>
+
+<h1 id="sorting">Sorting</h1>
<p>The story for <code>ORDER BY</code> is similar to <code>GROUP BY</code>.
The syntax looks like regular SQL, but Calcite must be sure that it can deliver
timely results. It therefore requires a monotonic expression on the leading edge
of your <code>ORDER BY</code> key.</p>
-<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">FLOOR</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">hour</span><span class="p">)</span> <span class="k">AS</span> <span class="n">rowtime</span><span class="p">,</span> <span class="n">productId</span><span class="p">,</span> <span class="n">orderId</span><span class="p">,</span> <span class="n">units</span>
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">CEIL</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">hour</span><span class="p">)</span> <span class="k">AS</span> <span class="n">rowtime</span><span class="p">,</span> <span class="n">productId</span><span class="p">,</span> <span class="n">orderId</span><span class="p">,</span> <span class="n">units</span>
<span class="k">FROM</span> <span class="n">Orders</span>
-<span class="k">ORDER</span> <span class="k">BY</span> <span class="n">FLOOR</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">hour</span><span class="p">)</span> <span class="k">ASC</span><span class="p">,</span> <span class="n">units</span> <span class="k">DESC</span><span class="p">;</span>
+<span class="k">ORDER</span> <span class="k">BY</span> <span class="n">CEIL</span><span class="p">(</span><span class="n">rowtime</span> <span class="k">TO</span> <span class="n">hour</span><span class="p">)</span> <span class="k">ASC</span><span class="p">,</span> <span class="n">units</span> <span class="k">DESC</span><span class="p">;</span>
<span class="n">rowtime</span> <span class="o">|</span> <span class="n">productId</span> <span class="o">|</span> <span class="n">orderId</span> <span class="o">|</span> <span class="n">units</span>
<span class="c1">----------+-----------+---------+-------</span>
@@ -1169,7 +1326,7 @@ which is only slightly less efficient.</
say, perform <code>GROUP BY</code> after a <code>UNION ALL</code>, Calcite will add an <code>ORDER BY</code>
implicitly, in order to make the GROUP BY algorithm possible.</p>
-<h2 id="table-constructor">Table constructor</h2>
+<h1 id="table-constructor">Table constructor</h1>
<p>The <code>VALUES</code> clause creates an inline table with a given set of rows.</p>
@@ -1180,7 +1337,7 @@ would never return any rows.</p>
<span class="n">ERROR</span><span class="p">:</span> <span class="n">Cannot</span> <span class="n">stream</span> <span class="k">VALUES</span></code></pre></div>
-<h2 id="sliding-windows">Sliding windows</h2>
+<h1 id="sliding-windows">Sliding windows</h1>
<p>Standard SQL features so-called âanalytic functionsâ that can be used in the
<code>SELECT</code> clause. Unlike <code>GROUP BY</code>, these do not collapse records. For each
@@ -1233,7 +1390,7 @@ into the query.</p>
<li>You can compute order-dependent functions such as <code>RANK</code> and median.</li>
</ul>
-<h2 id="cascading-windows">Cascading windows</h2>
+<h1 id="cascading-windows">Cascading windows</h1>
<p>What if we want a query that returns a result for every record, like a
sliding window, but resets totals on a fixed time period, like a
@@ -1248,9 +1405,10 @@ is an example:</p>
<p>It looks similar to a sliding window query, but the monotonic
expression occurs within the <code>PARTITION BY</code> clause of the window. As
-the rowtime moves from from 10:59:59 to 11:00:00, <code>FLOOR(rowtime TO
-HOUR)</code> changes from 10:00:00 to 11:00:00, and therefore a new
-partition starts. The first row to arrive in the new hour will start a
+the rowtime moves from from 10:59:59 to 11:00:00,
+<code>FLOOR(rowtime TO HOUR)</code> changes from 10:00:00 to 11:00:00,
+and therefore a new partition starts.
+The first row to arrive in the new hour will start a
new total; the second row will have a total that consists of two rows,
and so on.</p>
@@ -1260,60 +1418,286 @@ removes all sub-totals for that partitio
<p>Analytic functions that using cascading and sliding windows can be
combined in the same query.</p>
-<h2 id="state-of-the-stream">State of the stream</h2>
+<h1 id="joining-streams-to-tables">Joining streams to tables</h1>
+
+<p>There are two kinds of join where streams are concerned: stream-to-table
+join and stream-to-stream join.</p>
+
+<p>A stream-to-table join is straightforward if the contents of the table
+are not changing. This query enriches a stream of orders with
+each productâs list price:</p>
+
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">o</span><span class="p">.</span><span class="n">rowtime</span><span class="p">,</span> <span class="n">o</span><span class="p">.</span><span class="n">productId</span><span class="p">,</span> <span class="n">o</span><span class="p">.</span><span class="n">orderId</span><span class="p">,</span> <span class="n">o</span><span class="p">.</span><span class="n">units</span><span class="p">,</span>
+ <span class="n">p</span><span class="p">.</span><span class="n">name</span><span class="p">,</span> <span class="n">p</span><span class="p">.</span><span class="n">unitPrice</span>
+<span class="k">FROM</span> <span class="n">Orders</span> <span class="k">AS</span> <span class="n">o</span>
+<span class="k">JOIN</span> <span class="n">Products</span> <span class="k">AS</span> <span class="n">p</span>
+ <span class="k">ON</span> <span class="n">o</span><span class="p">.</span><span class="n">productId</span> <span class="o">=</span> <span class="n">p</span><span class="p">.</span><span class="n">productId</span><span class="p">;</span>
+
+ <span class="n">rowtime</span> <span class="o">|</span> <span class="n">productId</span> <span class="o">|</span> <span class="n">orderId</span> <span class="o">|</span> <span class="n">units</span> <span class="o">|</span> <span class="n">name</span> <span class="o">|</span> <span class="n">unitPrice</span>
+<span class="c1">----------+-----------+---------+-------+ -------+-----------</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">17</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mi">5</span> <span class="o">|</span> <span class="mi">4</span> <span class="o">|</span> <span class="n">Cheese</span> <span class="o">|</span> <span class="mi">17</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">17</span><span class="p">:</span><span class="mi">05</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">6</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">Beer</span> <span class="o">|</span> <span class="mi">0</span><span class="p">.</span><span class="mi">25</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">18</span><span class="p">:</span><span class="mi">05</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="mi">7</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">Wine</span> <span class="o">|</span> <span class="mi">6</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">18</span><span class="p">:</span><span class="mi">07</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mi">8</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="n">Cheese</span> <span class="o">|</span> <span class="mi">17</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">02</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">9</span> <span class="o">|</span> <span class="mi">6</span> <span class="o">|</span> <span class="n">Beer</span> <span class="o">|</span> <span class="mi">0</span><span class="p">.</span><span class="mi">25</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">04</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">Beer</span> <span class="o">|</span> <span class="mi">0</span><span class="p">.</span><span class="mi">25</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">09</span><span class="p">:</span><span class="mi">30</span> <span class="o">|</span> <span class="mi">40</span> <span class="o">|</span> <span class="mi">11</span> <span class="o">|</span> <span class="mi">12</span> <span class="o">|</span> <span class="n">Bread</span> <span class="o">|</span> <span class="mi">100</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">24</span><span class="p">:</span><span class="mi">11</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">12</span> <span class="o">|</span> <span class="mi">4</span> <span class="o">|</span> <span class="n">Beer</span> <span class="o">|</span> <span class="mi">0</span><span class="p">.</span><span class="mi">25</span></code></pre></div>
+
+<p>What should happen if the table is changing? For example,
+suppose the unit price of product 10 is increased to 0.35 at 11:00.
+Orders placed before 11:00 should have the old price, and orders
+placed after 11:00 should reflect the new price.</p>
+
+<p>One way to implement this is to have a table that keeps every version
+with a start and end effective date, <code>ProductVersions</code> in the following
+example:</p>
+
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="o">*</span>
+<span class="k">FROM</span> <span class="n">Orders</span> <span class="k">AS</span> <span class="n">o</span>
+<span class="k">JOIN</span> <span class="n">ProductVersions</span> <span class="k">AS</span> <span class="n">p</span>
+ <span class="k">ON</span> <span class="n">o</span><span class="p">.</span><span class="n">productId</span> <span class="o">=</span> <span class="n">p</span><span class="p">.</span><span class="n">productId</span>
+ <span class="k">AND</span> <span class="n">o</span><span class="p">.</span><span class="n">rowtime</span> <span class="k">BETWEEN</span> <span class="n">p</span><span class="p">.</span><span class="n">startDate</span> <span class="k">AND</span> <span class="n">p</span><span class="p">.</span><span class="n">endDate</span>
+
+ <span class="n">rowtime</span> <span class="o">|</span> <span class="n">productId</span> <span class="o">|</span> <span class="n">orderId</span> <span class="o">|</span> <span class="n">units</span> <span class="o">|</span> <span class="n">productId1</span> <span class="o">|</span> <span class="n">name</span> <span class="o">|</span> <span class="n">unitPrice</span>
+<span class="c1">----------+-----------+---------+-------+ -----------+--------+-----------</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">17</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mi">5</span> <span class="o">|</span> <span class="mi">4</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="n">Cheese</span> <span class="o">|</span> <span class="mi">17</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">17</span><span class="p">:</span><span class="mi">05</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">6</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="n">Beer</span> <span class="o">|</span> <span class="mi">0</span><span class="p">.</span><span class="mi">25</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">18</span><span class="p">:</span><span class="mi">05</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="mi">7</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="n">Wine</span> <span class="o">|</span> <span class="mi">6</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">18</span><span class="p">:</span><span class="mi">07</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mi">8</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="n">Cheese</span> <span class="o">|</span> <span class="mi">17</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">02</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">9</span> <span class="o">|</span> <span class="mi">6</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="n">Beer</span> <span class="o">|</span> <span class="mi">0</span><span class="p">.</span><span class="mi">35</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">04</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="n">Beer</span> <span class="o">|</span> <span class="mi">0</span><span class="p">.</span><span class="mi">35</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">09</span><span class="p">:</span><span class="mi">30</span> <span class="o">|</span> <span class="mi">40</span> <span class="o">|</span> <span class="mi">11</span> <span class="o">|</span> <span class="mi">12</span> <span class="o">|</span> <span class="mi">40</span> <span class="o">|</span> <span class="n">Bread</span> <span class="o">|</span> <span class="mi">100</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">24</span><span class="p">:</span><span class="mi">11</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">12</span> <span class="o">|</span> <span class="mi">4</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="n">Beer</span> <span class="o">|</span> <span class="mi">0</span><span class="p">.</span><span class="mi">35</span></code></pre></div>
+
+<p>The other way to implement this is to use a database with temporal support
+(the ability to find the contents of the database as it was at any moment
+in the past), and the system needs to know that the <code>rowtime</code> column of
+the <code>Orders</code> stream corresponds to the transaction timestamp of the
+<code>Products</code> table.</p>
+
+<p>For many applications, it is not worth the cost and effort of temporal
+support or a versioned table. It is acceptable to the application that
+the query gives different results when replayed: in this example, on replay,
+all orders of product 10 are assigned the later unit price, 0.35.</p>
+
+<h1 id="joining-streams-to-streams">Joining streams to streams</h1>
+
+<p>It makes sense to join two streams if the join condition somehow forces
+them to remain a finite distance from one another. In the following query,
+the ship date is within one hour of the order date:</p>
+
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">o</span><span class="p">.</span><span class="n">rowtime</span><span class="p">,</span> <span class="n">o</span><span class="p">.</span><span class="n">productId</span><span class="p">,</span> <span class="n">o</span><span class="p">.</span><span class="n">orderId</span><span class="p">,</span> <span class="n">s</span><span class="p">.</span><span class="n">rowtime</span> <span class="k">AS</span> <span class="n">shipTime</span>
+<span class="k">FROM</span> <span class="n">Orders</span> <span class="k">AS</span> <span class="n">o</span>
+<span class="k">JOIN</span> <span class="n">Shipments</span> <span class="k">AS</span> <span class="n">s</span>
+ <span class="k">ON</span> <span class="n">o</span><span class="p">.</span><span class="n">orderId</span> <span class="o">=</span> <span class="n">p</span><span class="p">.</span><span class="n">orderId</span>
+ <span class="k">AND</span> <span class="n">s</span><span class="p">.</span><span class="n">rowtime</span> <span class="k">BETWEEN</span> <span class="n">o</span><span class="p">.</span><span class="n">rowtime</span> <span class="k">AND</span> <span class="n">o</span><span class="p">.</span><span class="n">rowtime</span> <span class="o">+</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span><span class="p">;</span>
+
+ <span class="n">rowtime</span> <span class="o">|</span> <span class="n">productId</span> <span class="o">|</span> <span class="n">orderId</span> <span class="o">|</span> <span class="n">shipTime</span>
+<span class="c1">----------+-----------+---------+----------</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">17</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mi">5</span> <span class="o">|</span> <span class="mi">10</span><span class="p">:</span><span class="mi">55</span><span class="p">:</span><span class="mi">00</span>
+ <span class="mi">10</span><span class="p">:</span><span class="mi">17</span><span class="p">:</span><span class="mi">05</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">6</span> <span class="o">|</span> <span class="mi">10</span><span class="p">:</span><span class="mi">20</span><span class="p">:</span><span class="mi">00</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">02</span><span class="p">:</span><span class="mi">00</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">9</span> <span class="o">|</span> <span class="mi">11</span><span class="p">:</span><span class="mi">58</span><span class="p">:</span><span class="mi">00</span>
+ <span class="mi">11</span><span class="p">:</span><span class="mi">24</span><span class="p">:</span><span class="mi">11</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mi">12</span> <span class="o">|</span> <span class="mi">11</span><span class="p">:</span><span class="mi">44</span><span class="p">:</span><span class="mi">00</span></code></pre></div>
+
+<p>Note that quite a few orders do not appear, because they did not ship
+within an hour. By the time the system receives order 10, timestamped 11:24:11,
+it has already removed orders up to and including order 8, timestamped 10:18:07,
+from its hash table.</p>
+
+<p>As you can see, the âlock stepâ, tying together monotonic or quasi-monotonic
+columns of the two streams, is necessary for the system to make progress.
+It will refuse to execute a query if it cannot deduce a lock step.</p>
+
+<h1 id="dml">DML</h1>
+
+<p>Itâs not only queries that make sense against streams;
+it also makes sense to run DML statements (<code>INSERT</code>, <code>UPDATE</code>, <code>DELETE</code>,
+and also their rarer cousins <code>UPSERT</code> and <code>REPLACE</code>) against streams.</p>
+
+<p>DML is useful because it allows you do materialize streams
+or tables based on streams,
+and therefore save effort when values are used often.</p>
+
+<p>Consider how streaming applications often consist of pipelines of queries,
+each query transforming input stream(s) to output stream(s).
+The component of a pipeline can be a view:</p>
+
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">CREATE</span> <span class="k">VIEW</span> <span class="n">LargeOrders</span> <span class="k">AS</span>
+<span class="k">SELECT</span> <span class="n">STREAM</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">Orders</span> <span class="k">WHERE</span> <span class="n">units</span> <span class="o">></span> <span class="mi">1000</span><span class="p">;</span></code></pre></div>
+
+<p>or a standing <code>INSERT</code> statement:</p>
+
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">LargeOrders</span>
+<span class="k">SELECT</span> <span class="n">STREAM</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">Orders</span> <span class="k">WHERE</span> <span class="n">units</span> <span class="o">></span> <span class="mi">1000</span><span class="p">;</span></code></pre></div>
+
+<p>These look similar, and in both cases the next step(s) in the pipeline
+can read from <code>LargeOrders</code> without worrying how it was populated.
+There is a difference in efficiency: the <code>INSERT</code> statement does the
+same work no matter how many consumers there are; the view does work
+proportional to the number of consumers, and in particular, does no
+work if there are no consumers.</p>
+
+<p>Other forms of DML make sense for streams. For example, the following
+standing <code>UPSERT</code> statement maintains a table that materializes a summary
+of the last hour of orders:</p>
+
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="n">UPSERT</span> <span class="k">INTO</span> <span class="n">OrdersSummary</span>
+<span class="k">SELECT</span> <span class="n">STREAM</span> <span class="n">productId</span><span class="p">,</span>
+ <span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="n">OVER</span> <span class="n">lastHour</span> <span class="k">AS</span> <span class="k">c</span>
+<span class="k">FROM</span> <span class="n">Orders</span>
+<span class="n">WINDOW</span> <span class="n">lastHour</span> <span class="k">AS</span> <span class="p">(</span>
+ <span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">productId</span>
+ <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">rowtime</span>
+ <span class="n">RANGE</span> <span class="nb">INTERVAL</span> <span class="s1">'1'</span> <span class="n">HOUR</span> <span class="n">PRECEDING</span><span class="p">)</span></code></pre></div>
+
+<h1 id="punctuation">Punctuation</h1>
+
+<p>Punctuation[<a href="#ref5">5</a>] allows a stream query to make progress
+even if there are not enough values in a monotonic key to push the results out.</p>
+
+<p>(I prefer the term ârowtime boundsâ,
+and watermarks[<a href="#ref6">6</a>] are a related concept,
+but for these purposes, punctuation will suffice.)</p>
+
+<p>If a stream has punctuation enabled then it may not be sorted but is
+nevertheless sortable. So, for the purposes of semantics, it is sufficient
+to work in terms of sorted streams.</p>
+
+<p>By the way, an out-of-order stream is also sortable if it is <em>t-sorted</em>
+(i.e. every record is guaranteed to arrive within <em>t</em> seconds of its
+timestamp) or <em>k-sorted</em> (i.e. every record is guaranteed to be no more
+than <em>k</em> positions out of order). So queries on these streams can be
+planned similarly to queries on streams with punctuation.</p>
+
+<p>And, we often want to aggregate over attributes that are not
+time-based but are nevertheless monotonic. âThe number of times a team
+has shifted between winning-state and losing-stateâ is one such
+monotonic attribute. The system needs to figure out for itself that it
+is safe to aggregate over such an attribute; punctuation does not add
+any extra information.</p>
+
+<p>I have in mind some metadata (cost metrics) for the planner:</p>
+
+<ol>
+ <li>Is this stream sorted on a given attribute (or attributes)?</li>
+ <li>Is it possible to sort the stream on a given attribute? (For finite
+relations, the answer is always âyesâ; for streams it depends on the
+existence of punctuation, or linkage between the attributes and the
+sort key.)</li>
+ <li>What latency do we need to introduce in order to perform that sort?</li>
+ <li>What is the cost (in CPU, memory etc.) of performing that sort?</li>
+</ol>
+
+<p>We already have (1), in <a href="/apidocs/org/apache/calcite/rel/metadata/BuiltInMetadata.Collation.html">BuiltInMetadata.Collation</a>.
+For (2), the answer is always âtrueâ for finite relations.
+But weâll need to implement (2), (3) and (4) for streams.</p>
+
+<h1 id="state-of-the-stream">State of the stream</h1>
<p>Not all concepts in this article have been implemented in Calcite.
And others may be implemented in Calcite but not in a particular adapter
-such as Samza SQL [<a href="#ref3">3</a>].</p>
+such as SamzaSQL [<a href="#ref3">3</a>] [<a href="#ref4">4</a>].</p>
+
+<h2 id="implemented">Implemented</h2>
-<h3 id="implemented">Implemented</h3>
<ul>
- <li>Streaming SELECT, WHERE, GROUP BY, HAVING, UNION ALL, ORDER BY</li>
- <li>FLOOR and CEILING functions</li>
+ <li>Streaming <code>SELECT</code>, <code>WHERE</code>, <code>GROUP BY</code>, <code>HAVING</code>, <code>UNION ALL</code>, <code>ORDER BY</code></li>
+ <li><code>FLOOR</code> and <code>CEIL</code> functions</li>
<li>Monotonicity</li>
- <li>Streaming VALUES is disallowed</li>
+ <li>Streaming <code>VALUES</code> is disallowed</li>
</ul>
-<h3 id="not-implemented">Not implemented</h3>
+<h2 id="not-implemented">Not implemented</h2>
+
+<p>The following features are presented in this document as if Calcite
+supports them, but in fact it does not (yet). Full support means
+that the reference implementation supports the feature (including
+negative cases) and the TCK tests it.</p>
+
<ul>
- <li>Stream-to-stream JOIN</li>
- <li>Stream-to-table JOIN</li>
+ <li>Stream-to-stream <code>JOIN</code></li>
+ <li>Stream-to-table <code>JOIN</code></li>
<li>Stream on view</li>
- <li>Streaming UNION ALL with ORDER BY (merge)</li>
+ <li>Streaming <code>UNION ALL</code> with <code>ORDER BY</code> (merge)</li>
<li>Relational query on stream</li>
<li>Streaming windowed aggregation (sliding and cascading windows)</li>
- <li>Check that STREAM in sub-queries and views is ignored</li>
- <li>Check that streaming ORDER BY cannot have OFFSET or LIMIT</li>
+ <li>Check that <code>STREAM</code> in sub-queries and views is ignored</li>
+ <li>Check that streaming <code>ORDER BY</code> cannot have <code>OFFSET</code> or <code>LIMIT</code></li>
<li>Limited history; at run time, check that there is sufficient history
to run the query.</li>
+ <li><a href="https://issues.apache.org/jira/browse/CALCITE-1096">Quasi-monotonicity</a></li>
+ <li><code>HOP</code> and <code>TUMBLE</code> (and auxiliary <code>HOP_START</code>, <code>HOP_END</code>,
+<code>TUMBLE_START</code>, <code>TUMBLE_END</code>) functions</li>
</ul>
-<h3 id="to-do-in-this-document">To do in this document</h3>
+<h2 id="to-do-in-this-document">To do in this document</h2>
+
<ul>
- <li>Re-visit whether you can stream VALUES</li>
- <li>OVER clause to define window on stream</li>
- <li>Windowed aggregation</li>
- <li>Punctuation</li>
- <li>Stream-to-table join
- <ul>
- <li>Stream-to-table join where table is changing</li>
- </ul>
- </li>
- <li>Stream-to-stream join</li>
- <li>Relational queries on streams (e.g. âpie chartâ query)</li>
- <li>Diagrams for various window types</li>
+ <li>Re-visit whether you can stream <code>VALUES</code></li>
+ <li><code>OVER</code> clause to define window on stream</li>
+ <li>Consider whether to allow <code>CUBE</code> and <code>ROLLUP</code> in streaming queries,
+with an understanding that some levels of aggregation will never complete
+(because they have no monotonic expressions) and thus will never be emitted.</li>
+ <li>Fix the <code>UPSERT</code> example to remove records for products that have not
+occurred in the last hour.</li>
+ <li>DML that outputs to multiple streams; perhaps an extension to the standard
+<code>REPLACE</code> statement. </li>
+</ul>
+
+<h1 id="functions">Functions</h1>
+
+<p>The following functions are not present in standard SQL
+but are defined in streaming SQL.</p>
+
+<p>Scalar functions:</p>
+
+<ul>
+ <li><code>FLOOR(dateTime TO intervalType)</code> rounds a date, time or timestamp value
+down to a given interval type</li>
+ <li><code>CEIL(dateTime TO intervalType)</code> rounds a date, time or timestamp value
+up to a given interval type</li>
</ul>
-<h2 id="references">References</h2>
+<p>Partitioning functions:</p>
+
+<ul>
+ <li><code>HOP(t, emit, retain)</code> returns a collection of group keys for a row
+to be part of a hopping window</li>
+ <li><code>HOP(t, emit, retain, align)</code> returns a collection of group keys for a row
+to be part of a hopping window with a given alignment</li>
+ <li><code>TUMBLE(t, emit)</code> returns a group key for a row
+to be part of a tumbling window</li>
+ <li><code>TUMBLE(t, emit, align)</code> returns a group key for a row
+to be part of a tumbling window with a given alignment</li>
+</ul>
+
+<p><code>TUMBLE(t, e)</code> is equivalent to <code>TUMBLE(t, e, TIME '00:00:00')</code>.</p>
+
+<p><code>TUMBLE(t, e, a)</code> is equivalent to <code>HOP(t, e, e, a)</code>.</p>
+
+<p><code>HOP(t, e, r)</code> is equivalent to <code>HOP(t, e, r, TIME '00:00:00')</code>.</p>
+
+<h1 id="references">References</h1>
<ul>
<li>[<a name="ref1">1</a>]
-<a href="http://ilpubs.stanford.edu:8090/758/">Arasu, Arvind and Babu,
-Shivnath and Widom, Jennifer (2003) The CQL Continuous Query
+<a href="http://ilpubs.stanford.edu:8090/758/">Arvind Arasu, Shivnath Babu,
+and Jennifer Widom (2003) The CQL Continuous Query
Language: Semantic Foundations and Query Execution</a>.</li>
<li>[<a name="ref2">2</a>]
<a href="http://kafka.apache.org/documentation.html">Apache Kafka</a>.</li>
<li>[<a name="ref3">3</a>] <a href="http://samza.apache.org">Apache Samza</a>.</li>
+ <li>[<a name="ref4">4</a>] <a href="https://github.com/milinda/samza-sql">SamzaSQL</a>.</li>
+ <li>[<a name="ref5">5</a>]
+<a href="http://www.whitworth.edu/academic/department/mathcomputerscience/faculty/tuckerpeter/pdf/117896_final.pdf">Peter
+A. Tucker, David Maier, Tim Sheard, and Leonidas Fegaras (2003) Exploiting
+Punctuation Semantics in Continuous Data Streams</a>.</li>
+ <li>[<a name="ref6">6</a>]
+<a href="http://research.google.com/pubs/pub41378.html">Tyler Akidau,
+Alex Balikov, Kaya Bekiroglu, Slava Chernyak, Josh Haberman, Reuven Lax,
+Sam McVeety, Daniel Mills, Paul Nordstrom, and Sam Whittle (2013)
+MillWheel: Fault-Tolerant Stream Processing at Internet Scale</a>.</li>
</ul>
Added: calcite/site/img/pie-chart.png
URL: http://svn.apache.org/viewvc/calcite/site/img/pie-chart.png?rev=1731544&view=auto
==============================================================================
Binary file - no diff available.
Propchange: calcite/site/img/pie-chart.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: calcite/site/img/window-types.png
URL: http://svn.apache.org/viewvc/calcite/site/img/window-types.png?rev=1731544&view=auto
==============================================================================
Binary file - no diff available.
Propchange: calcite/site/img/window-types.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream