You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@calcite.apache.org by jh...@apache.org on 2015/06/01 19:56:25 UTC
[11/19] incubator-calcite git commit: [CALCITE-355] Web site
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_docs/stream.md
----------------------------------------------------------------------
diff --git a/site/_docs/stream.md b/site/_docs/stream.md
new file mode 100644
index 0000000..7418208
--- /dev/null
+++ b/site/_docs/stream.md
@@ -0,0 +1,644 @@
+---
+layout: docs
+title: Streaming
+permalink: /docs/stream.html
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+Calcite has extended SQL and relational algebra in order to support
+streaming queries.
+
+Streams are collections to records that flow continuously, and forever.
+Unlike tables, they are not typically stored on disk, but flow over the
+network and are held for short periods of time in memory.
+
+Streams complement tables because they represent what is happening in the
+present and future of the enterprise whereas tables represent the past.
+It is very common for a stream to be archived into a table.
+
+Like tables, you often want to query streams in a high-level language
+based on relational algebra, validated according to a schema, and optimized
+to take advantage of available resources and algorithms.
+
+Calcite's SQL is an extension to standard SQL, not another 'SQL-like' language.
+The distinction is important, for several reasons:
+
+* Streaming SQL is easy to learn for anyone who knows regular SQL.
+* The semantics are clear, because we aim to produce the same results on a
+ stream as if the same data were in a table.
+* You can write queries that combine streams and tables (or the history of
+ a stream, which is basically an in-memory table).
+* Lots of existing tools can generate standard SQL.
+
+If you don't use the `STREAM` keyword, you are back in regular
+standard SQL.
+
+## An example schema
+
+Our streaming SQL examples use the following schema:
+
+* `Orders (rowtime, productId, orderId, units)` - a stream and a table
+* `Products (rowtime, productId, name)` - a table
+* `Shipments (rowtime, orderId)` - a stream
+
+## A simple query
+
+Let's start with the simplest streaming query:
+
+{% highlight sql %}
+SELECT STREAM *
+FROM Orders;
+
+ rowtime | productId | orderId | units
+----------+-----------+---------+-------
+ 10:17:00 | 30 | 5 | 4
+ 10:17:05 | 10 | 6 | 1
+ 10:18:05 | 20 | 7 | 2
+ 10:18:07 | 30 | 8 | 20
+ 11:02:00 | 10 | 9 | 6
+ 11:04:00 | 10 | 10 | 1
+ 11:09:30 | 40 | 11 | 12
+ 11:24:11 | 10 | 12 | 4
+{% endhighlight %}
+
+This query reads all columns and rows from the `Orders` stream.
+Like any streaming query, it never terminates. It outputs a record whenever
+a record arrives in `Orders`.
+
+Type `Control-C` to terminate the query.
+
+The `STREAM` keyword is the main extension in streaming SQL. It tells the
+system that you are interested in incoming orders, not existing ones. The query
+
+{% highlight sql %}
+SELECT *
+FROM Orders;
+
+ rowtime | productId | orderId | units
+----------+-----------+---------+-------
+ 08:30:00 | 10 | 1 | 3
+ 08:45:10 | 20 | 2 | 1
+ 09:12:21 | 10 | 3 | 10
+ 09:27:44 | 30 | 4 | 2
+
+4 records returned.
+{% endhighlight %}
+
+is also valid, but will print out all existing orders and then terminate. We
+call it a *relational* query, as opposed to *streaming*. It has traditional
+SQL semantics.
+
+`Orders` is special, in that it has both a stream and a table. If you try to run
+a streaming query on a table, or a relational query on a stream, Calcite gives
+an error:
+
+{% highlight sql %}
+> SELECT * FROM Shipments;
+ERROR: Cannot convert stream 'SHIPMENTS' to a table
+
+> SELECT STREAM * FROM Products;
+ERROR: Cannot convert table 'PRODUCTS' to a stream
+{% endhighlight %}
+
+# Filtering rows
+
+Just as in regular SQL, you use a `WHERE` clause to filter rows:
+
+{% highlight sql %}
+SELECT STREAM *
+FROM Orders
+WHERE units > 3;
+
+ rowtime | productId | orderId | units
+----------+-----------+---------+-------
+ 10:17:00 | 30 | 5 | 4
+ 10:18:07 | 30 | 8 | 20
+ 11:02:00 | 10 | 9 | 6
+ 11:09:30 | 40 | 11 | 12
+ 11:24:11 | 10 | 12 | 4
+{% endhighlight %}
+
+# Projecting expressions
+
+Use expressions in the `SELECT` clause to choose which columns to return or
+compute expressions:
+
+{% highlight sql %}
+SELECT STREAM rowtime,
+ 'An order for ' || units || ' '
+ || CASE units WHEN 1 THEN 'unit' ELSE 'units' END
+ || ' of product #' || productId AS description
+FROM Orders;
+
+ rowtime | description
+----------+---------------------------------------
+ 10:17:00 | An order for 4 units of product #30
+ 10:17:05 | An order for 1 unit of product #10
+ 10:18:05 | An order for 2 units of product #20
+ 10:18:07 | An order for 20 units of product #30
+ 11:02:00 | An order by 6 units of product #10
+ 11:04:00 | An order by 1 unit of product #10
+ 11:09:30 | An order for 12 units of product #40
+ 11:24:11 | An order by 4 units of product #10
+{% endhighlight %}
+
+We recommend that you always include the `rowtime` column in the `SELECT`
+clause. Having a sorted timestamp in each stream and streaming query makes it
+possible to do advanced calculations later, such as `GROUP BY` and `JOIN`.
+
+# Tumbling windows
+
+There are several ways to compute aggregate functions on streams. The
+differences are:
+
+* How many rows come out for each row in?
+* Does each incoming value appear in one total, or more?
+* What defines the "window", the set of rows that contribute to a given output row?
+* Is the result a stream or a relation?
+
+First we'll look a *tumbling window*, which is defined by a streaming
+`GROUP BY`. Here is an example:
+
+{% highlight sql %}
+SELECT STREAM FLOOR(rowtime TO HOUR) AS rowtime,
+ productId,
+ COUNT(*) AS c,
+ SUM(units) AS units
+FROM Orders
+GROUP BY FLOOR(rowtime TO HOUR), productId;
+
+ rowtime | productId | c | units
+----------+-----------+---------+-------
+ 10:00:00 | 30 | 2 | 24
+ 10:00:00 | 10 | 1 | 1
+ 10:00:00 | 20 | 1 | 7
+ 11:00:00 | 10 | 3 | 11
+ 11:00:00 | 40 | 1 | 12
+{% endhighlight %}
+
+The result is a stream. At 11 o'clock, Calcite emits a sub-total for every
+`productId` that had an order since 10 o'clock. At 12 o'clock, it will emit
+the orders that occurred between 11:00 and 12:00. Each input row contributes to
+only one output row.
+
+How did Calcite know that the 10:00:00 sub-totals were complete at 11:00:00,
+so that it could emit them? It knows that `rowtime` is increasing, and it knows
+that `FLOOR(rowtime TO HOUR)` is also increasing. So, once it has seen a row
+at or after 11:00:00, it will never see a row that will contribute to a 10:00:00
+total.
+
+A column or expression that is increasing or decreasing is said to be
+*monotonic*. Without a monotonic expression in the `GROUP BY` clause, Calcite is
+not able to make progress, and it will not allow the query:
+
+{% highlight sql %}
+> SELECT STREAM productId,
+> COUNT(*) AS c,
+> SUM(units) AS units
+> FROM Orders
+> GROUP BY productId;
+ERROR: Streaming aggregation requires at least one monotonic expression in GROUP BY clause
+{% endhighlight %}
+
+Monotonic columns need to be declared in the schema. The monotonicity is
+enforced when records enter the stream and assumed by queries that read from
+that stream. We recommend that you give each stream a timestamp column called
+`rowtime`, but you can declare others, `orderId`, for example.
+
+# Filtering after aggregation
+
+As in standard SQL, you can apply a `HAVING` clause to filter rows emitted by
+a streaming `GROUP BY`:
+
+{% highlight sql %}
+SELECT STREAM FLOOR(rowtime TO HOUR) AS rowtime,
+ productId
+FROM Orders
+GROUP BY FLOOR(rowtime TO HOUR), productId
+HAVING COUNT(*) > 2 OR SUM(units) > 10;
+
+ rowtime | productId
+----------+-----------
+ 10:00:00 | 30
+ 11:00:00 | 10
+ 11:00:00 | 40
+{% endhighlight %}
+
+# Sub-queries, views and SQL's closure property
+
+The previous `HAVING` query can be expressed using a `WHERE` clause on a
+sub-query:
+
+{% highlight sql %}
+SELECT STREAM rowtime, productId
+FROM (
+ SELECT FLOOR(rowtime TO HOUR) AS rowtime,
+ productId,
+ COUNT(*) AS c,
+ SUM(units) AS su
+ FROM Orders
+ GROUP BY FLOOR(rowtime TO HOUR), productId)
+WHERE c > 2 OR su > 10;
+
+ rowtime | productId
+----------+-----------
+ 10:00:00 | 30
+ 11:00:00 | 10
+ 11:00:00 | 40
+{% endhighlight %}
+
+`HAVING` was introduced in the early days of SQL, when a way was needed to
+perform a filter *after* aggregation. (Recall that `WHERE` filters rows before
+they enter the `GROUP BY` clause.)
+
+Since then, SQL has become a mathematically closed language, which means that
+any operation you can perform on a table can also perform on a query.
+
+The *closure property* of SQL is extremely powerful. Not only does it render
+`HAVING` obsolete (or, at least, reduce it to syntactic sugar), it makes views
+possible:
+
+{% highlight sql %}
+CREATE VIEW HourlyOrderTotals (rowtime, productId, c, su) AS
+ SELECT FLOOR(rowtime TO HOUR),
+ productId,
+ COUNT(*),
+ SUM(units)
+ FROM Orders
+ GROUP BY FLOOR(rowtime TO HOUR), productId;
+
+SELECT STREAM rowtime, productId
+FROM HourlyOrderTotals
+WHERE c > 2 OR su > 10;
+
+ rowtime | productId
+----------+-----------
+ 10:00:00 | 30
+ 11:00:00 | 10
+ 11:00:00 | 40
+{% endhighlight %}
+
+Sub-queries in the `FROM` clause are sometimes referred to as "inline views",
+but really, they are more fundamental than views. Views are just a convenient
+way to carve your SQL into manageable chunks by giving the pieces names and
+storing them in the metadata repository.
+
+Many people find that nested queries and views are even more useful on streams
+than they are on relations. Streaming queries are pipelines of
+operators all running continuously, and often those pipelines get quite long.
+Nested queries and views help to express and manage those pipelines.
+
+And, by the way, a `WITH` clause can accomplish the same as a sub-query or
+a view:
+
+{% highlight sql %}
+WITH HourlyOrderTotals (rowtime, productId, c, su) AS (
+ SELECT FLOOR(rowtime TO HOUR),
+ productId,
+ COUNT(*),
+ SUM(units)
+ FROM Orders
+ GROUP BY FLOOR(rowtime TO HOUR), productId)
+SELECT STREAM rowtime, productId
+FROM HourlyOrderTotals
+WHERE c > 2 OR su > 10;
+
+ rowtime | productId
+----------+-----------
+ 10:00:00 | 30
+ 11:00:00 | 10
+ 11:00:00 | 40
+{% endhighlight %}
+
+## Converting between streams and relations
+
+Look back at the definition of the `HourlyOrderTotals` view.
+Is the view a stream or a relation?
+
+It does not contain the `STREAM` keyword, so it is a relation.
+However, it is a relation that can be converted into a stream.
+
+You can use it in both relational and streaming queries:
+
+{% highlight sql %}
+# A relation; will query the historic Orders table.
+# Returns the largest number of product #10 ever sold in one hour.
+SELECT max(su)
+FROM HourlyOrderTotals
+WHERE productId = 10;
+
+# A stream; will query the Orders stream.
+# Returns every hour in which at least one product #10 was sold.
+SELECT STREAM rowtime
+FROM HourlyOrderTotals
+WHERE productId = 10;
+{% endhighlight %}
+
+This approach is not limited to views and sub-queries.
+Following the approach set out in CQL [<a href="#ref1">1</a>], every query
+in streaming SQL is defined as a relational query and converted to a stream
+using the `STREAM` keyword in the top-most `SELECT`.
+
+If the `STREAM` keyword is present in sub-queries or view definitions, it has no
+effect.
+
+At query preparation time, Calcite figures out whether the relations referenced
+in the query can be converted to streams or historical relations.
+
+Sometimes a stream makes available some of its history (say the last 24 hours of
+data in an Apache Kafka [<a href="#ref2">2</a>] topic)
+but not all. At run time, Calcite figures out whether there is sufficient
+history to run the query, and if not, gives an error.
+
+## Hopping windows
+
+Previously we saw how to define a tumbling window using a `GROUP BY` clause.
+Each record contributed to a single sub-total record, the one containing its
+hour and product id.
+
+But suppose we want to emit, every hour, the number of each product ordered over
+the past three hours. To do this, we use `SELECT ... OVER` and a sliding window
+to combine multiple tumbling windows.
+
+{% highlight sql %}
+SELECT STREAM rowtime,
+ productId,
+ SUM(su) OVER w AS su,
+ SUM(c) OVER w AS c
+FROM HourlyTotals
+WINDOW w AS (
+ ORDER BY rowtime
+ PARTITION BY productId
+ RANGE INTERVAL '2' HOUR PRECEDING)
+{% endhighlight %}
+
+This query uses the `HourlyOrderTotals` view defined previously.
+The 2 hour interval combines the totals timestamped 09:00:00, 10:00:00 and
+11:00:00 for a particular product into a single total timestamped 11:00:00 and
+summarizing orders for that product between 09:00:00 and 12:00:00.
+
+## Limitations of tumbling and hopping windows
+
+In the present syntax, we acknowledge that it is not easy to create certain
+kinds of windows.
+
+First, let's consider tumbling windows over complex periods.
+
+The `FLOOR` and `CEIL` functions make is easy to create a tumbling window that
+emits on a whole time unit (say every hour, or every minute) but less easy to
+emit, say, every 15 minutes. One could imagine an extension to the `FLOOR`
+function that emits unique values on just about any periodic basis (say in 11
+minute intervals starting from midnight of the current day).
+
+Next, let's consider hopping windows whose retention period is not a multiple
+of its emission period. Say we want to output, at the top of each hour, the
+orders for the previous 7,007 seconds. If we were to simulate this hopping
+window using a sliding window over a tumbling window, as before, we would have
+to sum lots of 1-second windows (because 3,600 and 7,007 are co-prime).
+This is a lot of effort for both the system and the person writing the query.
+
+Calcite could perhaps solve this generalizing `GROUP BY` syntax, but we would
+be destroying the principle that an input row into a `GROUP BY` appears in
+precisely one output row.
+
+Calcite's SQL extensions for streaming queries are evolving. As we learn more
+about how people wish to query streams, we plan to make the language more
+expressive while remaining compatible with standard SQL and consistent with
+its principles, look and feel.
+
+## Sorting
+
+The story for `ORDER BY` is similar to `GROUP BY`.
+The syntax looks like regular SQL, but Calcite must be sure that it can deliver
+timely results. It therefore requires a monotonic expression on the leading edge
+of your `ORDER BY` key.
+
+{% highlight sql %}
+SELECT STREAM FLOOR(rowtime TO hour) AS rowtime, productId, orderId, units
+FROM Orders
+ORDER BY FLOOR(rowtime TO hour) ASC, units DESC;
+
+ rowtime | productId | orderId | units
+----------+-----------+---------+-------
+ 10:00:00 | 30 | 8 | 20
+ 10:00:00 | 30 | 5 | 4
+ 10:00:00 | 20 | 7 | 2
+ 10:00:00 | 10 | 6 | 1
+ 11:00:00 | 40 | 11 | 12
+ 11:00:00 | 10 | 9 | 6
+ 11:00:00 | 10 | 12 | 4
+ 11:00:00 | 10 | 10 | 1
+{% endhighlight %}
+
+Most queries will return results in the order that they were inserted,
+because the engine is using streaming algorithms, but you should not rely on it.
+For example, consider this:
+
+{% highlight sql %}
+SELECT STREAM *
+FROM Orders
+WHERE productId = 10
+UNION ALL
+SELECT STREAM *
+FROM Orders
+WHERE productId = 30;
+
+ rowtime | productId | orderId | units
+----------+-----------+---------+-------
+ 10:17:05 | 10 | 6 | 1
+ 10:17:00 | 30 | 5 | 4
+ 10:18:07 | 30 | 8 | 20
+ 11:02:00 | 10 | 9 | 6
+ 11:04:00 | 10 | 10 | 1
+ 11:24:11 | 10 | 12 | 4
+{% endhighlight %}
+
+The rows with `productId` = 30 are apparently out of order, probably because
+the `Orders` stream was partitioned on `productId` and the partitioned streams
+sent their data at different times.
+
+If you require a particular ordering, add an explicit `ORDER BY`:
+
+{% highlight sql %}
+SELECT STREAM *
+FROM Orders
+WHERE productId = 10
+UNION ALL
+SELECT STREAM *
+FROM Orders
+WHERE productId = 30
+ORDER BY rowtime;
+
+ rowtime | productId | orderId | units
+----------+-----------+---------+-------
+ 10:17:00 | 30 | 5 | 4
+ 10:17:05 | 10 | 6 | 1
+ 10:18:07 | 30 | 8 | 20
+ 11:02:00 | 10 | 9 | 6
+ 11:04:00 | 10 | 10 | 1
+ 11:24:11 | 10 | 12 | 4
+{% endhighlight %}
+
+Calcite will probably implement the `UNION ALL` by merging using `rowtime`,
+which is only slightly less efficient.
+
+You only need to add an `ORDER BY` to the outermost query. If you need to,
+say, perform `GROUP BY` after a `UNION ALL`, Calcite will add an `ORDER BY`
+implicitly, in order to make the GROUP BY algorithm possible.
+
+## Table constructor
+
+The `VALUES` clause creates an inline table with a given set of rows.
+
+Streaming is disallowed. The set of rows never changes, and therefore a stream
+would never return any rows.
+
+{% highlight sql %}
+> SELECT STREAM * FROM (VALUES (1, 'abc'));
+
+ERROR: Cannot stream VALUES
+{% endhighlight %}
+
+## Sliding windows
+
+Standard SQL features so-called "analytic functions" that can be used in the
+`SELECT` clause. Unlike `GROUP BY`, these do not collapse records. For each
+record that goes in, one record comes out. But the aggregate function is based
+on a window of many rows.
+
+Let's look at an example.
+
+{% highlight sql %}
+SELECT STREAM rowtime,
+ productId,
+ units,
+ SUM(units) OVER (ORDER BY rowtime RANGE INTERVAL '1' HOUR PRECEDING) unitsLastHour
+FROM Orders;
+{% endhighlight %}
+
+The feature packs a lot of power with little effort. You can have multiple
+functions in the `SELECT` clause, based on multiple window specifications.
+
+The following example returns orders whose average order size over the last
+10 minutes is greater than the average order size for the last week.
+
+{% highlight sql %}
+SELECT STREAM *
+FROM (
+ SELECT STREAM rowtime,
+ productId,
+ units,
+ AVG(units) OVER product (RANGE INTERVAL '10' MINUTE PRECEDING) AS m10,
+ AVG(units) OVER product (RANGE INTERVAL '7' DAY PRECEDING) AS d7
+ FROM Orders
+ WINDOW product AS (
+ ORDER BY rowtime
+ PARTITION BY productId))
+WHERE m10 > d7;
+{% endhighlight %}
+
+For conciseness, here we use a syntax where you partially define a window
+using a `WINDOW` clause and then refine the window in each `OVER` clause.
+You could also define all windows in the `WINDOW` clause, or all windows inline,
+if you wish.
+
+But the real power goes beyond syntax. Behind the scenes, this query is
+maintaining two tables, and adding and removing values from sub-totals using
+with FIFO queues. But you can access those tables without introducing a join
+into the query.
+
+Some other features of the windowed aggregation syntax:
+
+* You can define windows based on row count.
+* The window can reference rows that have not yet arrived.
+ (The stream will wait until they have arrived).
+* You can compute order-dependent functions such as `RANK` and median.
+
+## Cascading windows
+
+What if we want a query that returns a result for every record, like a
+sliding window, but resets totals on a fixed time period, like a
+tumbling window? Such a pattern is called a *cascading window*. Here
+is an example:
+
+{% highlight sql %}
+SELECT STREAM rowtime,
+ productId,
+ units,
+ SUM(units) OVER (PARTITION BY FLOOR(rowtime TO HOUR)) AS unitsSinceTopOfHour
+FROM Orders;
+{% endhighlight %}
+
+It looks similar to a sliding window query, but the monotonic
+expression occurs within the `PARTITION BY` clause of the window. As
+the rowtime moves from from 10:59:59 to 11:00:00, `FLOOR(rowtime TO
+HOUR)` changes from 10:00:00 to 11:00:00, and therefore a new
+partition starts. The first row to arrive in the new hour will start a
+new total; the second row will have a total that consists of two rows,
+and so on.
+
+Calcite knows that the old partition will never be used again, so
+removes all sub-totals for that partition from its internal storage.
+
+Analytic functions that using cascading and sliding windows can be
+combined in the same query.
+
+## State of the stream
+
+Not all concepts in this article have been implemented in Calcite.
+And others may be implemented in Calcite but not in a particular adapter
+such as Samza SQL [<a href="#ref3">3</a>].
+
+### Implemented
+* Streaming SELECT, WHERE, GROUP BY, HAVING, UNION ALL, ORDER BY
+* FLOOR and CEILING functions
+* Monotonicity
+* Streaming VALUES is disallowed
+
+### Not implemented
+* Stream-to-stream JOIN
+* Stream-to-table JOIN
+* Stream on view
+* Streaming UNION ALL with ORDER BY (merge)
+* Relational query on stream
+* Streaming windowed aggregation (sliding and cascading windows)
+* Check that STREAM in sub-queries and views is ignored
+* Check that streaming ORDER BY cannot have OFFSET or LIMIT
+* Limited history; at run time, check that there is sufficient history
+ to run the query.
+
+### To do in this document
+* Re-visit whether you can stream VALUES
+* OVER clause to define window on stream
+* Windowed aggregation
+* Punctuation
+* Stream-to-table join
+ * Stream-to-table join where table is changing
+* Stream-to-stream join
+* Relational queries on streams (e.g. "pie chart" query)
+* Diagrams for various window types
+
+## References
+
+* [<a name="ref1">1</a>]
+ <a href="http://ilpubs.stanford.edu:8090/758/">Arasu, Arvind and Babu,
+ Shivnath and Widom, Jennifer (2003) The CQL Continuous Query
+ Language: Semantic Foundations and Query Execution</a>.
+* [<a name="ref2">2</a>]
+ <a href="http://kafka.apache.org/documentation.html">Apache Kafka</a>.
+* [<a name="ref3">3</a>] <a href="http://samza.apache.org">Apache Samza</a>.
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_docs/tutorial.md
----------------------------------------------------------------------
diff --git a/site/_docs/tutorial.md b/site/_docs/tutorial.md
new file mode 100644
index 0000000..51ec187
--- /dev/null
+++ b/site/_docs/tutorial.md
@@ -0,0 +1,760 @@
+---
+layout: docs
+title: Tutorial
+permalink: /docs/tutorial.html
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+{% assign sourceRoot = "http://github.com/apache/incubator-calcite/blob/master" %}
+
+This is a step-by-step tutorial that shows how to build and connect to
+Calcite. It uses a simple adapter that makes a directory of CSV files
+appear to be a schema containing tables. Calcite does the rest, and
+provides a full SQL interface.
+
+Calcite-example-CSV is a fully functional adapter for
+Calcite that reads
+text files in
+<a href="http://en.wikipedia.org/wiki/Comma-separated_values">CSV
+(comma-separated values)</a> format. It is remarkable that a couple of
+hundred lines of Java code are sufficient to provide full SQL query
+capability.
+
+CSV also serves as a template for building adapters to other
+data formats. Even though there are not many lines of code, it covers
+several important concepts:
+
+* user-defined schema using SchemaFactory and Schema interfaces;
+* declaring schemas in a model JSON file;
+* declaring views in a model JSON file;
+* user-defined table using the Table interface;
+* determining the record type of a table;
+* a simple implementation of Table, using the ScannableTable interface, that
+ enumerates all rows directly;
+* a more advanced implementation that implements FilterableTable, and can
+ filter out rows according to simple predicates;
+* advanced implementation of Table, using TranslatableTable, that translates
+ to relational operators using planner rules.
+
+## Download and build
+
+You need Java (1.7 or higher; 1.8 preferred), git and maven (3.2.1 or later).
+
+{% highlight bash %}
+$ git clone https://github.com/apache/incubator-calcite.git
+$ cd incubator-calcite
+$ mvn install -DskipTests -Dcheckstyle.skip=true
+$ cd example/csv
+{% endhighlight %}
+
+## First queries
+
+Now let's connect to Calcite using
+<a href="https://github.com/julianhyde/sqlline">sqlline</a>, a SQL shell
+that is included in this project.
+
+{% highlight bash %}
+$ ./sqlline
+sqlline> !connect jdbc:calcite:model=target/test-classes/model.json admin admin
+{% endhighlight %}
+
+(If you are running Windows, the command is `sqlline.bat`.)
+
+Execute a metadata query:
+
+{% highlight bash %}
+sqlline> !tables
++------------+--------------+-------------+---------------+----------+------+
+| TABLE_CAT | TABLE_SCHEM | TABLE_NAME | TABLE_TYPE | REMARKS | TYPE |
++------------+--------------+-------------+---------------+----------+------+
+| null | SALES | DEPTS | TABLE | null | null |
+| null | SALES | EMPS | TABLE | null | null |
+| null | SALES | HOBBIES | TABLE | null | null |
+| null | metadata | COLUMNS | SYSTEM_TABLE | null | null |
+| null | metadata | TABLES | SYSTEM_TABLE | null | null |
++------------+--------------+-------------+---------------+----------+------+
+{% endhighlight %}
+
+(JDBC experts, note: sqlline's <code>!tables</code> command is just executing
+<a href="http://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html#getTables(java.lang.String, java.lang.String, java.lang.String, java.lang.String[])"><code>DatabaseMetaData.getTables()</code></a>
+behind the scenes.
+It has other commands to query JDBC metadata, such as <code>!columns</code> and <code>!describe</code>.)
+
+As you can see there are 5 tables in the system: tables
+<code>EMPS</code>, <code>DEPTS</code> and <code>HOBBIES</code> in the current
+<code>SALES</code> schema, and <code>COLUMNS</code> and
+<code>TABLES</code> in the system <code>metadata</code> schema. The
+system tables are always present in Calcite, but the other tables are
+provided by the specific implementation of the schema; in this case,
+the <code>EMPS</code> and <code>DEPTS</code> tables are based on the
+<code>EMPS.csv</code> and <code>DEPTS.csv</code> files in the
+<code>target/test-classes</code> directory.
+
+Let's execute some queries on those tables, to show that Calcite is providing
+a full implementation of SQL. First, a table scan:
+
+{% highlight bash %}
+sqlline> SELECT * FROM emps;
++--------+--------+---------+---------+----------------+--------+-------+---+
+| EMPNO | NAME | DEPTNO | GENDER | CITY | EMPID | AGE | S |
++--------+--------+---------+---------+----------------+--------+-------+---+
+| 100 | Fred | 10 | | | 30 | 25 | t |
+| 110 | Eric | 20 | M | San Francisco | 3 | 80 | n |
+| 110 | John | 40 | M | Vancouver | 2 | null | f |
+| 120 | Wilma | 20 | F | | 1 | 5 | n |
+| 130 | Alice | 40 | F | Vancouver | 2 | null | f |
++--------+--------+---------+---------+----------------+--------+-------+---+
+{% endhighlight %}
+
+Now JOIN and GROUP BY:
+
+{% highlight bash %}
+sqlline> SELECT d.name, COUNT(*)
+. . . .> FROM emps AS e JOIN depts AS d ON e.deptno = d.deptno
+. . . .> GROUP BY d.name;
++------------+---------+
+| NAME | EXPR$1 |
++------------+---------+
+| Sales | 1 |
+| Marketing | 2 |
++------------+---------+
+{% endhighlight %}
+
+Last, the VALUES operator generates a single row, and is a convenient
+way to test expressions and SQL built-in functions:
+
+{% highlight bash %}
+sqlline> VALUES CHAR_LENGTH('Hello, ' || 'world!');
++---------+
+| EXPR$0 |
++---------+
+| 13 |
++---------+
+{% endhighlight %}
+
+Calcite has many other SQL features. We don't have time to cover them
+here. Write some more queries to experiment.
+
+## Schema discovery
+
+Now, how did Calcite find these tables? Remember, core Calcite does not
+know anything about CSV files. (As a "database without a storage
+layer", Calcite doesn't know about any file formats.) Calcite knows about
+those tables because we told it to run code in the calcite-example-csv
+project.
+
+There are a couple of steps in that chain. First, we define a schema
+based on a schema factory class in a model file. Then the schema
+factory creates a schema, and the schema creates several tables, each
+of which knows how to get data by scanning a CSV file. Last, after
+Calcite has parsed the query and planned it to use those tables, Calcite
+invokes the tables to read the data as the query is being
+executed. Now let's look at those steps in more detail.
+
+On the JDBC connect string we gave the path of a model in JSON
+format. Here is the model:
+
+{% highlight json %}
+{
+ version: '1.0',
+ defaultSchema: 'SALES',
+ schemas: [
+ {
+ name: 'SALES',
+ type: 'custom',
+ factory: 'org.apache.calcite.adapter.csv.CsvSchemaFactory',
+ operand: {
+ directory: 'target/test-classes/sales'
+ }
+ }
+ ]
+}
+{% endhighlight %}
+
+The model defines a single schema called 'SALES'. The schema is
+powered by a plugin class,
+<a href="{{ sourceRoot }}/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvSchemaFactory.java">org.apache.calcite.adapter.csv.CsvSchemaFactory</a>,
+which is part of the
+calcite-example-csv project and implements the Calcite interface
+<a href="/apidocs/org/apache/calcite/schema/SchemaFactory.html">SchemaFactory</a>.
+Its <code>create</code> method instantiates a
+schema, passing in the <code>directory</code> argument from the model file:
+
+{% highlight java %}
+public Schema create(SchemaPlus parentSchema, String name,
+ Map<String, Object> operand) {
+ String directory = (String) operand.get("directory");
+ String flavorName = (String) operand.get("flavor");
+ CsvTable.Flavor flavor;
+ if (flavorName == null) {
+ flavor = CsvTable.Flavor.SCANNABLE;
+ } else {
+ flavor = CsvTable.Flavor.valueOf(flavorName.toUpperCase());
+ }
+ return new CsvSchema(
+ new File(directory),
+ flavor);
+}
+{% endhighlight %}
+
+Driven by the model, the schema factory instantiates a single schema
+called 'SALES'. The schema is an instance of
+<a href="{{ sourceRoot }}/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvSchema.java">org.apache.calcite.adapter.csv.CsvSchema</a>
+and implements the Calcite interface
+<a href="/calcite/apidocs/org/apache/calcite/schema/Schema.html">Schema</a>.
+
+A schema's job is to produce a list of tables. (It can also list sub-schemas and
+table-functions, but these are advanced features and calcite-example-csv does
+not support them.) The tables implement Calcite's
+<a href="/apidocs/org/apache/calcite/schema/Table.html">Table</a>
+interface. <code>CsvSchema</code> produces tables that are instances of
+<a href="{{ sourceRoot }}/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvTable.java">CsvTable</a>
+and its sub-classes.
+
+Here is the relevant code from <code>CsvSchema</code>, overriding the
+<code><a href="/apidocs/org/apache/calcite/schema/impl/AbstractSchema.html#getTableMap()">getTableMap()</a></code>
+method in the <code>AbstractSchema</code> base class.
+
+{% highlight java %}
+protected Map<String, Table> getTableMap() {
+ // Look for files in the directory ending in ".csv", ".csv.gz", ".json",
+ // ".json.gz".
+ File[] files = directoryFile.listFiles(
+ new FilenameFilter() {
+ public boolean accept(File dir, String name) {
+ final String nameSansGz = trim(name, ".gz");
+ return nameSansGz.endsWith(".csv")
+ || nameSansGz.endsWith(".json");
+ }
+ });
+ if (files == null) {
+ System.out.println("directory " + directoryFile + " not found");
+ files = new File[0];
+ }
+ // Build a map from table name to table; each file becomes a table.
+ final ImmutableMap.Builder<String, Table> builder = ImmutableMap.builder();
+ for (File file : files) {
+ String tableName = trim(file.getName(), ".gz");
+ final String tableNameSansJson = trimOrNull(tableName, ".json");
+ if (tableNameSansJson != null) {
+ JsonTable table = new JsonTable(file);
+ builder.put(tableNameSansJson, table);
+ continue;
+ }
+ tableName = trim(tableName, ".csv");
+ final Table table = createTable(file);
+ builder.put(tableName, table);
+ }
+ return builder.build();
+}
+
+/** Creates different sub-type of table based on the "flavor" attribute. */
+private Table createTable(File file) {
+ switch (flavor) {
+ case TRANSLATABLE:
+ return new CsvTranslatableTable(file, null);
+ case SCANNABLE:
+ return new CsvScannableTable(file, null);
+ case FILTERABLE:
+ return new CsvFilterableTable(file, null);
+ default:
+ throw new AssertionError("Unknown flavor " + flavor);
+ }
+}
+{% endhighlight %}
+
+The schema scans the directory and finds all files whose name ends
+with ".csv" and creates tables for them. In this case, the directory
+is <code>target/test-classes/sales</code> and contains files
+<code>EMPS.csv</code> and <code>DEPTS.csv</code>, which these become
+the tables <code>EMPS</code> and <code>DEPTS</code>.
+
+## Tables and views in schemas
+
+Note how we did not need to define any tables in the model; the schema
+generated the tables automatically.
+
+You can define extra tables,
+beyond those that are created automatically,
+using the <code>tables</code> property of a schema.
+
+Let's see how to create
+an important and useful type of table, namely a view.
+
+A view looks like a table when you are writing a query, but it doesn't store data.
+It derives its result by executing a query.
+The view is expanded while the query is being planned, so the query planner
+can often perform optimizations like removing expressions from the SELECT
+clause that are not used in the final result.
+
+Here is a schema that defines a view:
+
+{% highlight json %}
+{
+ version: '1.0',
+ defaultSchema: 'SALES',
+ schemas: [
+ {
+ name: 'SALES',
+ type: 'custom',
+ factory: 'org.apache.calcite.adapter.csv.CsvSchemaFactory',
+ operand: {
+ directory: 'target/test-classes/sales'
+ },
+ tables: [
+ {
+ name: 'FEMALE_EMPS',
+ type: 'view',
+ sql: 'SELECT * FROM emps WHERE gender = \'F\''
+ }
+ ]
+ }
+ ]
+}
+{% endhighlight %}
+
+The line <code>type: 'view'</code> tags <code>FEMALE_EMPS</code> as a view,
+as opposed to a regular table or a custom table.
+Note that single-quotes within the view definition are escaped using a
+back-slash, in the normal way for JSON.
+
+JSON doesn't make it easy to author long strings, so Calcite supports an
+alternative syntax. If your view has a long SQL statement, you can instead
+supply a list of lines rather than a single string:
+
+{% highlight json %}
+{
+ name: 'FEMALE_EMPS',
+ type: 'view',
+ sql: [
+ 'SELECT * FROM emps',
+ 'WHERE gender = \'F\''
+ ]
+}
+{% endhighlight %}
+
+Now we have defined a view, we can use it in queries just as if it were a table:
+
+{% highlight sql %}
+sqlline> SELECT e.name, d.name FROM female_emps AS e JOIN depts AS d on e.deptno = d.deptno;
++--------+------------+
+| NAME | NAME |
++--------+------------+
+| Wilma | Marketing |
++--------+------------+
+{% endhighlight %}
+
+## Custom tables
+
+Custom tables are tables whose implementation is driven by user-defined code.
+They don't need to live in a custom schema.
+
+There is an example in <code>model-with-custom-table.json</code>:
+
+{% highlight json %}
+{
+ version: '1.0',
+ defaultSchema: 'CUSTOM_TABLE',
+ schemas: [
+ {
+ name: 'CUSTOM_TABLE',
+ tables: [
+ {
+ name: 'EMPS',
+ type: 'custom',
+ factory: 'org.apache.calcite.adapter.csv.CsvTableFactory',
+ operand: {
+ file: 'target/test-classes/sales/EMPS.csv.gz',
+ flavor: "scannable"
+ }
+ }
+ ]
+ }
+ ]
+}
+{% endhighlight %}
+
+We can query the table in the usual way:
+
+{% highlight sql %}
+sqlline> !connect jdbc:calcite:model=target/test-classes/model-with-custom-table.json admin admin
+sqlline> SELECT empno, name FROM custom_table.emps;
++--------+--------+
+| EMPNO | NAME |
++--------+--------+
+| 100 | Fred |
+| 110 | Eric |
+| 110 | John |
+| 120 | Wilma |
+| 130 | Alice |
++--------+--------+
+{% endhighlight %}
+
+The schema is a regular one, and contains a custom table powered by
+<a href="{{ sourceRoot }}/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvTableFactory.java">org.apache.calcite.adapter.csv.CsvTableFactory</a>,
+which implements the Calcite interface
+<a href="/apidocs/org/apache/calcite/schema/TableFactory.html">TableFactory</a>.
+Its <code>create</code> method instantiates a <code>CsvScannableTable</code>,
+passing in the <code>file</code> argument from the model file:
+
+{% highlight java %}
+public CsvTable create(SchemaPlus schema, String name,
+ Map<String, Object> map, RelDataType rowType) {
+ String fileName = (String) map.get("file");
+ final File file = new File(fileName);
+ final RelProtoDataType protoRowType =
+ rowType != null ? RelDataTypeImpl.proto(rowType) : null;
+ return new CsvScannableTable(file, protoRowType);
+}
+{% endhighlight %}
+
+Implementing a custom table is often a simpler alternative to implementing
+a custom schema. Both approaches might end up creating a similar implementation
+of the <code>Table</code> interface, but for the custom table you don't
+need to implement metadata discovery. (<code>CsvTableFactory</code>
+creates a <code>CsvScannableTable</code>, just as <code>CsvSchema</code> does,
+but the table implementation does not scan the filesystem for .csv files.)
+
+Custom tables require more work for the author of the model (the author
+needs to specify each table and its file explicitly) but also give the author
+more control (say, providing different parameters for each table).
+
+## Comments in models
+
+Models can include comments using `/* ... */` and `//` syntax:
+
+{% highlight json %}
+{
+ version: '1.0',
+ /* Multi-line
+ comment. */
+ defaultSchema: 'CUSTOM_TABLE',
+ // Single-line comment.
+ schemas: [
+ ..
+ ]
+}
+{% endhighlight %}
+
+(Comments are not standard JSON, but are a harmless extension.)
+
+## Optimizing queries using planner rules
+
+The table implementations we have seen so far are fine as long as the tables
+don't contain a great deal of data. But if your customer table has, say, a
+hundred columns and a million rows, you would rather that the system did not
+retrieve all of the data for every query. You would like Calcite to negotiate
+with the adapter and find a more efficient way of accessing the data.
+
+This negotiation is a simple form of query optimization. Calcite supports query
+optimization by adding <i>planner rules</i>. Planner rules operate by
+looking for patterns in the query parse tree (for instance a project on top
+of a certain kind of table), and
+
+Planner rules are also extensible, like schemas and tables. So, if you have a
+data store that you want to access via SQL, you first define a custom table or
+schema, and then you define some rules to make the access efficient.
+
+To see this in action, let's use a planner rule to access
+a subset of columns from a CSV file. Let's run the same query against two very
+similar schemas:
+
+{% highlight sql %}
+sqlline> !connect jdbc:calcite:model=target/test-classes/model.json admin admin
+sqlline> explain plan for select name from emps;
++-----------------------------------------------------+
+| PLAN |
++-----------------------------------------------------+
+| EnumerableCalcRel(expr#0..9=[{inputs}], NAME=[$t1]) |
+| EnumerableTableAccessRel(table=[[SALES, EMPS]]) |
++-----------------------------------------------------+
+sqlline> !connect jdbc:calcite:model=target/test-classes/smart.json admin admin
+sqlline> explain plan for select name from emps;
++-----------------------------------------------------+
+| PLAN |
++-----------------------------------------------------+
+| EnumerableCalcRel(expr#0..9=[{inputs}], NAME=[$t1]) |
+| CsvTableScan(table=[[SALES, EMPS]]) |
++-----------------------------------------------------+
+{% endhighlight %}
+
+What causes the difference in plan? Let's follow the trail of evidence. In the
+<code>smart.json</code> model file, there is just one extra line:
+
+{% highlight json %}
+flavor: "translatable"
+{% endhighlight %}
+
+This causes a <code>CsvSchema</code> to be created with
+<code>flavor = TRANSLATABLE</code>,
+and its <code>createTable</code> method creates instances of
+<a href="{{ sourceRoot }}/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvTranslatableTable.java">CsvTranslatableTable</a>
+rather than a <code>CsvScannableTable</code>.
+
+<code>CsvTranslatableTable</code> implements the
+<code><a href="/apidocs/org/apache/calcite/schema/TranslatableTable.html#toRel()">TranslatableTable.toRel()</a></code>
+method to create
+<a href="{{ sourceRoot }}/example/csv/src/main/java/org/apache/calcite/adapter/csv/CsvTableScan.java">CsvTableScan</a>.
+Table scans are the leaves of a query operator tree.
+The usual implementation is
+<code><a href="/apidocs/org/apache/calcite/adapter/enumerable/EnumerableTableScan.html">EnumerableTableScan</a></code>,
+but we have created a distinctive sub-type that will cause rules to fire.
+
+Here is the rule in its entirety:
+
+{% highlight java %}
+public class CsvProjectTableScanRule extends RelOptRule {
+ public static final CsvProjectTableScanRule INSTANCE =
+ new CsvProjectTableScanRule();
+
+ private CsvProjectTableScanRule() {
+ super(
+ operand(Project.class,
+ operand(CsvTableScan.class, none())),
+ "CsvProjectTableScanRule");
+ }
+
+ @Override
+ public void onMatch(RelOptRuleCall call) {
+ final Project project = call.rel(0);
+ final CsvTableScan scan = call.rel(1);
+ int[] fields = getProjectFields(project.getProjects());
+ if (fields == null) {
+ // Project contains expressions more complex than just field references.
+ return;
+ }
+ call.transformTo(
+ new CsvTableScan(
+ scan.getCluster(),
+ scan.getTable(),
+ scan.csvTable,
+ fields));
+ }
+
+ private int[] getProjectFields(List<RexNode> exps) {
+ final int[] fields = new int[exps.size()];
+ for (int i = 0; i < exps.size(); i++) {
+ final RexNode exp = exps.get(i);
+ if (exp instanceof RexInputRef) {
+ fields[i] = ((RexInputRef) exp).getIndex();
+ } else {
+ return null; // not a simple projection
+ }
+ }
+ return fields;
+ }
+}
+{% endhighlight %}
+
+The constructor declares the pattern of relational expressions that will cause
+the rule to fire.
+
+The <code>onMatch</code> method generates a new relational expression and calls
+<code><a href="/apidocs/org/apache/calcite/plan/RelOptRuleCall.html#transformTo(org.apache.calcite.rel.RelNode)">RelOptRuleCall.transformTo()</a></code>
+to indicate that the rule has fired successfully.
+
+## The query optimization process
+
+There's a lot to say about how clever Calcite's query planner is, but we won't
+say it here. The cleverness is designed to take the burden off you, the writer
+of planner rules.
+
+First, Calcite doesn't fire rules in a prescribed order. The query optimization
+process follows many branches of a branching tree, just like a chess playing
+program examines many possible sequences of moves. If rules A and B both match a
+given section of the query operator tree, then Calcite can fire both.
+
+Second, Calcite uses cost in choosing between plans, but the cost model doesn't
+prevent rules from firing which may seem to be more expensive in the short term.
+
+Many optimizers have a linear optimization scheme. Faced with a choice between
+rule A and rule B, as above, such an optimizer needs to choose immediately. It
+might have a policy such as "apply rule A to the whole tree, then apply rule B
+to the whole tree", or apply a cost-based policy, applying the rule that
+produces the cheaper result.
+
+Calcite doesn't require such compromises.
+This makes it simple to combine various sets of rules.
+If, say you want to combine rules to recognize materialized views with rules to
+read from CSV and JDBC source systems, you just give Calcite the set of all
+rules and tell it to go at it.
+
+Calcite does use a cost model. The cost model decides which plan to ultimately
+use, and sometimes to prune the search tree to prevent the search space from
+exploding, but it never forces you to choose between rule A and rule B. This is
+important, because it avoids falling into local minima in the search space that
+are not actually optimal.
+
+Also (you guessed it) the cost model is pluggable, as are the table and query
+operator statistics it is based upon. But that can be a subject for later.
+
+## JDBC adapter
+
+The JDBC adapter maps a schema in a JDBC data source as a Calcite schema.
+
+For example, this schema reads from a MySQL "foodmart" database:
+
+{% highlight json %}
+{
+ version: '1.0',
+ defaultSchema: 'FOODMART',
+ schemas: [
+ {
+ name: 'FOODMART',
+ type: 'custom',
+ factory: 'org.apache.calcite.adapter.jdbc.JdbcSchema$Factory',
+ operand: {
+ jdbcDriver: 'com.mysql.jdbc.Driver',
+ jdbcUrl: 'jdbc:mysql://localhost/foodmart',
+ jdbcUser: 'foodmart',
+ jdbcPassword: 'foodmart'
+ }
+ }
+ ]
+}
+{% endhighlight %}
+
+(The FoodMart database will be familiar to those of you who have used
+the Mondrian OLAP engine, because it is Mondrian's main test data
+set. To load the data set, follow <a
+href="http://mondrian.pentaho.com/documentation/installation.php#2_Set_up_test_data">Mondrian's
+installation instructions</a>.)
+
+<b>Current limitations</b>: The JDBC adapter currently only pushes
+down table scan operations; all other processing (filtering, joins,
+aggregations and so forth) occurs within Calcite. Our goal is to push
+down as much processing as possible to the source system, translating
+syntax, data types and built-in functions as we go. If a Calcite query
+is based on tables from a single JDBC database, in principle the whole
+query should go to that database. If tables are from multiple JDBC
+sources, or a mixture of JDBC and non-JDBC, Calcite will use the most
+efficient distributed query approach that it can.
+
+## The cloning JDBC adapter
+
+The cloning JDBC adapter creates a hybrid database. The data is
+sourced from a JDBC database but is read into in-memory tables the
+first time each table is accessed. Calcite evaluates queries based on
+those in-memory tables, effectively a cache of the database.
+
+For example, the following model reads tables from a MySQL
+"foodmart" database:
+
+{% highlight json %}
+{
+ version: '1.0',
+ defaultSchema: 'FOODMART_CLONE',
+ schemas: [
+ {
+ name: 'FOODMART_CLONE',
+ type: 'custom',
+ factory: 'org.apache.calcite.adapter.clone.CloneSchema$Factory',
+ operand: {
+ jdbcDriver: 'com.mysql.jdbc.Driver',
+ jdbcUrl: 'jdbc:mysql://localhost/foodmart',
+ jdbcUser: 'foodmart',
+ jdbcPassword: 'foodmart'
+ }
+ }
+ ]
+}
+{% endhighlight %}
+
+Another technique is to build a clone schema on top of an existing
+schema. You use the <code>source</code> property to reference a schema
+defined earlier in the model, like this:
+
+{% highlight json %}
+{
+ version: '1.0',
+ defaultSchema: 'FOODMART_CLONE',
+ schemas: [
+ {
+ name: 'FOODMART',
+ type: 'custom',
+ factory: 'org.apache.calcite.adapter.jdbc.JdbcSchema$Factory',
+ operand: {
+ jdbcDriver: 'com.mysql.jdbc.Driver',
+ jdbcUrl: 'jdbc:mysql://localhost/foodmart',
+ jdbcUser: 'foodmart',
+ jdbcPassword: 'foodmart'
+ }
+ },
+ {
+ name: 'FOODMART_CLONE',
+ type: 'custom',
+ factory: 'org.apache.calcite.adapter.clone.CloneSchema$Factory',
+ operand: {
+ source: 'FOODMART'
+ }
+ }
+ ]
+}
+{% endhighlight %}
+
+You can use this approach to create a clone schema on any type of
+schema, not just JDBC.
+
+The cloning adapter isn't the be-all and end-all. We plan to develop
+more sophisticated caching strategies, and a more complete and
+efficient implementation of in-memory tables, but for now the cloning
+JDBC adapter shows what is possible and allows us to try out our
+initial implementations.
+
+## Further topics
+
+### Defining a custom schema
+
+(To be written.)
+
+### Modifying data
+
+How to enable DML operations (INSERT, UPDATE and DELETE) on your schema.
+
+(To be written.)
+
+### Calling conventions
+
+(To be written.)
+
+### Statistics and cost
+
+(To be written.)
+
+### Defining and using user-defined functions
+
+(To be written.)
+
+### Defining tables in a schema
+
+(To be written.)
+
+### Defining custom tables
+
+(To be written.)
+
+### Built-in SQL implementation
+
+How does Calcite implement SQL, if an adapter does not implement all of the core
+relational operators?
+
+(To be written.)
+
+### Table functions
+
+(To be written.)
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/anchor_links.html
----------------------------------------------------------------------
diff --git a/site/_includes/anchor_links.html b/site/_includes/anchor_links.html
new file mode 100644
index 0000000..c584ce5
--- /dev/null
+++ b/site/_includes/anchor_links.html
@@ -0,0 +1,33 @@
+<script>
+ var anchorForId = function (id) {
+ var anchor = document.createElement("a");
+ anchor.className = "header-link";
+ anchor.href = "#" + id;
+ anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
+ anchor.title = "Permalink";
+ return anchor;
+ };
+
+ var linkifyAnchors = function (level, containingElement) {
+ var headers = containingElement.getElementsByTagName("h" + level);
+ for (var h = 0; h < headers.length; h++) {
+ var header = headers[h];
+
+ if (typeof header.id !== "undefined" && header.id !== "") {
+ header.appendChild(anchorForId(header.id));
+ }
+ }
+ };
+
+ document.onreadystatechange = function () {
+ if (this.readyState === "complete") {
+ var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
+ if (!contentBlock) {
+ return;
+ }
+ for (var level = 1; level <= 6; level++) {
+ linkifyAnchors(level, contentBlock);
+ }
+ }
+ };
+</script>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/docs_contents.html
----------------------------------------------------------------------
diff --git a/site/_includes/docs_contents.html b/site/_includes/docs_contents.html
new file mode 100644
index 0000000..2ac64bb
--- /dev/null
+++ b/site/_includes/docs_contents.html
@@ -0,0 +1,8 @@
+<div class="unit one-fifth hide-on-mobiles">
+ <aside>
+ {% for section in site.data.docs %}
+ <h4>{{ section.title }}</h4>
+ {% include docs_ul.html items=section.docs %}
+ {% endfor %}
+ </aside>
+</div>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/docs_contents_mobile.html
----------------------------------------------------------------------
diff --git a/site/_includes/docs_contents_mobile.html b/site/_includes/docs_contents_mobile.html
new file mode 100644
index 0000000..b3e0110
--- /dev/null
+++ b/site/_includes/docs_contents_mobile.html
@@ -0,0 +1,10 @@
+<div class="docs-nav-mobile unit whole show-on-mobiles">
+ <select onchange="if (this.value) window.location.href=this.value">
+ <option value="">Navigate the docs…</option>
+ {% for section in site.data.docs %}
+ <optgroup label="{{ section.title }}">
+ {% include docs_option.html items=section.docs %}
+ </optgroup>
+ {% endfor %}
+ </select>
+</div>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/docs_option.html
----------------------------------------------------------------------
diff --git a/site/_includes/docs_option.html b/site/_includes/docs_option.html
new file mode 100644
index 0000000..0913a59
--- /dev/null
+++ b/site/_includes/docs_option.html
@@ -0,0 +1,11 @@
+{% assign items = include.items %}
+
+{% for item in items %}
+ {% assign item_url = item | prepend:"/docs/" | append:".html" %}
+
+ {% for p in site.docs %}
+ {% if p.url == item_url %}
+ <option value="{{ site.url }}{{ p.url }}">{{ p.title }}</option>
+ {% endif %}
+ {% endfor %}
+{% endfor %}
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/docs_ul.html
----------------------------------------------------------------------
diff --git a/site/_includes/docs_ul.html b/site/_includes/docs_ul.html
new file mode 100644
index 0000000..fa585e6
--- /dev/null
+++ b/site/_includes/docs_ul.html
@@ -0,0 +1,21 @@
+{% assign items = include.items %}
+
+<ul>
+{% for item in items %}
+ {% assign item_url = item | prepend:"/docs/" | append:".html" %}
+
+ {% if item_url == page.url %}
+ {% assign c = "current" %}
+ {% else %}
+ {% assign c = "" %}
+ {% endif %}
+
+ {% for p in site.docs %}
+ {% if p.url == item_url %}
+ <li class="{{ c }}"><a href="{{ site.url }}{{ p.url }}">{{ p.title }}</a></li>
+ {% break %}
+ {% endif %}
+ {% endfor %}
+
+{% endfor %}
+</ul>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/footer.html
----------------------------------------------------------------------
diff --git a/site/_includes/footer.html b/site/_includes/footer.html
new file mode 100644
index 0000000..1eb6b11
--- /dev/null
+++ b/site/_includes/footer.html
@@ -0,0 +1,19 @@
+<footer role="contentinfo">
+ <p>The contents of this website are © {{ site.time | date: '%Y' }}
+ <a href="https://www.apache.org/">Apache Software Foundation</a>
+ under the terms of
+ the <a href="https://www.apache.org/licenses/LICENSE-2.0.html">
+ Apache License v2</a>. Apache Calcite and its logo are
+ trademarks of the Apache Software Foundation.</p>
+
+ <p>Apache Calcite is an effort undergoing incubation at The Apache
+ Software Foundation (ASF), sponsored by the Apache
+ Incubator. Incubation is required of all newly accepted projects
+ until a further review indicates that the infrastructure,
+ communications, and decision making process have stabilized in a
+ manner consistent with other successful ASF projects. While
+ incubation status is not necessarily a reflection of the
+ completeness or stability of the code, it does indicate that the
+ project has yet to be fully endorsed by the ASF.</p>
+
+</footer>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/header.html
----------------------------------------------------------------------
diff --git a/site/_includes/header.html b/site/_includes/header.html
new file mode 100644
index 0000000..54f7748
--- /dev/null
+++ b/site/_includes/header.html
@@ -0,0 +1,18 @@
+<header role="banner">
+ <nav class="mobile-nav show-on-mobiles">
+ {% include primary-nav-items.html %}
+ </nav>
+ <div class="grid">
+ <div class="unit one-third center-on-mobiles">
+ <h1>
+ <a href="/">
+ <span class="sr-only">Apache Calcite</span>
+ <img src="/img/logo.png" width="226" height="140" alt="Calcite Logo">
+ </a>
+ </h1>
+ </div>
+ <nav class="main-nav unit two-thirds hide-on-mobiles">
+ {% include primary-nav-items.html %}
+ </nav>
+ </div>
+</header>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/news_contents.html
----------------------------------------------------------------------
diff --git a/site/_includes/news_contents.html b/site/_includes/news_contents.html
new file mode 100644
index 0000000..86a2431
--- /dev/null
+++ b/site/_includes/news_contents.html
@@ -0,0 +1,30 @@
+<div class="unit one-fifth hide-on-mobiles">
+ <aside>
+ <ul>
+ <li class="{% if page.title == 'News' %}current{% endif %}">
+ <a href="/news/">All News</a>
+ </li>
+ <li class="{% if page.title == 'Releases' %}current{% endif %}">
+ <a href="/news/releases/">Calcite Releases</a>
+ </li>
+ </ul>
+ <h4>Recent Releases</h4>
+ <ul>
+ {% for post in site.categories.release limit:5 %}
+ <li class="{% if page.title == post.title %}current{% endif %}">
+ <a href="{{ post.url }}">{{ post.version }}</a>
+ </li>
+ {% endfor %}
+ </ul>
+ <h4>Other News</h4>
+ <ul>
+ {% for post in site.posts %}
+ {% unless post.categories contains 'release' %}
+ <li class="{% if page.title == post.title %}current{% endif %}">
+ <a href="{{ post.url }}">{{ post.title }}</a>
+ </li>
+ {% endunless %}
+ {% endfor %}
+ </ul>
+ </aside>
+</div>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/news_contents_mobile.html
----------------------------------------------------------------------
diff --git a/site/_includes/news_contents_mobile.html b/site/_includes/news_contents_mobile.html
new file mode 100644
index 0000000..e8fb55e
--- /dev/null
+++ b/site/_includes/news_contents_mobile.html
@@ -0,0 +1,11 @@
+<div class="docs-nav-mobile unit whole show-on-mobiles">
+ <select onchange="if (this.value) window.location.href=this.value">
+ <option value="">Navigate the blog…</option>
+ <option value="/news/">Home</option>
+ <optgroup label="v1.x">
+ {% for post in site.posts %}
+ <option value="{{ post.url }}">{{ post.title }}</option>
+ {% endfor %}
+ </optgroup>
+ </select>
+</div>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/news_item.html
----------------------------------------------------------------------
diff --git a/site/_includes/news_item.html b/site/_includes/news_item.html
new file mode 100644
index 0000000..e543bc6
--- /dev/null
+++ b/site/_includes/news_item.html
@@ -0,0 +1,45 @@
+<article>
+ <h2>
+ <a href="{{ post.url }}">
+ {{ post.title }}
+ </a>
+ </h2>
+ <span class="post-category">
+ <span class="label">
+ {{ post.categories | array_to_sentence_string }}
+ </span>
+ </span>
+ <div class="post-meta">
+ <span class="post-date">
+ {{ post.date | date_to_string }}
+ </span>
+ {% capture homepage %}http://people.apache.org/~{{ post.author }}{% endcapture %}
+ {% capture avatar %}http://people.apache.org/~{{ post.author }}/{{ post.author }}.jpg{% endcapture %}
+ {% for c in site.data.contributors %}
+ {% if c.apacheId == post.author %}
+ {% if c.homepage %}
+ {% assign homepage = c.homepage %}
+ {% else %}
+ {% capture homepage %}http://github.com/{{ c.githubId }}{% endcapture %}
+ {% endif %}
+ {% if c.avatar %}
+ {% assign avatar = c.avatar %}
+ {% else %}
+ {% capture avatar %}http://github.com/{{ c.githubId }}.png{% endcapture %}
+ {% endif %}
+ {% endif %}
+ {% endfor %}
+ <a href="{{ homepage }}" class="post-author">
+ <img src="{{ avatar }}"
+ class="avatar" alt="{{ post.author }} avatar"
+ width="24" height="24">
+ {{ post.author }}
+ </a>
+ </div>
+ <div class="post-content">
+ {{ post.content }}
+ {% if post.categories contains 'release' %}
+ <p>See the <a href="/docs/history.html#{{ post.tag }}">release notes</a>.</p>
+ {% endif %}
+ </div>
+</article>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/primary-nav-items.html
----------------------------------------------------------------------
diff --git a/site/_includes/primary-nav-items.html b/site/_includes/primary-nav-items.html
new file mode 100644
index 0000000..297ce65
--- /dev/null
+++ b/site/_includes/primary-nav-items.html
@@ -0,0 +1,20 @@
+<ul>
+ <li class="{% if page.overview %}current{% endif %}">
+ <a href="/">Home</a>
+ </li>
+ <li class="{% if page.url contains '/docs/' %}current{% endif %}">
+ <a href="/docs/">Documentation</a>
+ </li>
+ <li class="{% if page.url contains '/talks/' %}current{% endif %}">
+ <a href="/talks/">Talks</a>
+ </li>
+ <li class="{% if page.url contains '/news/' %}current{% endif %}">
+ <a href="/news/">News</a>
+ </li>
+ <li class="{% if page.url contains '/help/' %}current{% endif %}">
+ <a href="/help/">Help</a>
+ </li>
+ <li class="{% if page.url contains '/develop/' %}current{% endif %}">
+ <a href="/develop/">Develop</a>
+ </li>
+</ul>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/section_nav.html
----------------------------------------------------------------------
diff --git a/site/_includes/section_nav.html b/site/_includes/section_nav.html
new file mode 100644
index 0000000..1b3bca1
--- /dev/null
+++ b/site/_includes/section_nav.html
@@ -0,0 +1,39 @@
+{% comment %}
+Map grabs the doc sections, giving us an array of arrays. Join, flattens all
+the items to a comma delimited string. Split turns it into an array again.
+{% endcomment %}
+{% assign docs = site.data.docs | map: 'docs' | join: ',' | split: ',' %}
+
+{% comment %}
+Because this is built for every page, lets find where we are in the ordered
+document list by comparing url strings. Then if there's something previous or
+next, lets build a link to it.
+{% endcomment %}
+
+{% for document in docs %}
+ {% assign document_url = document | prepend:"/docs/" | append:".html" %}
+ {% if document_url == page.url %}
+ <div class="section-nav">
+ <div class="left align-right">
+ {% if forloop.first %}
+ <span class="prev disabled">Previous</span>
+ {% else %}
+ {% assign previous = forloop.index0 | minus: 1 %}
+ {% assign previous_page = docs[previous] | prepend:"/docs/" | append:".html" %}
+ <a href="{{ previous_page }}" class="prev">Previous</a>
+ {% endif %}
+ </div>
+ <div class="right align-left">
+ {% if forloop.last %}
+ <span class="next disabled">Next</span>
+ {% else %}
+ {% assign next = forloop.index0 | plus: 1 %}
+ {% assign next_page = docs[next] | prepend:"/docs/" | append:".html" %}
+ <a href="{{ next_page }}" class="next">Next</a>
+ {% endif %}
+ </div>
+ </div>
+ <div class="clear"></div>
+ {% break %}
+ {% endif %}
+{% endfor %}
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_includes/top.html
----------------------------------------------------------------------
diff --git a/site/_includes/top.html b/site/_includes/top.html
new file mode 100644
index 0000000..a4c04e7
--- /dev/null
+++ b/site/_includes/top.html
@@ -0,0 +1,15 @@
+<!DOCTYPE HTML>
+<html lang="en-US">
+<head>
+ <meta charset="UTF-8">
+ <title>{{ page.title }}</title>
+ <meta name="viewport" content="width=device-width,initial-scale=1">
+ <meta name="generator" content="Jekyll v{{ jekyll.version }}">
+ <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
+ <link rel="stylesheet" href="/css/screen.css">
+ <link rel="icon" type="image/x-icon" href="/favicon.ico">
+ <!--[if lt IE 9]>
+ <script src="/js/html5shiv.min.js"></script>
+ <script src="/js/respond.min.js"></script>
+ <![endif]-->
+</head>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_layouts/default.html
----------------------------------------------------------------------
diff --git a/site/_layouts/default.html b/site/_layouts/default.html
new file mode 100644
index 0000000..f734c79
--- /dev/null
+++ b/site/_layouts/default.html
@@ -0,0 +1,12 @@
+{% include top.html %}
+
+<body class="wrap">
+ {% include header.html %}
+
+ {{ content }}
+
+ {% include footer.html %}
+ {% include anchor_links.html %}
+
+</body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_layouts/docs.html
----------------------------------------------------------------------
diff --git a/site/_layouts/docs.html b/site/_layouts/docs.html
new file mode 100644
index 0000000..a0a8a5c
--- /dev/null
+++ b/site/_layouts/docs.html
@@ -0,0 +1,23 @@
+---
+layout: default
+---
+
+ <section class="docs">
+ <div class="grid">
+
+ {% include docs_contents_mobile.html %}
+
+ <div class="unit four-fifths">
+ <article>
+ <h1>{{ page.title }}</h1>
+ {{ content }}
+ {% include section_nav.html %}
+ </article>
+ </div>
+
+ {% include docs_contents.html %}
+
+ <div class="clear"></div>
+
+ </div>
+ </section>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_layouts/external.html
----------------------------------------------------------------------
diff --git a/site/_layouts/external.html b/site/_layouts/external.html
new file mode 100644
index 0000000..b934309
--- /dev/null
+++ b/site/_layouts/external.html
@@ -0,0 +1,9 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <title>{{ page.title }}</title>
+
+ <meta http-equiv="refresh" content="0;url={{ page.external_url }}">
+ </head>
+ <body><!-- Google Analytics JavaScript --></body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_layouts/news.html
----------------------------------------------------------------------
diff --git a/site/_layouts/news.html b/site/_layouts/news.html
new file mode 100644
index 0000000..8f7945f
--- /dev/null
+++ b/site/_layouts/news.html
@@ -0,0 +1,19 @@
+---
+layout: default
+---
+
+ <section class="news">
+ <div class="grid">
+
+ {% include news_contents_mobile.html %}
+
+ <div class="unit four-fifths">
+ {{ content }}
+ </div>
+
+ {% include news_contents.html %}
+
+ <div class="clear"></div>
+
+ </div>
+ </section>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_layouts/news_item.html
----------------------------------------------------------------------
diff --git a/site/_layouts/news_item.html b/site/_layouts/news_item.html
new file mode 100644
index 0000000..7981dc1
--- /dev/null
+++ b/site/_layouts/news_item.html
@@ -0,0 +1,48 @@
+---
+layout: news
+---
+
+<article>
+ <h2>
+ {{ page.title }}
+ <a href="{{ page.url }}" class="permalink" title="Permalink">∞</a>
+ </h2>
+ <span class="post-category">
+ <span class="label">
+ {{ page.categories | array_to_sentence_string }}
+ </span>
+ </span>
+ <div class="post-meta">
+ <span class="post-date">
+ {{ page.date | date_to_string }}
+ </span>
+ {% capture homepage %}http://people.apache.org/~{{ page.author }}{% endcapture %}
+ {% capture avatar %}http://people.apache.org/~{{ page.author }}/{{ page.author }}.jpg{% endcapture %}
+ {% for c in site.data.contributors %}
+ {% if c.apacheId == page.author %}
+ {% if c.homepage %}
+ {% assign homepage = c.homepage %}
+ {% else %}
+ {% capture homepage %}http://github.com/{{ c.githubId }}{% endcapture %}
+ {% endif %}
+ {% if c.avatar %}
+ {% assign avatar = c.avatar %}
+ {% else %}
+ {% capture avatar %}http://github.com/{{ c.githubId }}.png{% endcapture %}
+ {% endif %}
+ {% endif %}
+ {% endfor %}
+ <a href="{{ homepage }}" class="post-author">
+ <img src="{{ avatar }}"
+ class="avatar" alt="{{ page.author }} avatar"
+ width="24" height="24">
+ {{ page.author }}
+ </a>
+ </div>
+ <div class="post-content">
+ {{ content }}
+ {% if page.categories contains 'release' %}
+ <p>See the <a href="/docs/history.html#{{ page.tag }}">release notes</a>.</p>
+ {% endif %}
+ </div>
+</article>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_layouts/page.html
----------------------------------------------------------------------
diff --git a/site/_layouts/page.html b/site/_layouts/page.html
new file mode 100644
index 0000000..bae31bb
--- /dev/null
+++ b/site/_layouts/page.html
@@ -0,0 +1,18 @@
+---
+layout: default
+---
+
+<section class="standalone">
+ <div class="grid">
+
+ <div class="unit whole">
+ <article>
+ <h1>{{ page.title }}</h1>
+ {{ content }}
+ </article>
+ </div>
+
+ <div class="clear"></div>
+
+ </div>
+</section>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_posts/2014-06-27-release-0.8.0-incubating.md
----------------------------------------------------------------------
diff --git a/site/_posts/2014-06-27-release-0.8.0-incubating.md b/site/_posts/2014-06-27-release-0.8.0-incubating.md
new file mode 100644
index 0000000..32188eb
--- /dev/null
+++ b/site/_posts/2014-06-27-release-0.8.0-incubating.md
@@ -0,0 +1,11 @@
+---
+layout: news_item
+date: "2014-06-27 00:00:00 -0800"
+author: jhyde
+version: 0.8
+tag: v0-8
+sha: 3da850a1
+categories: [release]
+---
+
+This is the first release under the Apache incubator process.
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_posts/2014-08-19-release-0.9.0-incubating.md
----------------------------------------------------------------------
diff --git a/site/_posts/2014-08-19-release-0.9.0-incubating.md b/site/_posts/2014-08-19-release-0.9.0-incubating.md
new file mode 100644
index 0000000..1826487
--- /dev/null
+++ b/site/_posts/2014-08-19-release-0.9.0-incubating.md
@@ -0,0 +1,12 @@
+---
+layout: news_item
+date: "2014-08-19 00:00:00 -0800"
+author: jhyde
+version: 0.9.0-incubating
+fullVersion: apache-optiq-0.9.0-incubating
+tag: v0-9-0
+sha: 45e5269b
+categories: [release]
+---
+
+This is the first release under the Apache incubator process.
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_posts/2014-10-02-release-0.9.1-incubating.md
----------------------------------------------------------------------
diff --git a/site/_posts/2014-10-02-release-0.9.1-incubating.md b/site/_posts/2014-10-02-release-0.9.1-incubating.md
new file mode 100644
index 0000000..ce0d6d0
--- /dev/null
+++ b/site/_posts/2014-10-02-release-0.9.1-incubating.md
@@ -0,0 +1,11 @@
+---
+layout: news_item
+date: "2014-10-02 00:00:00 -0800"
+author: jhyde
+version: 0.9.1-incubating
+tag: v0-9-1
+sha: 68012573
+categories: [release]
+---
+
+This is the first release as Calcite. (The project was previously called Optiq.)
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_posts/2014-11-05-release-0.9.2-incubating.md
----------------------------------------------------------------------
diff --git a/site/_posts/2014-11-05-release-0.9.2-incubating.md b/site/_posts/2014-11-05-release-0.9.2-incubating.md
new file mode 100644
index 0000000..ef0746b
--- /dev/null
+++ b/site/_posts/2014-11-05-release-0.9.2-incubating.md
@@ -0,0 +1,14 @@
+---
+layout: news_item
+date: "2014-11-05 00:00:00 -0800"
+author: jhyde
+version: 0.9.2-incubating
+tag: v0-9-2
+sha: 0404fd23
+categories: [release]
+---
+
+A fairly minor release, and last release before we rename all of the
+packages and lots of classes, in what we expect to call 1.0. If you
+have an existing application, it's worth upgrading to this first,
+before you move on to 1.0.
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_posts/2015-01-31-release-1.0.0-incubating.md
----------------------------------------------------------------------
diff --git a/site/_posts/2015-01-31-release-1.0.0-incubating.md b/site/_posts/2015-01-31-release-1.0.0-incubating.md
new file mode 100644
index 0000000..ef9467f
--- /dev/null
+++ b/site/_posts/2015-01-31-release-1.0.0-incubating.md
@@ -0,0 +1,24 @@
+---
+layout: news_item
+date: "2015-01-31 19:03:07 -0800"
+author: jhyde
+version: 1.0.0-incubating
+tag: v1-0-0
+sha: 2dd83f2
+categories: [release]
+---
+
+Calcite's first major release.
+
+Since the previous release we have re-organized the into the `org.apache.calcite`
+namespace. To make migration of your code easier, we have described the
+<a href="https://issues.apache.org/jira/secure/attachment/12681620/mapping.txt">mapping from old to new class names</a>
+as an attachment to
+[<a href="https://issues.apache.org/jira/browse/CALCITE-296">CALCITE-296</a>].
+
+The release adds SQL support for `GROUPING SETS`, `EXTEND`, `UPSERT` and sequences;
+a remote JDBC driver;
+improvements to the planner engine and built-in planner rules;
+improvements to the algorithms that implement the relational algebra,
+including an interpreter that can evaluate queries without compilation;
+and fixes about 30 bugs.
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_posts/2015-03-13-release-1.1.0-incubating.md
----------------------------------------------------------------------
diff --git a/site/_posts/2015-03-13-release-1.1.0-incubating.md b/site/_posts/2015-03-13-release-1.1.0-incubating.md
new file mode 100644
index 0000000..e921459
--- /dev/null
+++ b/site/_posts/2015-03-13-release-1.1.0-incubating.md
@@ -0,0 +1,23 @@
+---
+layout: news_item
+date: "2015-03-13 19:03:07 -0800"
+author: jhyde
+version: 1.1.0-incubating
+tag: v1-1-0
+sha: f10ea367
+categories: [release]
+---
+
+This Calcite release makes it possible to exploit physical properties
+of relational expressions to produce more efficient plans, introducing
+collation and distribution as traits, `Exchange` relational operator,
+and several new forms of metadata.
+
+We add experimental support for streaming SQL.
+
+This release drops support for JDK 1.6; Calcite now requires 1.7 or
+later.
+
+We have introduced static `create` methods for many sub-classes of
+`RelNode`. We strongly suggest that you use these rather than
+calling constructors directly.
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_posts/2015-04-07-release-1.2.0-incubating.md
----------------------------------------------------------------------
diff --git a/site/_posts/2015-04-07-release-1.2.0-incubating.md b/site/_posts/2015-04-07-release-1.2.0-incubating.md
new file mode 100644
index 0000000..43bb6e3
--- /dev/null
+++ b/site/_posts/2015-04-07-release-1.2.0-incubating.md
@@ -0,0 +1,23 @@
+---
+layout: news_item
+date: "2015-04-07 19:03:07 -0800"
+author: jhyde
+version: 1.2.0-incubating
+tag: v1-2-0
+sha: d60f2aa
+categories: [release]
+---
+
+A short release, less than a month after 1.1.
+
+There have been many changes to Avatica, hugely improving its coverage of the
+JDBC API and overall robustness. A new provider, `JdbcMeta`, allows
+you to remote an existing JDBC driver.
+
+[<a href="https://issues.apache.org/jira/browse/CALCITE-606">CALCITE-606</a>]
+improves how the planner propagates traits such as collation and
+distribution among relational expressions.
+
+[<a href="https://issues.apache.org/jira/browse/CALCITE-613">CALCITE-613</a>]
+and [<a href="https://issues.apache.org/jira/browse/CALCITE-307">CALCITE-307</a>]
+improve implicit and explicit conversions in SQL.
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_posts/2015-04-24-new-committers.md
----------------------------------------------------------------------
diff --git a/site/_posts/2015-04-24-new-committers.md b/site/_posts/2015-04-24-new-committers.md
new file mode 100644
index 0000000..a0798c2
--- /dev/null
+++ b/site/_posts/2015-04-24-new-committers.md
@@ -0,0 +1,16 @@
+---
+layout: news_item
+title: "Calcite adds 5 committers"
+date: "2015-04-24 19:03:07 -0800"
+author: jhyde
+categories: [team]
+---
+
+The Calcite project management committee today added five new
+committers for their work on Calcite. Welcome all!
+
+* Aman Sinha
+* Jesús Camacho-Rodríguez
+* Jinfeng Ni
+* John Pullokkaran
+* Nick Dimiduk
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_posts/2015-05-30-release-1.3.0-incubating.md
----------------------------------------------------------------------
diff --git a/site/_posts/2015-05-30-release-1.3.0-incubating.md b/site/_posts/2015-05-30-release-1.3.0-incubating.md
new file mode 100644
index 0000000..3ea81ba
--- /dev/null
+++ b/site/_posts/2015-05-30-release-1.3.0-incubating.md
@@ -0,0 +1,15 @@
+---
+layout: news_item
+date: "2015-05-30 23:05:37 +0000"
+author: jhyde
+version: 1.3.0-incubating
+categories: [release]
+tag: v1-3-0
+sha: 495f1859
+---
+
+Mainly bug-fixes, but this release adds support for
+<a href="https://issues.apache.org/jira/browse/CALCITE-505">modifiable views</a>
+and
+<a href="https://issues.apache.org/jira/browse/CALCITE-704">filtered aggregate functions</a>
+and various improvements to Avatica.
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_sass/_font-awesome.scss
----------------------------------------------------------------------
diff --git a/site/_sass/_font-awesome.scss b/site/_sass/_font-awesome.scss
new file mode 100644
index 0000000..d90676c
--- /dev/null
+++ b/site/_sass/_font-awesome.scss
@@ -0,0 +1,25 @@
+/*!
+ * Font Awesome 4.2.0 by @davegandy - http://fontawesome.io - @fontawesome
+ * License - http://fontawesome.io/license (Font: SIL OFL 1.1, CSS: MIT License)
+ */
+@font-face {
+ font-family: 'FontAwesome';
+ src: url('../fonts/fontawesome-webfont.eot?v=4.2.0');
+ src: url('../fonts/fontawesome-webfont.eot?#iefix&v=4.2.0') format('embedded-opentype'), url('../fonts/fontawesome-webfont.woff?v=4.2.0') format('woff'), url('../fonts/fontawesome-webfont.ttf?v=4.2.0') format('truetype'), url('../fonts/fontawesome-webfont.svg?v=4.2.0#fontawesomeregular') format('svg');
+ font-weight: normal;
+ font-style: normal;
+}
+.fa {
+ display: inline-block;
+ font: normal normal normal 14px/1 FontAwesome;
+ font-size: inherit;
+ text-rendering: auto;
+ -webkit-font-smoothing: antialiased;
+ -moz-osx-font-smoothing: grayscale;
+}
+.fa-link:before {
+ content: "\f0c1";
+}
+.fa-pencil:before {
+ content: "\f040";
+}
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_sass/_gridism.scss
----------------------------------------------------------------------
diff --git a/site/_sass/_gridism.scss b/site/_sass/_gridism.scss
new file mode 100644
index 0000000..61698ef
--- /dev/null
+++ b/site/_sass/_gridism.scss
@@ -0,0 +1,124 @@
+/*
+ * Gridism
+ * A simple, responsive, and handy CSS grid by @cobyism
+ * https://github.com/cobyism/gridism
+ */
+
+/* Preserve some sanity */
+.grid,
+.unit {
+ -webkit-box-sizing: border-box;
+ -moz-box-sizing: border-box;
+ box-sizing: border-box;
+}
+
+/* Set up some rules to govern the grid */
+.grid {
+ display: block;
+ clear: both;
+}
+.grid .unit {
+ float: left;
+ width: 100%;
+ padding: 10px;
+}
+
+/* This ensures the outer gutters are equal to the (doubled) inner gutters. */
+.grid .unit:first-child { padding-left: 20px; }
+.grid .unit:last-child { padding-right: 20px; }
+
+/* Nested grids already have padding though, so let’s nuke it */
+.unit .unit:first-child { padding-left: 0; }
+.unit .unit:last-child { padding-right: 0; }
+.unit .grid:first-child > .unit { padding-top: 0; }
+.unit .grid:last-child > .unit { padding-bottom: 0; }
+
+/* Let people nuke the gutters/padding completely in a couple of ways */
+.no-gutters .unit,
+.unit.no-gutters {
+ padding: 0 !important;
+}
+
+/* Wrapping at a maximum width is optional */
+.wrap .grid,
+.grid.wrap {
+ max-width: 978px;
+ margin: 0 auto;
+}
+
+/* Width classes also have shorthand versions numbered as fractions
+ * For example: for a grid unit 1/3 (one third) of the parent width,
+ * simply apply class="w-1-3" to the element. */
+.grid .whole, .grid .w-1-1 { width: 100%; }
+.grid .half, .grid .w-1-2 { width: 50%; }
+.grid .one-third, .grid .w-1-3 { width: 33.3332%; }
+.grid .two-thirds, .grid .w-2-3 { width: 66.6665%; }
+.grid .one-quarter,
+.grid .one-fourth, .grid .w-1-4 { width: 25%; }
+.grid .three-quarters,
+.grid .three-fourths, .grid .w-3-4 { width: 75%; }
+.grid .one-fifth, .grid .w-1-5 { width: 20%; }
+.grid .two-fifths, .grid .w-2-5 { width: 40%; }
+.grid .three-fifths, .grid .w-3-5 { width: 60%; }
+.grid .four-fifths, .grid .w-4-5 { width: 80%; }
+.grid .golden-small, .grid .w-g-s { width: 38.2716%; } /* Golden section: smaller piece */
+.grid .golden-large, .grid .w-g-l { width: 61.7283%; } /* Golden section: larger piece */
+
+/* Clearfix after every .grid */
+.grid {
+ *zoom: 1;
+}
+.grid:before, .grid:after {
+ display: table;
+ content: "";
+ line-height: 0;
+}
+.grid:after {
+ clear: both;
+}
+
+/* Utility classes */
+.align-center { text-align: center; }
+.align-left { text-align: left; }
+.align-right { text-align: right; }
+.pull-left { float: left; }
+.pull-right { float: right; }
+
+/* A property for a better rendering of images in units: in
+ this way bigger pictures are just resized if the unit
+ becomes smaller */
+.unit img {
+ max-width: 100%;
+}
+
+/* Responsive Stuff */
+@media screen and (max-width: 568px) {
+ /* Stack anything that isn’t full-width on smaller screens
+ and doesn't provide the no-stacking-on-mobiles class */
+ .grid:not(.no-stacking-on-mobiles) > .unit {
+ width: 100% !important;
+ padding-left: 20px;
+ padding-right: 20px;
+ }
+ .unit .grid .unit {
+ padding-left: 0px;
+ padding-right: 0px;
+ }
+
+ /* Sometimes, you just want to be different on small screens */
+ .center-on-mobiles {
+ text-align: center !important;
+ }
+ .hide-on-mobiles {
+ display: none !important;
+ }
+}
+
+/* Expand the wrap a bit further on larger screens */
+@media screen and (min-width: 1180px) {
+ .wider .grid,
+ .grid.wider {
+ max-width: 1180px;
+ margin: 0 auto;
+ }
+}
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_sass/_mixins.scss
----------------------------------------------------------------------
diff --git a/site/_sass/_mixins.scss b/site/_sass/_mixins.scss
new file mode 100644
index 0000000..5b9bb43
--- /dev/null
+++ b/site/_sass/_mixins.scss
@@ -0,0 +1,38 @@
+@mixin box-shadow($shadow...) {
+ -webkit-box-shadow: $shadow;
+ -moz-box-shadow: $shadow;
+ box-shadow: $shadow;
+}
+
+@mixin border-radius($radius...) {
+ -webkit-border-radius: $radius;
+ -moz-border-radius: $radius;
+ border-radius: $radius;
+}
+
+@mixin border-top-left-radius($radius...) {
+ -webkit-border-top-left-radius: $radius;
+ -moz-border-radius-topleft: $radius;
+ border-top-left-radius: $radius;
+}
+
+@mixin border-top-right-radius($radius...) {
+ -webkit-border-top-right-radius: $radius;
+ -moz-border-radius-topright: $radius;
+ border-top-right-radius: $radius;
+}
+
+@mixin transition($transition...) {
+ -webkit-transition: $transition;
+ -moz-transition: $transition;
+ -o-transition: $transition;
+ transition: $transition;
+}
+
+@mixin user-select($select...) {
+ -webkit-user-select: $select; /* Chrome all / Safari all */
+ -moz-user-select: $select; /* Firefox all */
+ -ms-user-select: $select; /* IE 10+ */
+ -o-user-select: $select;
+ user-select: $select;
+}
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/5c049bc8/site/_sass/_normalize.scss
----------------------------------------------------------------------
diff --git a/site/_sass/_normalize.scss b/site/_sass/_normalize.scss
new file mode 100644
index 0000000..f6e0b65
--- /dev/null
+++ b/site/_sass/_normalize.scss
@@ -0,0 +1 @@
+/*! normalize.css v3.0.2 | MIT License | git.io/normalize */html{font-family:sans-serif;-ms-text-size-adjust:100%;-webkit-text-size-adjust:100%}body{margin:0}article,aside,details,figcaption,figure,footer,header,hgroup,main,menu,nav,section,summary{display:block}audio,canvas,progress,video{display:inline-block;vertical-align:baseline}audio:not([controls]){display:none;height:0}[hidden],template{display:none}a{background-color:transparent}a:active,a:hover{outline:0}abbr[title]{border-bottom:1px dotted}b,strong{font-weight:700}dfn{font-style:italic}h1{font-size:2em;margin:.67em 0}mark{background:#ff0;color:#000}small{font-size:80%}sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}sup{top:-.5em}sub{bottom:-.25em}img{border:0}svg:not(:root){overflow:hidden}figure{margin:1em 40px}hr{-moz-box-sizing:content-box;box-sizing:content-box;height:0}pre{overflow:auto}code,kbd,pre,samp{font-family:monospace,monospace;font-size:1em}button,input,optgroup,select,textarea{
color:inherit;font:inherit;margin:0}button{overflow:visible}button,select{text-transform:none}button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}button[disabled],html input[disabled]{cursor:default}button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}input{line-height:normal}input[type=checkbox],input[type=radio]{box-sizing:border-box;padding:0}input[type=number]::-webkit-inner-spin-button,input[type=number]::-webkit-outer-spin-button{height:auto}input[type=search]{-webkit-appearance:textfield;-moz-box-sizing:content-box;-webkit-box-sizing:content-box;box-sizing:content-box}input[type=search]::-webkit-search-cancel-button,input[type=search]::-webkit-search-decoration{-webkit-appearance:none}fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}legend{border:0;padding:0}textarea{overflow:auto}optgroup{font-weight:700}table{border-collapse:collapse;border-spacing:0}td,th{padding:0}
\ No newline at end of file