You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@calcite.apache.org by jh...@apache.org on 2015/04/17 20:35:40 UTC
[1/8] incubator-calcite git commit: [CALCITE-507] Update HOWTO.md
with running integration tests
Repository: incubator-calcite
Updated Branches:
refs/heads/master 28d640af3 -> b18a4df3d
[CALCITE-507] Update HOWTO.md with running integration tests
Project: http://git-wip-us.apache.org/repos/asf/incubator-calcite/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-calcite/commit/ca871d37
Tree: http://git-wip-us.apache.org/repos/asf/incubator-calcite/tree/ca871d37
Diff: http://git-wip-us.apache.org/repos/asf/incubator-calcite/diff/ca871d37
Branch: refs/heads/master
Commit: ca871d3776646e94e6b1a1d61692c6d80777952f
Parents: b2b53b1
Author: Vladimir Sitnikov <si...@gmail.com>
Authored: Tue Mar 10 23:46:12 2015 +0300
Committer: Julian Hyde <jh...@apache.org>
Committed: Thu Apr 16 02:21:43 2015 -0700
----------------------------------------------------------------------
doc/HOWTO.md | 71 +++++++++++++++++++++++++++++++++++++++++++++++--------
1 file changed, 61 insertions(+), 10 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/ca871d37/doc/HOWTO.md
----------------------------------------------------------------------
diff --git a/doc/HOWTO.md b/doc/HOWTO.md
index ae7a167..09538ec 100644
--- a/doc/HOWTO.md
+++ b/doc/HOWTO.md
@@ -45,36 +45,85 @@ The test suite will run by default when you build, unless you specify
`-DskipTests`:
```bash
-$ mvn clean
+$ mvn clean # Note: mvn clean install does not work, use mvn clean && mvn install
$ mvn -DskipTests install
```
There are other options that control which tests are run, and in what
environment, as follows.
-* `-Dcalcite.test.db=DB` (where db is `hsqldb` or `mysql`) allows you
+* `-Dcalcite.test.db=DB` (where db is `h2`, `hsqldb`, `mysql`, or `postgresql`) allows you
to change the JDBC data source for the test suite. Calcite's test
suite requires a JDBC data source populated with the foodmart data
set.
* `hsqldb`, the default, uses an in-memory hsqldb database.
- * `mysql` uses a MySQL database in `jdbc:mysql://localhost/foodmart`.
- It is somewhat faster than hsqldb, but you need to populate it
- manually.
+ * all others access test virtual machine (see [integration tests](HOWTO.md#Running-integration-tests) below)
+ `mysql` and `postgresql` might be somewhat faster than hsqldb, but you need to populate it (i.e. provision a VM).
* `-Dcalcite.debug` prints extra debugging information to stdout.
* `-Dcalcite.test.slow` enables tests that take longer to execute. For
example, there are tests that create virtual TPC-H and TPC-DS schemas
in-memory and run tests from those benchmarks.
-* `-Dcalcite.test.mongodb=true` enables tests that run against
- MongoDB. MongoDB must be installed, running, and
- [populated with the zips.json data set](HOWTO.md#mongodb-adapter).
* `-Dcalcite.test.splunk=true` enables tests that run against Splunk.
Splunk must be installed and running.
-To execute tests against mongodb, mysql only, use the following command:
+## Running integration tests
+
+For testing Calcite's external adapters, a test virtual machine should be used.
+The VM includes H2, HSQLDB, MySQL, MongoDB, and PostgreSQL.
+
+Test VM requires 5GiB of disk space and it takes 30 minutes to build.
+
+Note: you can use [calcite-test-dataset](https://github.com/vlsi/calcite-test-dataset)
+ to populate your own database, however it is recommended to use test VM so the test environment can be reproduced.
+
+### VM preparation
+
+0) Install dependencies: [Vagrant](https://www.vagrantup.com/) and [VirtualBox](https://www.virtualbox.org/)
+
+1) Clone https://github.com/vlsi/calcite-test-dataset.git at the same level as calcite repository.
+For instance:
+```bash
+code
+ +-- calcite
+ +-- calcite-test-dataset
+```
+
+Note: integration tests search for ../calcite-test-dataset or ../../calcite-test-dataset.
+ You can specify full path via calcite.test.dataset system property.
+
+2) Build and start the VM:
```bash
-mvn -Dtest=foo -DfailIfNoTests=false -Pit verify
+cd calcite-test-dataset && mvn install
```
+### VM management
+
+Test VM is provisioned by Vagrant, so regular Vagrant `vagrant up` and `vagrant halt` should be used to start and stop the VM.
+The connection strings for different databases are listed in [calcite-test-dataset](https://github.com/vlsi/calcite-test-dataset) readme.
+
+### Suggested test flow
+
+Note: test VM should be started before you launch integration tests. Calcite itself does not start/stop the VM.
+
+Command line:
+* Executing regular unit tests (does not require external data): no change. `mvn test` or `mvn install`.
+* Executing all tests, for all the DBs: `mvn verify -Pit`. `it` stands for "integration-test". `mvn install -Pit` works as well.
+* Executing just tests for external DBs, excluding unit tests: `mvn -Dtest=foo -DfailIfNoTests=false -Pit verify`
+* Executing just MongoDB tests: `cd mongo; mvn verify -Pit`
+
+From within IDE:
+* Executing regular unit tests: no change.
+* Executing MongoDB tests: run `MongoAdapterIT.java` as usual (no additional properties are required)
+* Executing MySQL tests: run `JdbcTest` and `JdbcAdapterTest` with setting `-Dcalcite.test.db=mysql`
+* Executing PostgreSQL tests: run `JdbcTest` and `JdbcAdapterTest` with setting `-Dcalcite.test.db=postgresql`
+
+### Integration tests technical details
+
+Tests with external data are executed at maven's integration-test phase.
+We do not currently use pre-integration-test/post-integration-test, however we could use that in future.
+The verification of build pass/failure is performed at verify phase.
+Integration tests should be named `...IT.java`, so they are not picked up on unit test execution.
+
## Contributing
We welcome contributions.
@@ -147,6 +196,8 @@ See the <a href="TUTORIAL.md">tutorial</a>.
First, download and install Calcite,
and <a href="http://www.mongodb.org/downloads">install MongoDB</a>.
+Note: you can use MongoDB from integration test virtual machine above.
+
Import MongoDB's zipcode data set into MongoDB:
```bash
[4/8] incubator-calcite git commit: [CALCITE-590] Update MongoDB test
suite to calcite-test-dataset
Posted by jh...@apache.org.
[CALCITE-590] Update MongoDB test suite to calcite-test-dataset
Project: http://git-wip-us.apache.org/repos/asf/incubator-calcite/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-calcite/commit/1fd18a3e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-calcite/tree/1fd18a3e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-calcite/diff/1fd18a3e
Branch: refs/heads/master
Commit: 1fd18a3e9af3863d26db822b707cd7107d79b744
Parents: 28d640a
Author: Vladimir Sitnikov <si...@gmail.com>
Authored: Mon Feb 9 08:45:14 2015 +0300
Committer: Julian Hyde <jh...@apache.org>
Committed: Thu Apr 16 02:21:43 2015 -0700
----------------------------------------------------------------------
avatica-server/pom.xml | 8 -
avatica/pom.xml | 8 -
core/pom.xml | 37 +-
doc/HOWTO.md | 5 +
mongodb/pom.xml | 12 -
.../org/apache/calcite/test/MongoAdapterIT.java | 744 +++++++++++++++++++
.../apache/calcite/test/MongoAdapterTest.java | 726 ------------------
plus/pom.xml | 4 -
pom.xml | 38 +
9 files changed, 820 insertions(+), 762 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/1fd18a3e/avatica-server/pom.xml
----------------------------------------------------------------------
diff --git a/avatica-server/pom.xml b/avatica-server/pom.xml
index 794ab8e..c176079 100644
--- a/avatica-server/pom.xml
+++ b/avatica-server/pom.xml
@@ -94,14 +94,6 @@ limitations under the License.
<build>
<plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-surefire-plugin</artifactId>
- <configuration>
- <excludes />
- </configuration>
- </plugin>
-
<!-- Parent module has the same plugin and does the work of
generating -sources.jar for each project. But without the
plugin declared here, IDEs don't know the sources are
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/1fd18a3e/avatica/pom.xml
----------------------------------------------------------------------
diff --git a/avatica/pom.xml b/avatica/pom.xml
index ea779e2..603b025 100644
--- a/avatica/pom.xml
+++ b/avatica/pom.xml
@@ -57,14 +57,6 @@ limitations under the License.
<build>
<plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-surefire-plugin</artifactId>
- <configuration>
- <excludes />
- </configuration>
- </plugin>
-
<!-- Parent module has the same plugin and does the work of
generating -sources.jar for each project. But without the
plugin declared here, IDEs don't know the sources are
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/1fd18a3e/core/pom.xml
----------------------------------------------------------------------
diff --git a/core/pom.xml b/core/pom.xml
index 1b1bb46..479aca6 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -147,10 +147,6 @@ limitations under the License.
<includes>
<include>org/apache/calcite/test/CalciteSuite.java</include>
</includes>
- <threadCount>1</threadCount>
- <perCoreThreadCount>true</perCoreThreadCount>
- <parallel>both</parallel>
- <argLine>-Xmx1536m -XX:MaxPermSize=256m</argLine>
</configuration>
</plugin>
<plugin>
@@ -261,6 +257,39 @@ limitations under the License.
<profiles>
<profile>
+ <id>it</id>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-failsafe-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>failsafe-integration-test</id>
+ <phase/>
+ </execution>
+ <execution>
+ <id>failsafe-test-mysql</id>
+ <goals>
+ <goal>integration-test</goal>
+ </goals>
+ <phase>integration-test</phase>
+ <configuration>
+ <includes>
+ <include>org/apache/calcite/test/JdbcAdapterTest.java</include>
+ <include>org/apache/calcite/test/JdbcTest.java</include>
+ </includes>
+ <systemPropertyVariables>
+ <calcite.test.db>mysql</calcite.test.db>
+ </systemPropertyVariables>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+ <profile>
<!-- CALCITE-539: workaround for MSHARED-394: Avoid rewrite of
destination in DefaultMavenFileFilter#filterFile when producing
the same contents -->
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/1fd18a3e/doc/HOWTO.md
----------------------------------------------------------------------
diff --git a/doc/HOWTO.md b/doc/HOWTO.md
index 9086e99..ae7a167 100644
--- a/doc/HOWTO.md
+++ b/doc/HOWTO.md
@@ -70,6 +70,11 @@ environment, as follows.
* `-Dcalcite.test.splunk=true` enables tests that run against Splunk.
Splunk must be installed and running.
+To execute tests against mongodb, mysql only, use the following command:
+```bash
+mvn -Dtest=foo -DfailIfNoTests=false -Pit verify
+```
+
## Contributing
We welcome contributions.
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/1fd18a3e/mongodb/pom.xml
----------------------------------------------------------------------
diff --git a/mongodb/pom.xml b/mongodb/pom.xml
index 2758f59..409ae51 100644
--- a/mongodb/pom.xml
+++ b/mongodb/pom.xml
@@ -117,18 +117,6 @@ limitations under the License.
</execution>
</executions>
</plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-surefire-plugin</artifactId>
- <configuration>
- <includes>
- <include>org/apache/calcite/test/MongoAdapterTest.java</include>
- </includes>
- <threadCount>6</threadCount>
- <parallel>both</parallel>
- <argLine>-Xmx1024m</argLine>
- </configuration>
- </plugin>
</plugins>
</build>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/1fd18a3e/mongodb/src/test/java/org/apache/calcite/test/MongoAdapterIT.java
----------------------------------------------------------------------
diff --git a/mongodb/src/test/java/org/apache/calcite/test/MongoAdapterIT.java b/mongodb/src/test/java/org/apache/calcite/test/MongoAdapterIT.java
new file mode 100644
index 0000000..46cb2f1
--- /dev/null
+++ b/mongodb/src/test/java/org/apache/calcite/test/MongoAdapterIT.java
@@ -0,0 +1,744 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.test;
+
+import org.apache.calcite.linq4j.Ord;
+import org.apache.calcite.util.Bug;
+import org.apache.calcite.util.Pair;
+import org.apache.calcite.util.Util;
+
+import com.google.common.base.Function;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+
+import org.junit.Ignore;
+import org.junit.Test;
+
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.Collections;
+import java.util.List;
+
+import static org.hamcrest.CoreMatchers.equalTo;
+import static org.junit.Assert.assertThat;
+
+/**
+ * Tests for the {@code org.apache.calcite.adapter.mongodb} package.
+ *
+ * <p>Before calling this test, you need to populate MongoDB with the "zips"
+ * data set (as described in HOWTO.md)
+ * and "foodmart" data set, as follows:</p>
+ *
+ * <blockquote><code>
+ * git clone https://github.com/vlsi/test-dataset
+ * cd test-dataset
+ * mvn install
+ * </code></blockquote>
+ *
+ * This will create a virtual machine with MongoDB and test dataset.
+ */
+public class MongoAdapterIT {
+ public static final String MONGO_FOODMART_SCHEMA = " {\n"
+ + " type: 'custom',\n"
+ + " name: '_foodmart',\n"
+ + " factory: 'org.apache.calcite.adapter.mongodb.MongoSchemaFactory',\n"
+ + " operand: {\n"
+ + " host: 'localhost',\n"
+ + " database: 'foodmart'\n"
+ + " }\n"
+ + " },\n"
+ + " {\n"
+ + " name: 'foodmart',\n"
+ + " tables: [\n"
+ + " {\n"
+ + " name: 'sales_fact_1997',\n"
+ + " type: 'view',\n"
+ + " sql: 'select cast(_MAP[\\'product_id\\'] AS double) AS \"product_id\" from \"_foodmart\".\"sales_fact_1997\"'\n"
+ + " },\n"
+ + " {\n"
+ + " name: 'sales_fact_1998',\n"
+ + " type: 'view',\n"
+ + " sql: 'select cast(_MAP[\\'product_id\\'] AS double) AS \"product_id\" from \"_foodmart\".\"sales_fact_1998\"'\n"
+ + " },\n"
+ + " {\n"
+ + " name: 'store',\n"
+ + " type: 'view',\n"
+ + " sql: 'select cast(_MAP[\\'store_id\\'] AS double) AS \"store_id\", cast(_MAP[\\'store_name\\'] AS varchar(20)) AS \"store_name\" from \"_foodmart\".\"store\"'\n"
+ + " },\n"
+ + " {\n"
+ + " name: 'warehouse',\n"
+ + " type: 'view',\n"
+ + " sql: 'select cast(_MAP[\\'warehouse_id\\'] AS double) AS \"warehouse_id\", cast(_MAP[\\'warehouse_state_province\\'] AS varchar(20)) AS \"warehouse_state_province\" from \"_foodmart\".\"warehouse\"'\n"
+ + " }\n"
+ + " ]\n"
+ + " }\n";
+
+ public static final String MONGO_FOODMART_MODEL = "{\n"
+ + " version: '1.0',\n"
+ + " defaultSchema: 'foodmart',\n"
+ + " schemas: [\n"
+ + MONGO_FOODMART_SCHEMA
+ + " ]\n"
+ + "}";
+
+ /** Connection factory based on the "mongo-zips" model. */
+ public static final ImmutableMap<String, String> ZIPS =
+ ImmutableMap.of("model",
+ MongoAdapterIT.class.getResource("/mongo-zips-model.json")
+ .getPath());
+
+ /** Connection factory based on the "mongo-zips" model. */
+ public static final ImmutableMap<String, String> FOODMART =
+ ImmutableMap.of("model",
+ MongoAdapterIT.class.getResource("/mongo-foodmart-model.json")
+ .getPath());
+
+ /** Whether to run Mongo tests. Enabled by default, however test is only
+ * included if "it" profile is activated ({@code -Pit}). To disable,
+ * specify {@code -Dcalcite.test.mongodb=false} on the Java command line. */
+ public static final boolean ENABLED =
+ Boolean.valueOf(System.getProperty("calcite.test.mongodb", "true"));
+
+ /** Whether to run this test. */
+ protected boolean enabled() {
+ return ENABLED;
+ }
+
+ /** Returns a function that checks that a particular MongoDB pipeline is
+ * generated to implement a query. */
+ private static Function<List, Void> mongoChecker(final String... strings) {
+ return new Function<List, Void>() {
+ public Void apply(List actual) {
+ Object[] actualArray =
+ actual == null || actual.isEmpty()
+ ? null
+ : ((List) actual.get(0)).toArray();
+ CalciteAssert.assertArrayEqual("expected MongoDB query not found",
+ strings, actualArray);
+ return null;
+ }
+ };
+ }
+
+ /** Similar to {@link CalciteAssert#checkResultUnordered}, but filters strings
+ * before comparing them. */
+ static Function<ResultSet, Void> checkResultUnordered(
+ final String... lines) {
+ return new Function<ResultSet, Void>() {
+ public Void apply(ResultSet resultSet) {
+ try {
+ final List<String> expectedList = Lists.newArrayList(lines);
+ Collections.sort(expectedList);
+
+ final List<String> actualList = Lists.newArrayList();
+ CalciteAssert.toStringList(resultSet, actualList);
+ for (int i = 0; i < actualList.size(); i++) {
+ String s = actualList.get(i);
+ actualList.set(i,
+ s.replaceAll("\\.0;", ";").replaceAll("\\.0$", ""));
+ }
+ Collections.sort(actualList);
+
+ assertThat(actualList, equalTo(expectedList));
+ return null;
+ } catch (SQLException e) {
+ throw new RuntimeException(e);
+ }
+ }
+ };
+ }
+
+ @Test public void testSort() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select * from zips order by state")
+ .returnsCount(29353)
+ .explainContains("PLAN=MongoToEnumerableConverter\n"
+ + " MongoSort(sort0=[$4], dir0=[ASC])\n"
+ + " MongoProject(CITY=[CAST(ITEM($0, 'city')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"], LONGITUDE=[CAST(ITEM(ITEM($0, 'loc'), 0)):FLOAT], LATITUDE=[CAST(ITEM(ITEM($0, 'loc'), 1)):FLOAT], POP=[CAST(ITEM($0, 'pop')):INTEGER], STATE=[CAST(ITEM($0, 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"], ID=[CAST(ITEM($0, '_id')):VARCHAR(5) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"])\n"
+ + " MongoTableScan(table=[[mongo_raw, zips]])");
+ }
+
+ @Test public void testSortLimit() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select state, id from zips\n"
+ + "order by state, id offset 2 rows fetch next 3 rows only")
+ .returns("STATE=AK; ID=99503\n"
+ + "STATE=AK; ID=99504\n"
+ + "STATE=AK; ID=99505\n")
+ .queryContains(
+ mongoChecker(
+ "{$project: {STATE: '$state', ID: '$_id'}}",
+ "{$sort: {STATE: 1, ID: 1}}",
+ "{$skip: 2}",
+ "{$limit: 3}"));
+ }
+
+ @Test public void testOffsetLimit() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select state, id from zips\n"
+ + "offset 2 fetch next 3 rows only")
+ .runs()
+ .queryContains(
+ mongoChecker(
+ "{$skip: 2}",
+ "{$limit: 3}",
+ "{$project: {STATE: '$state', ID: '$_id'}}"));
+ }
+
+ @Test public void testLimit() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select state, id from zips\n"
+ + "fetch next 3 rows only")
+ .runs()
+ .queryContains(
+ mongoChecker(
+ "{$limit: 3}",
+ "{$project: {STATE: '$state', ID: '$_id'}}"));
+ }
+
+ @Ignore
+ @Test public void testFilterSort() {
+ // LONGITUDE and LATITUDE are null because of CALCITE-194.
+ Util.discard(Bug.CALCITE_194_FIXED);
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select * from zips\n"
+ + "where city = 'SPRINGFIELD' and id >= '70000'\n"
+ + "order by state, id")
+ .returns(""
+ + "CITY=SPRINGFIELD; LONGITUDE=null; LATITUDE=null; POP=752; STATE=AR; ID=72157\n"
+ + "CITY=SPRINGFIELD; LONGITUDE=null; LATITUDE=null; POP=1992; STATE=CO; ID=81073\n"
+ + "CITY=SPRINGFIELD; LONGITUDE=null; LATITUDE=null; POP=5597; STATE=LA; ID=70462\n"
+ + "CITY=SPRINGFIELD; LONGITUDE=null; LATITUDE=null; POP=32384; STATE=OR; ID=97477\n"
+ + "CITY=SPRINGFIELD; LONGITUDE=null; LATITUDE=null; POP=27521; STATE=OR; ID=97478\n")
+ .queryContains(
+ mongoChecker(
+ "{\n"
+ + " $match: {\n"
+ + " city: \"SPRINGFIELD\",\n"
+ + " _id: {\n"
+ + " $gte: \"70000\"\n"
+ + " }\n"
+ + " }\n"
+ + "}",
+ "{$project: {CITY: '$city', LONGITUDE: '$loc[0]', LATITUDE: '$loc[1]', POP: '$pop', STATE: '$state', ID: '$_id'}}",
+ "{$sort: {STATE: 1, ID: 1}}"))
+ .explainContains("PLAN=MongoToEnumerableConverter\n"
+ + " MongoSort(sort0=[$4], sort1=[$5], dir0=[ASC], dir1=[ASC])\n"
+ + " MongoProject(CITY=[CAST(ITEM($0, 'city')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"], LONGITUDE=[CAST(ITEM(ITEM($0, 'loc'), 0)):FLOAT], LATITUDE=[CAST(ITEM(ITEM($0, 'loc'), 1)):FLOAT], POP=[CAST(ITEM($0, 'pop')):INTEGER], STATE=[CAST(ITEM($0, 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"], ID=[CAST(ITEM($0, '_id')):VARCHAR(5) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"])\n"
+ + " MongoFilter(condition=[AND(=(CAST(ITEM($0, 'city')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\", 'SPRINGFIELD'), >=(CAST(ITEM($0, '_id')):VARCHAR(5) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\", '70000'))])\n"
+ + " MongoTableScan(table=[[mongo_raw, zips]])");
+ }
+
+ @Test public void testFilterSortDesc() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select * from zips\n"
+ + "where pop BETWEEN 20000 AND 20100\n"
+ + "order by state desc, pop")
+ .limit(4)
+ .returns(""
+ + "CITY=SHERIDAN; LONGITUDE=null; LATITUDE=null; POP=20025; STATE=WY; ID=82801\n"
+ + "CITY=MOUNTLAKE TERRAC; LONGITUDE=null; LATITUDE=null; POP=20059; STATE=WA; ID=98043\n"
+ + "CITY=FALMOUTH; LONGITUDE=null; LATITUDE=null; POP=20039; STATE=VA; ID=22405\n"
+ + "CITY=FORT WORTH; LONGITUDE=null; LATITUDE=null; POP=20012; STATE=TX; ID=76104\n");
+ }
+
+ @Test public void testUnionPlan() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .withModel(MONGO_FOODMART_MODEL)
+ .query("select * from \"sales_fact_1997\"\n"
+ + "union all\n"
+ + "select * from \"sales_fact_1998\"")
+ .explainContains("PLAN=EnumerableUnion(all=[true])\n"
+ + " MongoToEnumerableConverter\n"
+ + " MongoProject(product_id=[CAST(ITEM($0, 'product_id')):DOUBLE])\n"
+ + " MongoTableScan(table=[[_foodmart, sales_fact_1997]])\n"
+ + " MongoToEnumerableConverter\n"
+ + " MongoProject(product_id=[CAST(ITEM($0, 'product_id')):DOUBLE])\n"
+ + " MongoTableScan(table=[[_foodmart, sales_fact_1998]])")
+ .limit(2)
+ .returns(
+ checkResultUnordered(
+ "product_id=337", "product_id=1512"));
+ }
+
+ @Ignore(
+ "java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Double")
+ @Test public void testFilterUnionPlan() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .withModel(MONGO_FOODMART_MODEL)
+ .query("select * from (\n"
+ + " select * from \"sales_fact_1997\"\n"
+ + " union all\n"
+ + " select * from \"sales_fact_1998\")\n"
+ + "where \"product_id\" = 1")
+ .runs();
+ }
+
+ /** Tests that we don't generate multiple constraints on the same column.
+ * MongoDB doesn't like it. If there is an '=', it supersedes all other
+ * operators. */
+ @Test public void testFilterRedundant() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query(
+ "select * from zips where state > 'CA' and state < 'AZ' and state = 'OK'")
+ .runs()
+ .queryContains(
+ mongoChecker(
+ "{\n"
+ + " $match: {\n"
+ + " state: \"OK\"\n"
+ + " }\n"
+ + "}",
+ "{$project: {CITY: '$city', LONGITUDE: '$loc[0]', LATITUDE: '$loc[1]', POP: '$pop', STATE: '$state', ID: '$_id'}}"));
+ }
+
+ @Test public void testSelectWhere() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .withModel(MONGO_FOODMART_MODEL)
+ .query(
+ "select * from \"warehouse\" where \"warehouse_state_province\" = 'CA'")
+ .explainContains("PLAN=MongoToEnumerableConverter\n"
+ + " MongoProject(warehouse_id=[CAST(ITEM($0, 'warehouse_id')):DOUBLE], warehouse_state_province=[CAST(ITEM($0, 'warehouse_state_province')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"])\n"
+ + " MongoFilter(condition=[=(CAST(ITEM($0, 'warehouse_state_province')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\", 'CA')])\n"
+ + " MongoTableScan(table=[[_foodmart, warehouse]])")
+ .returns(
+ checkResultUnordered(
+ "warehouse_id=6; warehouse_state_province=CA",
+ "warehouse_id=7; warehouse_state_province=CA",
+ "warehouse_id=14; warehouse_state_province=CA",
+ "warehouse_id=24; warehouse_state_province=CA"))
+ .queryContains(
+ // Per https://issues.apache.org/jira/browse/CALCITE-164,
+ // $match must occur before $project for good performance.
+ mongoChecker(
+ "{\n"
+ + " $match: {\n"
+ + " warehouse_state_province: \"CA\"\n"
+ + " }\n"
+ + "}",
+ "{$project: {warehouse_id: 1, warehouse_state_province: 1}}"));
+ }
+
+ @Test public void testInPlan() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .withModel(MONGO_FOODMART_MODEL)
+ .query("select \"store_id\", \"store_name\" from \"store\"\n"
+ + "where \"store_name\" in ('Store 1', 'Store 10', 'Store 11', 'Store 15', 'Store 16', 'Store 24', 'Store 3', 'Store 7')")
+ .returns(
+ checkResultUnordered(
+ "store_id=1; store_name=Store 1",
+ "store_id=3; store_name=Store 3",
+ "store_id=7; store_name=Store 7",
+ "store_id=10; store_name=Store 10",
+ "store_id=11; store_name=Store 11",
+ "store_id=15; store_name=Store 15",
+ "store_id=16; store_name=Store 16",
+ "store_id=24; store_name=Store 24"))
+ .queryContains(
+ mongoChecker(
+ "{\n"
+ + " $match: {\n"
+ + " $or: [\n"
+ + " {\n"
+ + " store_name: \"Store 1\"\n"
+ + " },\n"
+ + " {\n"
+ + " store_name: \"Store 10\"\n"
+ + " },\n"
+ + " {\n"
+ + " store_name: \"Store 11\"\n"
+ + " },\n"
+ + " {\n"
+ + " store_name: \"Store 15\"\n"
+ + " },\n"
+ + " {\n"
+ + " store_name: \"Store 16\"\n"
+ + " },\n"
+ + " {\n"
+ + " store_name: \"Store 24\"\n"
+ + " },\n"
+ + " {\n"
+ + " store_name: \"Store 3\"\n"
+ + " },\n"
+ + " {\n"
+ + " store_name: \"Store 7\"\n"
+ + " }\n"
+ + " ]\n"
+ + " }\n"
+ + "}",
+ "{$project: {store_id: 1, store_name: 1}}"));
+ }
+
+ /** Simple query based on the "mongo-zips" model. */
+ @Test public void testZips() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select state, city from zips")
+ .returnsCount(29353);
+ }
+
+ @Test public void testCountGroupByEmpty() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select count(*) from zips")
+ .returns("EXPR$0=29353\n")
+ .explainContains("PLAN=MongoToEnumerableConverter\n"
+ + " MongoAggregate(group=[{}], EXPR$0=[COUNT()])\n"
+ + " MongoTableScan(table=[[mongo_raw, zips]])")
+ .queryContains(
+ mongoChecker(
+ "{$group: {_id: {}, 'EXPR$0': {$sum: 1}}}"));
+ }
+
+ @Ignore("Returns 2 instead of 2*29353 == 58706")
+ @Test public void testCountGroupByEmptyMultiplyBy2() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select count(*)*2 from zips")
+ .returns("EXPR$0=58706\n")
+ .queryContains(
+ mongoChecker(
+ "{$group: {_id: {}, 'EXPR$0': {$sum: 1}}}"));
+ }
+
+ @Test public void testGroupByOneColumnNotProjected() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select count(*) from zips group by state order by 1")
+ .limit(2)
+ .returns("EXPR$0=24\n"
+ + "EXPR$0=53\n")
+ .queryContains(
+ mongoChecker(
+ "{$project: {STATE: '$state'}}",
+ "{$group: {_id: '$STATE', 'EXPR$0': {$sum: 1}}}",
+ "{$project: {STATE: '$_id', 'EXPR$0': '$EXPR$0'}}",
+ "{$project: {'EXPR$0': 1}}",
+ "{$sort: {EXPR$0: 1}}"));
+ }
+
+ @Test public void testGroupByOneColumn() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query(
+ "select state, count(*) as c from zips group by state order by state")
+ .limit(2)
+ .returns("STATE=AK; C=195\n"
+ + "STATE=AL; C=567\n")
+ .queryContains(
+ mongoChecker(
+ "{$project: {STATE: '$state'}}",
+ "{$group: {_id: '$STATE', C: {$sum: 1}}}",
+ "{$project: {STATE: '$_id', C: '$C'}}",
+ "{$sort: {STATE: 1}}"));
+ }
+
+ @Test public void testGroupByOneColumnReversed() {
+ // Note extra $project compared to testGroupByOneColumn.
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query(
+ "select count(*) as c, state from zips group by state order by state")
+ .limit(2)
+ .returns("C=195; STATE=AK\n"
+ + "C=567; STATE=AL\n")
+ .queryContains(
+ mongoChecker(
+ "{$project: {STATE: '$state'}}",
+ "{$group: {_id: '$STATE', C: {$sum: 1}}}",
+ "{$project: {STATE: '$_id', C: '$C'}}",
+ "{$project: {C: 1, STATE: 1}}",
+ "{$sort: {STATE: 1}}"));
+ }
+
+ @Test public void testGroupByHaving() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select state, count(*) as c from zips\n"
+ + "group by state having count(*) > 1500 order by state")
+ .returns("STATE=CA; C=1516\n"
+ + "STATE=NY; C=1595\n"
+ + "STATE=TX; C=1671\n")
+ .queryContains(
+ mongoChecker(
+ "{$project: {STATE: '$state'}}",
+ "{$group: {_id: '$STATE', C: {$sum: 1}}}",
+ "{$project: {STATE: '$_id', C: '$C'}}",
+ "{\n"
+ + " $match: {\n"
+ + " C: {\n"
+ + " $gt: 1500\n"
+ + " }\n"
+ + " }\n"
+ + "}",
+ "{$sort: {STATE: 1}}"));
+ }
+
+ @Ignore("https://issues.apache.org/jira/browse/CALCITE-270")
+ @Test public void testGroupByHaving2() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select state, count(*) as c from zips\n"
+ + "group by state having sum(pop) > 12000000")
+ .returns("STATE=NY; C=1596\n"
+ + "STATE=TX; C=1676\n"
+ + "STATE=FL; C=826\n"
+ + "STATE=CA; C=1523\n")
+ .queryContains(
+ mongoChecker(
+ "{$project: {STATE: '$state', POP: '$pop'}}",
+ "{$group: {_id: '$STATE', C: {$sum: 1}, _2: {$sum: '$POP'}}}",
+ "{$project: {STATE: '$_id', C: '$C', _2: '$_2'}}",
+ "{\n"
+ + " $match: {\n"
+ + " _2: {\n"
+ + " $gt: 12000000\n"
+ + " }\n"
+ + " }\n"
+ + "}",
+ "{$project: {STATE: 1, C: 1}}"));
+ }
+
+ @Test public void testGroupByMinMaxSum() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select count(*) as c, state,\n"
+ + " min(pop) as min_pop, max(pop) as max_pop, sum(pop) as sum_pop\n"
+ + "from zips group by state order by state")
+ .limit(2)
+ .returns("C=195; STATE=AK; MIN_POP=0; MAX_POP=32383; SUM_POP=544698\n"
+ + "C=567; STATE=AL; MIN_POP=0; MAX_POP=44165; SUM_POP=4040587\n")
+ .queryContains(
+ mongoChecker(
+ "{$project: {POP: '$pop', STATE: '$state'}}",
+ "{$group: {_id: '$STATE', C: {$sum: 1}, MIN_POP: {$min: '$POP'}, MAX_POP: {$max: '$POP'}, SUM_POP: {$sum: '$POP'}}}",
+ "{$project: {STATE: '$_id', C: '$C', MIN_POP: '$MIN_POP', MAX_POP: '$MAX_POP', SUM_POP: '$SUM_POP'}}",
+ "{$project: {C: 1, STATE: 1, MIN_POP: 1, MAX_POP: 1, SUM_POP: 1}}",
+ "{$sort: {STATE: 1}}"));
+ }
+
+ @Test public void testGroupComposite() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select count(*) as c, state, city from zips\n"
+ + "group by state, city order by c desc limit 2")
+ .returns("C=93; STATE=TX; CITY=HOUSTON\n"
+ + "C=56; STATE=CA; CITY=LOS ANGELES\n")
+ .queryContains(
+ mongoChecker(
+ "{$project: {CITY: '$city', STATE: '$state'}}",
+ "{$group: {_id: {CITY: '$CITY', STATE: '$STATE'}, C: {$sum: 1}}}",
+ "{$project: {_id: 0, CITY: '$_id.CITY', STATE: '$_id.STATE', C: '$C'}}",
+ "{$sort: {C: -1}}",
+ "{$limit: 2}",
+ "{$project: {C: 1, STATE: 1, CITY: 1}}"));
+ }
+
+ @Test public void testDistinctCount() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select state, count(distinct city) as cdc from zips\n"
+ + "where state in ('CA', 'TX') group by state order by state")
+ .returns("STATE=CA; CDC=1072\n"
+ + "STATE=TX; CDC=1233\n")
+ .queryContains(
+ mongoChecker(
+ "{\n"
+ + " $match: {\n"
+ + " $or: [\n"
+ + " {\n"
+ + " state: \"CA\"\n"
+ + " },\n"
+ + " {\n"
+ + " state: \"TX\"\n"
+ + " }\n"
+ + " ]\n"
+ + " }\n"
+ + "}",
+ "{$project: {CITY: '$city', STATE: '$state'}}",
+ "{$group: {_id: {CITY: '$CITY', STATE: '$STATE'}}}",
+ "{$project: {_id: 0, CITY: '$_id.CITY', STATE: '$_id.STATE'}}",
+ "{$group: {_id: '$STATE', CDC: {$sum: {$cond: [ {$eq: ['CITY', null]}, 0, 1]}}}}",
+ "{$project: {STATE: '$_id', CDC: '$CDC'}}",
+ "{$sort: {STATE: 1}}"));
+ }
+
+ @Test public void testDistinctCountOrderBy() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select state, count(distinct city) as cdc\n"
+ + "from zips\n"
+ + "group by state\n"
+ + "order by cdc desc limit 5")
+ .returns("STATE=NY; CDC=1370\n"
+ + "STATE=PA; CDC=1369\n"
+ + "STATE=TX; CDC=1233\n"
+ + "STATE=IL; CDC=1148\n"
+ + "STATE=CA; CDC=1072\n")
+ .queryContains(
+ mongoChecker(
+ "{$project: {STATE: '$state', CITY: '$city'}}",
+ "{$group: {_id: {STATE: '$STATE', CITY: '$CITY'}}}",
+ "{$project: {_id: 0, STATE: '$_id.STATE', CITY: '$_id.CITY'}}",
+ "{$group: {_id: '$STATE', CDC: {$sum: {$cond: [ {$eq: ['CITY', null]}, 0, 1]}}}}",
+ "{$project: {STATE: '$_id', CDC: '$CDC'}}",
+ "{$sort: {CDC: -1}}",
+ "{$limit: 5}"));
+ }
+
+ @Test public void testProject() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select state, city, 0 as zero from zips order by state, city")
+ .limit(2)
+ .returns("STATE=AK; CITY=AKHIOK; ZERO=0\n"
+ + "STATE=AK; CITY=AKIACHAK; ZERO=0\n")
+ .queryContains(
+ mongoChecker(
+ "{$project: {CITY: '$city', STATE: '$state'}}",
+ "{$sort: {STATE: 1, CITY: 1}}",
+ "{$project: {STATE: 1, CITY: 1, ZERO: {$ifNull: [null, 0]}}}"));
+ }
+
+ @Test public void testFilter() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select state, city from zips where state = 'CA'")
+ .limit(2)
+ .returns("STATE=CA; CITY=LOS ANGELES\n"
+ + "STATE=CA; CITY=LOS ANGELES\n")
+ .explainContains("PLAN=MongoToEnumerableConverter\n"
+ + " MongoProject(STATE=[CAST(ITEM($0, 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"], CITY=[CAST(ITEM($0, 'city')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"])\n"
+ + " MongoFilter(condition=[=(CAST(ITEM($0, 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\", 'CA')])\n"
+ + " MongoTableScan(table=[[mongo_raw, zips]])");
+ }
+
+ /** MongoDB's predicates are handed (they can only accept literals on the
+ * right-hand size) so it's worth testing that we handle them right both
+ * ways around. */
+ @Test public void testFilterReversed() {
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select state, city from zips where 'WI' < state")
+ .limit(2)
+ .returns("STATE=WV; CITY=BLUEWELL\n"
+ + "STATE=WV; CITY=ATHENS\n");
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(ZIPS)
+ .query("select state, city from zips where state > 'WI'")
+ .limit(2)
+ .returns("STATE=WV; CITY=BLUEWELL\n"
+ + "STATE=WV; CITY=ATHENS\n");
+ }
+
+ @Ignore
+ @Test public void testFoodmartQueries() {
+ final List<Pair<String, String>> queries = JdbcTest.getFoodmartQueries();
+ for (Ord<Pair<String, String>> query : Ord.zip(queries)) {
+// if (query.i != 29) continue;
+ if (query.e.left.contains("agg_")) {
+ continue;
+ }
+ final CalciteAssert.AssertQuery query1 =
+ CalciteAssert.that()
+ .enable(enabled())
+ .with(FOODMART)
+ .query(query.e.left);
+ if (query.e.right != null) {
+ query1.returns(query.e.right);
+ } else {
+ query1.runs();
+ }
+ }
+ }
+
+ /** Test case for
+ * <a href="https://issues.apache.org/jira/browse/CALCITE-286">CALCITE-286</a>,
+ * "Error casting MongoDB date". */
+ @Test public void testDate() {
+ // Assumes that you have created the following collection before running
+ // this test:
+ //
+ // $ mongo
+ // > use test
+ // switched to db test
+ // > db.createCollection("datatypes")
+ // { "ok" : 1 }
+ // > db.datatypes.insert( {
+ // "_id" : ObjectId("53655599e4b0c980df0a8c27"),
+ // "_class" : "com.ericblue.Test",
+ // "date" : ISODate("2012-09-05T07:00:00Z"),
+ // "value" : 1231,
+ // "ownerId" : "531e7789e4b0853ddb861313"
+ // } )
+ CalciteAssert.that()
+ .enable(enabled())
+ .withModel("{\n"
+ + " version: '1.0',\n"
+ + " defaultSchema: 'test',\n"
+ + " schemas: [\n"
+ + " {\n"
+ + " type: 'custom',\n"
+ + " name: 'test',\n"
+ + " factory: 'org.apache.calcite.adapter.mongodb.MongoSchemaFactory',\n"
+ + " operand: {\n"
+ + " host: 'localhost',\n"
+ + " database: 'test'\n"
+ + " }\n"
+ + " }\n"
+ + " ]\n"
+ + "}")
+ .query("select cast(_MAP['date'] as DATE) from \"datatypes\"")
+ .returnsUnordered("EXPR$0=2012-09-05");
+ }
+}
+
+// End MongoAdapterIT.java
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/1fd18a3e/mongodb/src/test/java/org/apache/calcite/test/MongoAdapterTest.java
----------------------------------------------------------------------
diff --git a/mongodb/src/test/java/org/apache/calcite/test/MongoAdapterTest.java b/mongodb/src/test/java/org/apache/calcite/test/MongoAdapterTest.java
deleted file mode 100644
index bc9a9dc..0000000
--- a/mongodb/src/test/java/org/apache/calcite/test/MongoAdapterTest.java
+++ /dev/null
@@ -1,726 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to you under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.calcite.test;
-
-import org.apache.calcite.linq4j.Ord;
-import org.apache.calcite.util.Bug;
-import org.apache.calcite.util.Pair;
-import org.apache.calcite.util.Util;
-
-import com.google.common.base.Function;
-import com.google.common.collect.ImmutableList;
-import com.google.common.collect.ImmutableMap;
-import com.google.common.collect.Lists;
-
-import org.junit.Assert;
-import org.junit.Ignore;
-import org.junit.Test;
-
-import java.sql.ResultSet;
-import java.sql.SQLException;
-import java.util.Collections;
-import java.util.List;
-
-import static org.hamcrest.CoreMatchers.equalTo;
-import static org.junit.Assert.assertThat;
-
-/**
- * Tests for the {@code org.apache.calcite.adapter.mongodb} package.
- *
- * <p>Before calling this test, you need to populate MongoDB with the "zips"
- * data set (as described in HOWTO.md)
- * and "foodmart" data set, as follows:</p>
- *
- * <blockquote><code>
- * JAR=~/.m2/repository/pentaho/mondrian-data-foodmart-json/
- * 0.3/mondrian-data-foodmart-json-0.3.jar<br>
- * mkdir /tmp/foodmart<br>
- * cd /tmp/foodmart<br>
- * jar xvf $JAR<br>
- * for i in *.json; do<br>
- * mongoimport --db foodmart --collection ${i/.json/} --file $i<br>
- * done<br>
- * </code></blockquote>
- */
-public class MongoAdapterTest {
- public static final String MONGO_FOODMART_SCHEMA = " {\n"
- + " type: 'custom',\n"
- + " name: '_foodmart',\n"
- + " factory: 'org.apache.calcite.adapter.mongodb.MongoSchemaFactory',\n"
- + " operand: {\n"
- + " host: 'localhost',\n"
- + " database: 'foodmart'\n"
- + " }\n"
- + " },\n"
- + " {\n"
- + " name: 'foodmart',\n"
- + " tables: [\n"
- + " {\n"
- + " name: 'sales_fact_1997',\n"
- + " type: 'view',\n"
- + " sql: 'select cast(_MAP[\\'product_id\\'] AS double) AS \"product_id\" from \"_foodmart\".\"sales_fact_1997\"'\n"
- + " },\n"
- + " {\n"
- + " name: 'sales_fact_1998',\n"
- + " type: 'view',\n"
- + " sql: 'select cast(_MAP[\\'product_id\\'] AS double) AS \"product_id\" from \"_foodmart\".\"sales_fact_1998\"'\n"
- + " },\n"
- + " {\n"
- + " name: 'store',\n"
- + " type: 'view',\n"
- + " sql: 'select cast(_MAP[\\'store_id\\'] AS double) AS \"store_id\", cast(_MAP[\\'store_name\\'] AS varchar(20)) AS \"store_name\" from \"_foodmart\".\"store\"'\n"
- + " },\n"
- + " {\n"
- + " name: 'warehouse',\n"
- + " type: 'view',\n"
- + " sql: 'select cast(_MAP[\\'warehouse_id\\'] AS double) AS \"warehouse_id\", cast(_MAP[\\'warehouse_state_province\\'] AS varchar(20)) AS \"warehouse_state_province\" from \"_foodmart\".\"warehouse\"'\n"
- + " }\n"
- + " ]\n"
- + " }\n";
-
- public static final String MONGO_FOODMART_MODEL = "{\n"
- + " version: '1.0',\n"
- + " defaultSchema: 'foodmart',\n"
- + " schemas: [\n"
- + MONGO_FOODMART_SCHEMA
- + " ]\n"
- + "}";
-
- /** Connection factory based on the "mongo-zips" model. */
- public static final ImmutableMap<String, String> ZIPS =
- ImmutableMap.of("model",
- MongoAdapterTest.class.getResource("/mongo-zips-model.json")
- .getPath());
-
- /** Connection factory based on the "mongo-zips" model. */
- public static final ImmutableMap<String, String> FOODMART =
- ImmutableMap.of("model",
- MongoAdapterTest.class.getResource("/mongo-foodmart-model.json")
- .getPath());
-
- /** Whether to run Mongo tests. Disabled by default, because we do not expect
- * Mongo to be installed and populated with the FoodMart data set. To enable,
- * specify {@code -Dcalcite.test.mongodb=true} on the Java command line. */
- public static final boolean ENABLED =
- Boolean.getBoolean("calcite.test.mongodb");
-
- /** Whether to run this test. */
- protected boolean enabled() {
- return ENABLED;
- }
-
- /** Returns a function that checks that a particular MongoDB pipeline is
- * generated to implement a query. */
- private static Function<List, Void> mongoChecker(final String... strings) {
- return new Function<List, Void>() {
- public Void apply(List actual) {
- if (!actual.contains(ImmutableList.copyOf(strings))) {
- Assert.fail("expected MongoDB query not found; actual: " + actual);
- }
- return null;
- }
- };
- }
-
- /** Similar to {@link CalciteAssert#checkResultUnordered}, but filters strings
- * before comparing them. */
- static Function<ResultSet, Void> checkResultUnordered(
- final String... lines) {
- return new Function<ResultSet, Void>() {
- public Void apply(ResultSet resultSet) {
- try {
- final List<String> expectedList = Lists.newArrayList(lines);
- Collections.sort(expectedList);
-
- final List<String> actualList = Lists.newArrayList();
- CalciteAssert.toStringList(resultSet, actualList);
- for (int i = 0; i < actualList.size(); i++) {
- String s = actualList.get(i);
- actualList.set(i,
- s.replaceAll("\\.0;", ";").replaceAll("\\.0$", ""));
- }
- Collections.sort(actualList);
-
- assertThat(actualList, equalTo(expectedList));
- return null;
- } catch (SQLException e) {
- throw new RuntimeException(e);
- }
- }
- };
- }
-
- @Test public void testSort() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select * from zips order by state")
- .returnsCount(29467)
- .explainContains("PLAN=MongoToEnumerableConverter\n"
- + " MongoSort(sort0=[$4], dir0=[ASC])\n"
- + " MongoProject(CITY=[CAST(ITEM($0, 'city')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"], LONGITUDE=[CAST(ITEM(ITEM($0, 'loc'), 0)):FLOAT NOT NULL], LATITUDE=[CAST(ITEM(ITEM($0, 'loc'), 1)):FLOAT NOT NULL], POP=[CAST(ITEM($0, 'pop')):INTEGER], STATE=[CAST(ITEM($0, 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"], ID=[CAST(ITEM($0, '_id')):VARCHAR(5) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"])\n"
- + " MongoTableScan(table=[[mongo_raw, zips]])");
- }
-
- @Test public void testSortLimit() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, id from zips\n"
- + "order by state, id offset 2 rows fetch next 3 rows only")
- .returns("STATE=AK; ID=99502\n"
- + "STATE=AK; ID=99503\n"
- + "STATE=AK; ID=99504\n")
- .queryContains(
- mongoChecker(
- "{$project: {STATE: '$state', ID: '$_id'}}",
- "{$sort: {STATE: 1, ID: 1}}",
- "{$skip: 2}",
- "{$limit: 3}"));
- }
-
- @Test public void testOffsetLimit() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, id from zips\n"
- + "offset 2 fetch next 3 rows only")
- .runs()
- .queryContains(
- mongoChecker(
- "{$skip: 2}",
- "{$limit: 3}",
- "{$project: {STATE: '$state', ID: '$_id'}}"));
- }
-
- @Test public void testLimit() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, id from zips\n"
- + "fetch next 3 rows only")
- .runs()
- .queryContains(
- mongoChecker(
- "{$limit: 3}",
- "{$project: {STATE: '$state', ID: '$_id'}}"));
- }
-
- @Test public void testFilterSort() {
- // LONGITUDE and LATITUDE are null because of CALCITE-194.
- Util.discard(Bug.CALCITE_194_FIXED);
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select * from zips\n"
- + "where city = 'SPRINGFIELD' and id between '20000' and '30000'\n"
- + "order by state")
- .returns(""
- + "CITY=SPRINGFIELD; LONGITUDE=null; LATITUDE=null; POP=2184; STATE=SC; ID=29146\n"
- + "CITY=SPRINGFIELD; LONGITUDE=null; LATITUDE=null; POP=16811; STATE=VA; ID=22150\n"
- + "CITY=SPRINGFIELD; LONGITUDE=null; LATITUDE=null; POP=32161; STATE=VA; ID=22153\n"
- + "CITY=SPRINGFIELD; LONGITUDE=null; LATITUDE=null; POP=1321; STATE=WV; ID=26763\n")
- .queryContains(
- mongoChecker(
- "{\n"
- + " $match: {\n"
- + " city: \"SPRINGFIELD\",\n"
- + " _id: {\n"
- + " $lte: \"30000\",\n"
- + " $gte: \"20000\"\n"
- + " }\n"
- + " }\n"
- + "}",
- "{$project: {CITY: '$city', LONGITUDE: '$loc[0]', LATITUDE: '$loc[1]', POP: '$pop', STATE: '$state', ID: '$_id'}}",
- "{$sort: {STATE: 1}}"))
- .explainContains("PLAN=MongoToEnumerableConverter\n"
- + " MongoSort(sort0=[$4], dir0=[ASC])\n"
- + " MongoProject(CITY=[CAST(ITEM($0, 'city')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"], LONGITUDE=[CAST(ITEM(ITEM($0, 'loc'), 0)):FLOAT NOT NULL], LATITUDE=[CAST(ITEM(ITEM($0, 'loc'), 1)):FLOAT NOT NULL], POP=[CAST(ITEM($0, 'pop')):INTEGER], STATE=[CAST(ITEM($0, 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"], ID=[CAST(ITEM($0, '_id')):VARCHAR(5) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"])\n"
- + " MongoFilter(condition=[AND(=(CAST(ITEM($0, 'city')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\", 'SPRINGFIELD'), >=(CAST(ITEM($0, '_id')):VARCHAR(5) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\", '20000'), <=(CAST(ITEM($0, '_id')):VARCHAR(5) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\", '30000'))])\n"
- + " MongoTableScan(table=[[mongo_raw, zips]])");
- }
-
- @Test public void testFilterSortDesc() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select * from zips\n"
- + "where pop BETWEEN 20000 AND 20100\n"
- + "order by state desc, pop")
- .limit(4)
- .returns(""
- + "CITY=SHERIDAN; LONGITUDE=null; LATITUDE=null; POP=20025; STATE=WY; ID=82801\n"
- + "CITY=MOUNTLAKE TERRAC; LONGITUDE=null; LATITUDE=null; POP=20059; STATE=WA; ID=98043\n"
- + "CITY=FALMOUTH; LONGITUDE=null; LATITUDE=null; POP=20039; STATE=VA; ID=22405\n"
- + "CITY=FORT WORTH; LONGITUDE=null; LATITUDE=null; POP=20012; STATE=TX; ID=76104\n");
- }
-
- @Test public void testUnionPlan() {
- CalciteAssert.that()
- .enable(enabled())
- .withModel(MONGO_FOODMART_MODEL)
- .query("select * from \"sales_fact_1997\"\n"
- + "union all\n"
- + "select * from \"sales_fact_1998\"")
- .explainContains("PLAN=EnumerableUnionRel(all=[true])\n"
- + " MongoToEnumerableConverter\n"
- + " MongoProject(product_id=[CAST(ITEM($0, 'product_id')):DOUBLE])\n"
- + " MongoTableScan(table=[[_foodmart, sales_fact_1997]])\n"
- + " MongoToEnumerableConverter\n"
- + " MongoProject(product_id=[CAST(ITEM($0, 'product_id')):DOUBLE])\n"
- + " MongoTableScan(table=[[_foodmart, sales_fact_1998]])\n")
- .limit(2)
- .returns(
- checkResultUnordered(
- "product_id=337", "product_id=1512"));
- }
-
- @Test public void testFilterUnionPlan() {
- CalciteAssert.that()
- .enable(enabled())
- .withModel(MONGO_FOODMART_MODEL)
- .query("select * from (\n"
- + " select * from \"sales_fact_1997\"\n"
- + " union all\n"
- + " select * from \"sales_fact_1998\")\n"
- + "where \"product_id\" = 1")
- .runs();
- }
-
- /** Tests that we don't generate multiple constraints on the same column.
- * MongoDB doesn't like it. If there is an '=', it supersedes all other
- * operators. */
- @Test public void testFilterRedundant() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query(
- "select * from zips where state > 'CA' and state < 'AZ' and state = 'OK'")
- .runs()
- .queryContains(
- mongoChecker(
- "{\n"
- + " $match: {\n"
- + " state: \"OK\"\n"
- + " }\n"
- + "}",
- "{$project: {CITY: '$city', LONGITUDE: '$loc[0]', LATITUDE: '$loc[1]', POP: '$pop', STATE: '$state', ID: '$_id'}}"));
- }
-
- @Test public void testSelectWhere() {
- CalciteAssert.that()
- .enable(enabled())
- .withModel(MONGO_FOODMART_MODEL)
- .query(
- "select * from \"warehouse\" where \"warehouse_state_province\" = 'CA'")
- .explainContains("PLAN=MongoToEnumerableConverter\n"
- + " MongoProject(warehouse_id=[CAST(ITEM($0, 'warehouse_id')):DOUBLE], warehouse_state_province=[CAST(ITEM($0, 'warehouse_state_province')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"])\n"
- + " MongoFilter(condition=[=(CAST(ITEM($0, 'warehouse_state_province')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\", 'CA')])\n"
- + " MongoTableScan(table=[[_foodmart, warehouse]])")
- .returns(
- checkResultUnordered(
- "warehouse_id=6; warehouse_state_province=CA",
- "warehouse_id=7; warehouse_state_province=CA",
- "warehouse_id=14; warehouse_state_province=CA",
- "warehouse_id=24; warehouse_state_province=CA"))
- .queryContains(
- // Per https://issues.apache.org/jira/browse/CALCITE-164,
- // $match must occur before $project for good performance.
- mongoChecker(
- "{\n"
- + " $match: {\n"
- + " warehouse_state_province: \"CA\"\n"
- + " }\n"
- + "}",
- "{$project: {warehouse_id: 1, warehouse_state_province: 1}}"));
- }
-
- @Test public void testInPlan() {
- CalciteAssert.that()
- .enable(enabled())
- .withModel(MONGO_FOODMART_MODEL)
- .query("select \"store_id\", \"store_name\" from \"store\"\n"
- + "where \"store_name\" in ('Store 1', 'Store 10', 'Store 11', 'Store 15', 'Store 16', 'Store 24', 'Store 3', 'Store 7')")
- .returns(
- checkResultUnordered(
- "store_id=1; store_name=Store 1",
- "store_id=3; store_name=Store 3",
- "store_id=7; store_name=Store 7",
- "store_id=10; store_name=Store 10",
- "store_id=11; store_name=Store 11",
- "store_id=15; store_name=Store 15",
- "store_id=16; store_name=Store 16",
- "store_id=24; store_name=Store 24"))
- .queryContains(
- mongoChecker(
- "{\n"
- + " $match: {\n"
- + " $or: [\n"
- + " {\n"
- + " store_name: \"Store 1\"\n"
- + " },\n"
- + " {\n"
- + " store_name: \"Store 10\"\n"
- + " },\n"
- + " {\n"
- + " store_name: \"Store 11\"\n"
- + " },\n"
- + " {\n"
- + " store_name: \"Store 15\"\n"
- + " },\n"
- + " {\n"
- + " store_name: \"Store 16\"\n"
- + " },\n"
- + " {\n"
- + " store_name: \"Store 24\"\n"
- + " },\n"
- + " {\n"
- + " store_name: \"Store 3\"\n"
- + " },\n"
- + " {\n"
- + " store_name: \"Store 7\"\n"
- + " }\n"
- + " ]\n"
- + " }\n"
- + "}",
- "{$project: {store_id: 1, store_name: 1}}"));
- }
-
- /** Simple query based on the "mongo-zips" model. */
- @Test public void testZips() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, city from zips")
- .returnsCount(29467);
- }
-
- @Test public void testCountGroupByEmpty() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select count(*) from zips")
- .returns("EXPR$0=29467\n")
- .explainContains("PLAN=MongoToEnumerableConverter\n"
- + " MongoAggregate(group=[{}], EXPR$0=[COUNT()])\n"
- + " MongoProject(DUMMY=[0])\n"
- + " MongoTableScan(table=[[mongo_raw, zips]])")
- .queryContains(
- mongoChecker(
- "{$project: {DUMMY: {$ifNull: [null, 0]}}}",
- "{$group: {_id: {}, 'EXPR$0': {$sum: 1}}}"));
- }
-
- @Test public void testGroupByOneColumnNotProjected() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select count(*) from zips group by state")
- .limit(2)
- .returns("EXPR$0=659\n"
- + "EXPR$0=484\n")
- .queryContains(
- mongoChecker(
- "{$project: {STATE: '$state'}}",
- "{$group: {_id: '$STATE', 'EXPR$0': {$sum: 1}}}",
- "{$project: {STATE: '$_id', 'EXPR$0': '$EXPR$0'}}",
- "{$project: {'EXPR$0': 1}}"));
- }
-
- @Test public void testGroupByOneColumn() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, count(*) as c from zips group by state")
- .limit(2)
- .returns("STATE=WV; C=659\n"
- + "STATE=WA; C=484\n")
- .queryContains(
- mongoChecker(
- "{$project: {STATE: '$state'}}",
- "{$group: {_id: '$STATE', C: {$sum: 1}}}",
- "{$project: {STATE: '$_id', C: '$C'}}"));
- }
-
- @Test public void testGroupByOneColumnReversed() {
- // Note extra $project compared to testGroupByOneColumn.
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select count(*) as c, state from zips group by state")
- .limit(2)
- .returns("C=659; STATE=WV\n"
- + "C=484; STATE=WA\n")
- .queryContains(
- mongoChecker(
- "{$project: {STATE: '$state'}}",
- "{$group: {_id: '$STATE', C: {$sum: 1}}}",
- "{$project: {STATE: '$_id', C: '$C'}}",
- "{$project: {C: 1, STATE: 1}}"));
- }
-
- @Test public void testGroupByHaving() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, count(*) as c from zips\n"
- + "group by state having count(*) > 1500")
- .returns("STATE=NY; C=1596\n"
- + "STATE=TX; C=1676\n"
- + "STATE=CA; C=1523\n")
- .queryContains(
- mongoChecker(
- "{$project: {STATE: '$state'}}",
- "{$group: {_id: '$STATE', C: {$sum: 1}}}",
- "{$project: {STATE: '$_id', C: '$C'}}",
- "{\n"
- + " $match: {\n"
- + " C: {\n"
- + " $gt: 1500\n"
- + " }\n"
- + " }\n"
- + "}"));
- }
-
- @Ignore("https://issues.apache.org/jira/browse/CALCITE-270")
- @Test public void testGroupByHaving2() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, count(*) as c from zips\n"
- + "group by state having sum(pop) > 12000000")
- .returns("STATE=NY; C=1596\n"
- + "STATE=TX; C=1676\n"
- + "STATE=FL; C=826\n"
- + "STATE=CA; C=1523\n")
- .queryContains(
- mongoChecker(
- "{$project: {STATE: '$state', POP: '$pop'}}",
- "{$group: {_id: '$STATE', C: {$sum: 1}, _2: {$sum: '$POP'}}}",
- "{$project: {STATE: '$_id', C: '$C', _2: '$_2'}}",
- "{\n"
- + " $match: {\n"
- + " _2: {\n"
- + " $gt: 12000000\n"
- + " }\n"
- + " }\n"
- + "}",
- "{$project: {STATE: 1, C: 1}}"));
- }
-
- @Test public void testGroupByMinMaxSum() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select count(*) as c, state,\n"
- + " min(pop) as min_pop, max(pop) as max_pop, sum(pop) as sum_pop\n"
- + "from zips group by state")
- .limit(2)
- .returns("C=195; STATE=AK; MIN_POP=0; MAX_POP=32383; SUM_POP=544698\n"
- + "C=567; STATE=AL; MIN_POP=0; MAX_POP=44165; SUM_POP=4040587\n")
- .queryContains(
- mongoChecker(
- "{$project: {STATE: '$state', POP: '$pop'}}",
- "{$group: {_id: '$STATE', C: {$sum: 1}, MIN_POP: {$min: '$POP'}, MAX_POP: {$max: '$POP'}, SUM_POP: {$sum: '$POP'}}}",
- "{$project: {STATE: '$_id', C: '$C', MIN_POP: '$MIN_POP', MAX_POP: '$MAX_POP', SUM_POP: '$SUM_POP'}}",
- "{$project: {C: 1, STATE: 1, MIN_POP: 1, MAX_POP: 1, SUM_POP: 1}}",
- "{$sort: {STATE: 1}}"));
- }
-
- @Test public void testGroupComposite() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select count(*) as c, state, city from zips\n"
- + "group by state, city order by c desc limit 2")
- .returns("C=93; STATE=TX; CITY=HOUSTON\n"
- + "C=56; STATE=CA; CITY=LOS ANGELES\n")
- .queryContains(
- mongoChecker(
- "{$project: {STATE: '$state', CITY: '$city'}}",
- "{$group: {_id: {STATE: '$STATE', CITY: '$CITY'}, C: {$sum: 1}}}",
- "{$project: {_id: 0, STATE: '$_id.STATE', CITY: '$_id.CITY', C: '$C'}}",
- "{$sort: {C: -1}}",
- "{$limit: 2}",
- "{$project: {C: 1, STATE: 1, CITY: 1}}"));
- }
-
- @Test public void testDistinctCount() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, count(distinct city) as cdc from zips\n"
- + "where state in ('CA', 'TX') group by state")
- .returns("STATE=CA; CDC=1079\n"
- + "STATE=TX; CDC=1238\n")
- .queryContains(
- mongoChecker(
- "{\n"
- + " $match: {\n"
- + " $or: [\n"
- + " {\n"
- + " state: \"CA\"\n"
- + " },\n"
- + " {\n"
- + " state: \"TX\"\n"
- + " }\n"
- + " ]\n"
- + " }\n"
- + "}",
- "{$project: {STATE: '$state', CITY: '$city'}}",
- "{$group: {_id: {STATE: '$STATE', CITY: '$CITY'}}}",
- "{$project: {_id: 0, STATE: '$_id.STATE', CITY: '$_id.CITY'}}",
- "{$group: {_id: '$STATE', CDC: {$sum: {$cond: [ {$eq: ['CITY', null]}, 0, 1]}}}}",
- "{$project: {STATE: '$_id', CDC: '$CDC'}}"));
- }
-
- @Test public void testDistinctCountOrderBy() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, count(distinct city) as cdc\n"
- + "from zips\n"
- + "group by state\n"
- + "order by cdc desc limit 5")
- .returns("STATE=NY; CDC=1371\n"
- + "STATE=PA; CDC=1369\n"
- + "STATE=TX; CDC=1238\n"
- + "STATE=IL; CDC=1151\n"
- + "STATE=CA; CDC=1079\n")
- .queryContains(
- mongoChecker(
- "{$project: {STATE: '$state', CITY: '$city'}}",
- "{$group: {_id: {STATE: '$STATE', CITY: '$CITY'}}}",
- "{$project: {_id: 0, STATE: '$_id.STATE', CITY: '$_id.CITY'}}",
- "{$group: {_id: '$STATE', CDC: {$sum: {$cond: [ {$eq: ['CITY', null]}, 0, 1]}}}}",
- "{$project: {STATE: '$_id', CDC: '$CDC'}}",
- "{$sort: {CDC: -1}}",
- "{$limit: 5}"));
- }
-
- @Test public void testProject() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, city, 0 as zero from zips")
- .limit(2)
- .returns("STATE=AL; CITY=ACMAR; ZERO=0\n"
- + "STATE=AL; CITY=ADAMSVILLE; ZERO=0\n")
- .queryContains(
- mongoChecker(
- "{$project: {CITY: '$city', STATE: '$state'}}",
- "{$sort: {STATE: 1, CITY: 1}}",
- "{$project: {STATE: 1, CITY: 1, ZERO: {$ifNull: [null, 0]}}}"));
- }
-
- @Test public void testFilter() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, city from zips where state = 'CA'")
- .limit(2)
- .returns("STATE=CA; CITY=LOS ANGELES\n"
- + "STATE=CA; CITY=LOS ANGELES\n")
- .explainContains("PLAN=MongoToEnumerableConverter\n"
- + " MongoProject(STATE=[CAST(ITEM($0, 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"], CITY=[CAST(ITEM($0, 'city')):VARCHAR(20) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\"])\n"
- + " MongoFilter(condition=[=(CAST(ITEM($0, 'state')):VARCHAR(2) CHARACTER SET \"ISO-8859-1\" COLLATE \"ISO-8859-1$en_US$primary\", 'CA')])\n"
- + " MongoTableScan(table=[[mongo_raw, zips]])");
- }
-
- /** MongoDB's predicates are handed (they can only accept literals on the
- * right-hand size) so it's worth testing that we handle them right both
- * ways around. */
- @Test public void testFilterReversed() {
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, city from zips where 'WI' < state")
- .limit(2)
- .returns("STATE=WV; CITY=BLUEWELL\n"
- + "STATE=WV; CITY=ATHENS\n");
- CalciteAssert.that()
- .enable(enabled())
- .with(ZIPS)
- .query("select state, city from zips where state > 'WI'")
- .limit(2)
- .returns("STATE=WV; CITY=BLUEWELL\n"
- + "STATE=WV; CITY=ATHENS\n");
- }
-
- @Ignore
- @Test public void testFoodmartQueries() {
- final List<Pair<String, String>> queries = JdbcTest.getFoodmartQueries();
- for (Ord<Pair<String, String>> query : Ord.zip(queries)) {
-// if (query.i != 29) continue;
- if (query.e.left.contains("agg_")) {
- continue;
- }
- final CalciteAssert.AssertQuery query1 =
- CalciteAssert.that()
- .enable(enabled())
- .with(FOODMART)
- .query(query.e.left);
- if (query.e.right != null) {
- query1.returns(query.e.right);
- } else {
- query1.runs();
- }
- }
- }
-
- /** Test case for
- * <a href="https://issues.apache.org/jira/browse/CALCITE-286">CALCITE-286</a>,
- * "Error casting MongoDB date". */
- @Test public void testDate() {
- // Assumes that you have created the following collection before running
- // this test:
- //
- // $ mongo
- // > use test
- // switched to db test
- // > db.createCollection("datatypes")
- // { "ok" : 1 }
- // > db.datatypes.insert( {
- // "_id" : ObjectId("53655599e4b0c980df0a8c27"),
- // "_class" : "com.ericblue.Test",
- // "date" : ISODate("2012-09-05T07:00:00Z"),
- // "value" : 1231,
- // "ownerId" : "531e7789e4b0853ddb861313"
- // } )
- CalciteAssert.that()
- .enable(enabled())
- .withModel("{\n"
- + " version: '1.0',\n"
- + " defaultSchema: 'test',\n"
- + " schemas: [\n"
- + " {\n"
- + " type: 'custom',\n"
- + " name: 'test',\n"
- + " factory: 'org.apache.calcite.adapter.mongodb.MongoSchemaFactory',\n"
- + " operand: {\n"
- + " host: 'localhost',\n"
- + " database: 'test'\n"
- + " }\n"
- + " }\n"
- + " ]\n"
- + "}")
- .query("select cast(_MAP['date'] as DATE) from \"datatypes\"")
- .returnsUnordered("EXPR$0=2012-09-05");
- }
-}
-
-// End MongoAdapterTest.java
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/1fd18a3e/plus/pom.xml
----------------------------------------------------------------------
diff --git a/plus/pom.xml b/plus/pom.xml
index a06489f..51d08e7 100644
--- a/plus/pom.xml
+++ b/plus/pom.xml
@@ -132,10 +132,6 @@ limitations under the License.
<includes>
<include>org/apache/calcite/test/PlusSuite.java</include>
</includes>
- <threadCount>1</threadCount>
- <perCoreThreadCount>true</perCoreThreadCount>
- <parallel>both</parallel>
- <argLine>-Xmx2g</argLine>
</configuration>
</plugin>
</plugins>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/1fd18a3e/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 6572ebb..1b9f832 100644
--- a/pom.xml
+++ b/pom.xml
@@ -442,6 +442,27 @@ limitations under the License.
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18</version>
+ <executions>
+ <execution>
+ <id>failsafe-integration-test</id>
+ <goals>
+ <goal>integration-test</goal>
+ </goals>
+ <phase>integration-test</phase>
+ <configuration>
+ <threadCount>6</threadCount>
+ <parallel>both</parallel>
+ <argLine>-Xmx1024m</argLine>
+ </configuration>
+ </execution>
+ <execution>
+ <id>failsafe-verify</id>
+ <goals>
+ <goal>verify</goal>
+ </goals>
+ <phase>verify</phase>
+ </execution>
+ </executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
@@ -458,6 +479,12 @@ limitations under the License.
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18</version>
+ <configuration>
+ <threadCount>1</threadCount>
+ <perCoreThreadCount>true</perCoreThreadCount>
+ <parallel>both</parallel>
+ <argLine>-Xmx1536m -XX:MaxPermSize=256m</argLine>
+ </configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
@@ -609,6 +636,17 @@ limitations under the License.
</build>
</profile>
<profile>
+ <id>it</id>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-failsafe-plugin</artifactId>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+ <profile>
<!-- CALCITE-537: workaround for MRRESOURCES-91
Avoid overwrite of the destination file if the produced
contents is the same.
[7/8] incubator-calcite git commit: [CALCITE-686] SqlNode.unparse
produces invalid SQL
Posted by jh...@apache.org.
[CALCITE-686] SqlNode.unparse produces invalid SQL
Enable SqlUnParserTest and RelOptPlanReaderTest
Project: http://git-wip-us.apache.org/repos/asf/incubator-calcite/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-calcite/commit/057f847b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-calcite/tree/057f847b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-calcite/diff/057f847b
Branch: refs/heads/master
Commit: 057f847bf6a918ee767f597be2705d70145cabac
Parents: 01183de
Author: Julian Hyde <jh...@apache.org>
Authored: Thu Apr 16 14:50:06 2015 -0700
Committer: Julian Hyde <jh...@apache.org>
Committed: Fri Apr 17 00:19:58 2015 -0700
----------------------------------------------------------------------
.../apache/calcite/sql/SqlBinaryOperator.java | 7 +-
.../apache/calcite/sql/SqlInfixOperator.java | 4 -
.../apache/calcite/sql/SqlInternalOperator.java | 4 +-
.../java/org/apache/calcite/sql/SqlKind.java | 2 +-
.../org/apache/calcite/sql/SqlOperator.java | 10 ++
.../apache/calcite/sql/SqlSpecialOperator.java | 4 +-
.../java/org/apache/calcite/sql/SqlUpdate.java | 3 -
.../java/org/apache/calcite/sql/SqlUtil.java | 11 +-
.../java/org/apache/calcite/sql/SqlWith.java | 5 +-
.../org/apache/calcite/sql/SqlWithItem.java | 2 +-
.../calcite/sql/fun/SqlExtendOperator.java | 63 ++++++++
.../calcite/sql/fun/SqlRollupOperator.java | 65 ++++++++
.../calcite/sql/fun/SqlStdOperatorTable.java | 12 +-
.../calcite/plan/RelOptPlanReaderTest.java | 4 +-
.../calcite/sql/parser/SqlParserTest.java | 157 +++++++++++--------
.../org/apache/calcite/test/CalciteSuite.java | 4 +
16 files changed, 255 insertions(+), 102 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/SqlBinaryOperator.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/SqlBinaryOperator.java b/core/src/main/java/org/apache/calcite/sql/SqlBinaryOperator.java
index 0c0b491..bdcf961 100644
--- a/core/src/main/java/org/apache/calcite/sql/SqlBinaryOperator.java
+++ b/core/src/main/java/org/apache/calcite/sql/SqlBinaryOperator.java
@@ -81,8 +81,7 @@ public class SqlBinaryOperator extends SqlOperator {
}
/**
- * Returns whether this operator should be surrounded by space when
- * unparsed.
+ * {@inheritDoc}
*
* <p>Returns true for most operators but false for the '.' operator;
* consider
@@ -90,10 +89,8 @@ public class SqlBinaryOperator extends SqlOperator {
* <blockquote>
* <pre>x.y + 5 * 6</pre>
* </blockquote>
- *
- * @return whether this operator should be surrounded by space
*/
- boolean needsSpace() {
+ @Override boolean needsSpace() {
return !getName().equals(".");
}
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/SqlInfixOperator.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/SqlInfixOperator.java b/core/src/main/java/org/apache/calcite/sql/SqlInfixOperator.java
index 2f26878..df5d176 100644
--- a/core/src/main/java/org/apache/calcite/sql/SqlInfixOperator.java
+++ b/core/src/main/java/org/apache/calcite/sql/SqlInfixOperator.java
@@ -76,10 +76,6 @@ public class SqlInfixOperator extends SqlSpecialOperator {
operand.e.unparse(writer, leftPrec, getLeftPrec());
}
}
-
- boolean needsSpace() {
- return true;
- }
}
// End SqlInfixOperator.java
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/SqlInternalOperator.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/SqlInternalOperator.java b/core/src/main/java/org/apache/calcite/sql/SqlInternalOperator.java
index deba96b..6f1e33c 100644
--- a/core/src/main/java/org/apache/calcite/sql/SqlInternalOperator.java
+++ b/core/src/main/java/org/apache/calcite/sql/SqlInternalOperator.java
@@ -44,14 +44,14 @@ public class SqlInternalOperator extends SqlSpecialOperator {
public SqlInternalOperator(
String name,
SqlKind kind) {
- super(name, kind, 2, true, ReturnTypes.ARG0, null, OperandTypes.VARIADIC);
+ this(name, kind, 2);
}
public SqlInternalOperator(
String name,
SqlKind kind,
int prec) {
- super(name, kind, prec, true, ReturnTypes.ARG0, null, null);
+ this(name, kind, prec, true, ReturnTypes.ARG0, null, OperandTypes.VARIADIC);
}
public SqlInternalOperator(
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/SqlKind.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/SqlKind.java b/core/src/main/java/org/apache/calcite/sql/SqlKind.java
index 5fa715e..3843231 100644
--- a/core/src/main/java/org/apache/calcite/sql/SqlKind.java
+++ b/core/src/main/java/org/apache/calcite/sql/SqlKind.java
@@ -652,7 +652,7 @@ public enum SqlKind {
SELECT, JOIN, OTHER_FUNCTION, CAST, TRIM, FLOOR, CEIL,
LITERAL_CHAIN, JDBC_FN, PRECEDING, FOLLOWING, ORDER_BY,
NULLS_FIRST, NULLS_LAST, COLLECTION_TABLE, TABLESAMPLE,
- WITH, WITH_ITEM));
+ VALUES, WITH, WITH_ITEM));
/**
* Category consisting of all DML operators.
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/SqlOperator.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/SqlOperator.java b/core/src/main/java/org/apache/calcite/sql/SqlOperator.java
index 08325ab..e8914b2 100644
--- a/core/src/main/java/org/apache/calcite/sql/SqlOperator.java
+++ b/core/src/main/java/org/apache/calcite/sql/SqlOperator.java
@@ -493,6 +493,16 @@ public abstract class SqlOperator {
}
/**
+ * Returns whether this operator should be surrounded by space when
+ * unparsed.
+ *
+ * @return whether this operator should be surrounded by space
+ */
+ boolean needsSpace() {
+ return true;
+ }
+
+ /**
* Validates and determines coercibility and resulting collation name of
* binary operator if needed.
*/
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/SqlSpecialOperator.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/SqlSpecialOperator.java b/core/src/main/java/org/apache/calcite/sql/SqlSpecialOperator.java
index 22b748d..85da998 100644
--- a/core/src/main/java/org/apache/calcite/sql/SqlSpecialOperator.java
+++ b/core/src/main/java/org/apache/calcite/sql/SqlSpecialOperator.java
@@ -32,14 +32,14 @@ public class SqlSpecialOperator extends SqlOperator {
public SqlSpecialOperator(
String name,
SqlKind kind) {
- super(name, kind, 2, true, null, null, null);
+ this(name, kind, 2);
}
public SqlSpecialOperator(
String name,
SqlKind kind,
int prec) {
- super(name, kind, prec, true, null, null, null);
+ this(name, kind, prec, true, null, null, null);
}
public SqlSpecialOperator(
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/SqlUpdate.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/SqlUpdate.java b/core/src/main/java/org/apache/calcite/sql/SqlUpdate.java
index 23afb22..5d8ebfa 100644
--- a/core/src/main/java/org/apache/calcite/sql/SqlUpdate.java
+++ b/core/src/main/java/org/apache/calcite/sql/SqlUpdate.java
@@ -162,9 +162,6 @@ public class SqlUpdate extends SqlCall {
final int opLeft = getOperator().getLeftPrec();
final int opRight = getOperator().getRightPrec();
targetTable.unparse(writer, opLeft, opRight);
- if (targetColumnList != null) {
- targetColumnList.unparse(writer, opLeft, opRight);
- }
if (alias != null) {
writer.keyword("AS");
alias.unparse(writer, opLeft, opRight);
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/SqlUtil.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/SqlUtil.java b/core/src/main/java/org/apache/calcite/sql/SqlUtil.java
index 4ee7ece..405b385 100644
--- a/core/src/main/java/org/apache/calcite/sql/SqlUtil.java
+++ b/core/src/main/java/org/apache/calcite/sql/SqlUtil.java
@@ -272,19 +272,18 @@ public abstract class SqlUtil {
SqlWriter writer,
int leftPrec,
int rightPrec) {
- SqlBinaryOperator binop = (SqlBinaryOperator) operator;
assert call.operandCount() == 2;
final SqlWriter.Frame frame =
writer.startList(
- (binop instanceof SqlSetOperator)
+ (operator instanceof SqlSetOperator)
? SqlWriter.FrameTypeEnum.SETOP
: SqlWriter.FrameTypeEnum.SIMPLE);
- call.operand(0).unparse(writer, leftPrec, binop.getLeftPrec());
- final boolean needsSpace = binop.needsSpace();
+ call.operand(0).unparse(writer, leftPrec, operator.getLeftPrec());
+ final boolean needsSpace = operator.needsSpace();
writer.setNeedWhitespace(needsSpace);
- writer.sep(binop.getName());
+ writer.sep(operator.getName());
writer.setNeedWhitespace(needsSpace);
- call.operand(1).unparse(writer, binop.getRightPrec(), rightPrec);
+ call.operand(1).unparse(writer, operator.getRightPrec(), rightPrec);
writer.endList(frame);
}
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/SqlWith.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/SqlWith.java b/core/src/main/java/org/apache/calcite/sql/SqlWith.java
index f0cd2c3..e680fe8 100644
--- a/core/src/main/java/org/apache/calcite/sql/SqlWith.java
+++ b/core/src/main/java/org/apache/calcite/sql/SqlWith.java
@@ -99,7 +99,10 @@ public class SqlWith extends SqlCall {
node.unparse(writer, 0, 0);
}
writer.endList(frame1);
- with.body.unparse(writer, 0, 0);
+ final SqlWriter.Frame frame2 =
+ writer.startList(SqlWriter.FrameTypeEnum.SIMPLE);
+ with.body.unparse(writer, 100, 100);
+ writer.endList(frame2);
writer.endList(frame);
}
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/SqlWithItem.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/SqlWithItem.java b/core/src/main/java/org/apache/calcite/sql/SqlWithItem.java
index 7fe5c62..63692c7 100644
--- a/core/src/main/java/org/apache/calcite/sql/SqlWithItem.java
+++ b/core/src/main/java/org/apache/calcite/sql/SqlWithItem.java
@@ -93,7 +93,7 @@ public class SqlWithItem extends SqlCall {
withItem.columnList.unparse(writer, getLeftPrec(), getRightPrec());
}
writer.keyword("AS");
- withItem.query.unparse(writer, getLeftPrec(), getRightPrec());
+ withItem.query.unparse(writer, 10, 10);
}
@Override public SqlCall createCall(SqlLiteral functionQualifier,
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/fun/SqlExtendOperator.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/fun/SqlExtendOperator.java b/core/src/main/java/org/apache/calcite/sql/fun/SqlExtendOperator.java
new file mode 100644
index 0000000..6e0cfb7
--- /dev/null
+++ b/core/src/main/java/org/apache/calcite/sql/fun/SqlExtendOperator.java
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import org.apache.calcite.linq4j.Ord;
+import org.apache.calcite.sql.SqlCall;
+import org.apache.calcite.sql.SqlInternalOperator;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.SqlNode;
+import org.apache.calcite.sql.SqlNodeList;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.SqlWriter;
+
+/** {@code EXTEND} operator.
+ *
+ * <p>Adds columns to a table's schema, as in
+ * {@code SELECT ... FROM emp EXTEND (horoscope VARCHAR(100))}.
+ *
+ * <p>Not standard SQL. Added to Calcite to support Phoenix, but can be used to
+ * achieve schema-on-query against other adapters.
+ */
+class SqlExtendOperator extends SqlInternalOperator {
+ public SqlExtendOperator() {
+ super("EXTEND", SqlKind.EXTEND, MDX_PRECEDENCE);
+ }
+
+ @Override public void unparse(SqlWriter writer, SqlCall call, int leftPrec,
+ int rightPrec) {
+ final SqlOperator operator = call.getOperator();
+ assert call.operandCount() == 2;
+ final SqlWriter.Frame frame =
+ writer.startList(SqlWriter.FrameTypeEnum.SIMPLE);
+ call.operand(0).unparse(writer, leftPrec, operator.getLeftPrec());
+ writer.setNeedWhitespace(true);
+ writer.sep(operator.getName());
+ final SqlNodeList list = call.operand(1);
+ final SqlWriter.Frame frame2 = writer.startList("(", ")");
+ for (Ord<SqlNode> node2 : Ord.zip(list)) {
+ if (node2.i > 0 && node2.i % 2 == 0) {
+ writer.sep(",");
+ }
+ node2.e.unparse(writer, 2, 3);
+ }
+ writer.endList(frame2);
+ writer.endList(frame);
+ }
+}
+
+// End SqlExtendOperator.java
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/fun/SqlRollupOperator.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/fun/SqlRollupOperator.java b/core/src/main/java/org/apache/calcite/sql/fun/SqlRollupOperator.java
new file mode 100644
index 0000000..6e3efc8
--- /dev/null
+++ b/core/src/main/java/org/apache/calcite/sql/fun/SqlRollupOperator.java
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql.fun;
+
+import org.apache.calcite.sql.SqlCall;
+import org.apache.calcite.sql.SqlInternalOperator;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.SqlNode;
+import org.apache.calcite.sql.SqlNodeList;
+import org.apache.calcite.sql.SqlWriter;
+
+/**
+ * Operator that appears in a {@code GROUP BY} clause: {@code CUBE},
+ * {@code ROLLUP}, {@code GROUPING SETS}.
+ */
+class SqlRollupOperator extends SqlInternalOperator {
+ public SqlRollupOperator(String name, SqlKind kind) {
+ super(name, kind, 4);
+ }
+
+ @Override public void unparse(SqlWriter writer, SqlCall call, int leftPrec,
+ int rightPrec) {
+ unparseCube(writer, call);
+ }
+
+ private static void unparseCube(SqlWriter writer, SqlCall call) {
+ writer.keyword(call.getOperator().getName());
+ final SqlWriter.Frame frame =
+ writer.startList(SqlWriter.FrameTypeEnum.FUN_CALL, "(", ")");
+ for (SqlNode operand : call.getOperandList()) {
+ writer.sep(",");
+ if (operand.getKind() == SqlKind.ROW) {
+ final SqlWriter.Frame frame2 =
+ writer.startList(SqlWriter.FrameTypeEnum.SIMPLE, "(", ")");
+ for (SqlNode operand2 : ((SqlCall) operand).getOperandList()) {
+ writer.sep(",");
+ operand2.unparse(writer, 0, 0);
+ }
+ writer.endList(frame2);
+ } else if (operand instanceof SqlNodeList
+ && ((SqlNodeList) operand).size() == 0) {
+ writer.keyword("()");
+ } else {
+ operand.unparse(writer, 0, 0);
+ }
+ }
+ writer.endList(frame);
+ }
+}
+
+// End SqlRollupOperator.java
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java b/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java
index ef7133b..0a86799 100644
--- a/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java
+++ b/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java
@@ -152,17 +152,17 @@ public class SqlStdOperatorTable extends ReflectiveSqlOperatorTable {
/** {@code CUBE} operator, occurs within {@code GROUP BY} clause
* or nested within a {@code GROUPING SETS}. */
public static final SqlInternalOperator CUBE =
- new SqlInternalOperator("CUBE", SqlKind.CUBE);
+ new SqlRollupOperator("CUBE", SqlKind.CUBE);
/** {@code ROLLUP} operator, occurs within {@code GROUP BY} clause
* or nested within a {@code GROUPING SETS}. */
public static final SqlInternalOperator ROLLUP =
- new SqlInternalOperator("ROLLUP", SqlKind.ROLLUP);
+ new SqlRollupOperator("ROLLUP", SqlKind.ROLLUP);
/** {@code GROUPING SETS} operator, occurs within {@code GROUP BY} clause
* or nested within a {@code GROUPING SETS}. */
public static final SqlInternalOperator GROUPING_SETS =
- new SqlInternalOperator("GROUPING_SETS", SqlKind.GROUPING_SETS);
+ new SqlRollupOperator("GROUPING SETS", SqlKind.GROUPING_SETS);
/** {@code GROUPING} function. Occurs in similar places to an aggregate
* function ({@code SELECT}, {@code HAVING} clause, etc. of an aggregate
@@ -178,10 +178,8 @@ public class SqlStdOperatorTable extends ReflectiveSqlOperatorTable {
public static final SqlGroupingIdFunction GROUPING_ID =
new SqlGroupingIdFunction();
- /** {@code EXTEND} operator to add columns to a table's schema, as in
- * {@code SELECT ... FROM emp EXTEND (horoscope VARCHAR(100))}. */
- public static final SqlInternalOperator EXTEND =
- new SqlInternalOperator("EXTEND", SqlKind.EXTEND);
+ /** {@code EXTEND} operator. */
+ public static final SqlInternalOperator EXTEND = new SqlExtendOperator();
/**
* String concatenation operator, '<code>||</code>'.
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/test/java/org/apache/calcite/plan/RelOptPlanReaderTest.java
----------------------------------------------------------------------
diff --git a/core/src/test/java/org/apache/calcite/plan/RelOptPlanReaderTest.java b/core/src/test/java/org/apache/calcite/plan/RelOptPlanReaderTest.java
index 8c95301..777cf5b 100644
--- a/core/src/test/java/org/apache/calcite/plan/RelOptPlanReaderTest.java
+++ b/core/src/test/java/org/apache/calcite/plan/RelOptPlanReaderTest.java
@@ -44,8 +44,8 @@ public class RelOptPlanReaderTest {
// in org.apache.calcite.adapter.jdbc.JdbcRules outer class
assertThat(relJson.classToTypeName(JdbcRules.JdbcProject.class),
- is("JdbcProjectRel"));
- assertThat(relJson.typeNameToClass("JdbcProjectRel"),
+ is("JdbcProject"));
+ assertThat(relJson.typeNameToClass("JdbcProject"),
equalTo((Class) JdbcRules.JdbcProject.class));
try {
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/test/java/org/apache/calcite/sql/parser/SqlParserTest.java
----------------------------------------------------------------------
diff --git a/core/src/test/java/org/apache/calcite/sql/parser/SqlParserTest.java b/core/src/test/java/org/apache/calcite/sql/parser/SqlParserTest.java
index 9b32900..11eaf7c 100644
--- a/core/src/test/java/org/apache/calcite/sql/parser/SqlParserTest.java
+++ b/core/src/test/java/org/apache/calcite/sql/parser/SqlParserTest.java
@@ -354,7 +354,7 @@ public class SqlParserTest {
check(
"values a between c and d and e and f between g and h",
- "(VALUES (ROW((((`A` BETWEEN ASYMMETRIC `C` AND `D`) AND `E`) AND (`F` BETWEEN ASYMMETRIC `G` AND `H`)))))");
+ "VALUES (ROW((((`A` BETWEEN ASYMMETRIC `C` AND `D`) AND `E`) AND (`F` BETWEEN ASYMMETRIC `G` AND `H`))))");
checkFails(
"values a between b or c^",
@@ -371,17 +371,17 @@ public class SqlParserTest {
// precedence of BETWEEN is higher than AND and OR, but lower than '+'
check(
"values a between b and c + 2 or d and e",
- "(VALUES (ROW(((`A` BETWEEN ASYMMETRIC `B` AND (`C` + 2)) OR (`D` AND `E`)))))");
+ "VALUES (ROW(((`A` BETWEEN ASYMMETRIC `B` AND (`C` + 2)) OR (`D` AND `E`))))");
// '=' and BETWEEN have same precedence, and are left-assoc
check(
"values x = a between b and c = d = e",
- "(VALUES (ROW(((((`X` = `A`) BETWEEN ASYMMETRIC `B` AND `C`) = `D`) = `E`))))");
+ "VALUES (ROW(((((`X` = `A`) BETWEEN ASYMMETRIC `B` AND `C`) = `D`) = `E`)))");
// AND doesn't match BETWEEN if it's between parentheses!
check(
"values a between b or (c and d) or e and f",
- "(VALUES (ROW((`A` BETWEEN ASYMMETRIC ((`B` OR (`C` AND `D`)) OR `E`) AND `F`))))");
+ "VALUES (ROW((`A` BETWEEN ASYMMETRIC ((`B` OR (`C` AND `D`)) OR `E`) AND `F`)))");
}
@Test public void testOperateOnColumn() {
@@ -549,33 +549,33 @@ public class SqlParserTest {
check(
"values a and b like c",
- "(VALUES (ROW((`A` AND (`B` LIKE `C`)))))");
+ "VALUES (ROW((`A` AND (`B` LIKE `C`))))");
// LIKE has higher precedence than AND
check(
"values a and b like c escape d and e",
- "(VALUES (ROW(((`A` AND (`B` LIKE `C` ESCAPE `D`)) AND `E`))))");
+ "VALUES (ROW(((`A` AND (`B` LIKE `C` ESCAPE `D`)) AND `E`)))");
// LIKE has same precedence as '='; LIKE is right-assoc, '=' is left
check(
"values a = b like c = d",
- "(VALUES (ROW(((`A` = `B`) LIKE (`C` = `D`)))))");
+ "VALUES (ROW(((`A` = `B`) LIKE (`C` = `D`))))");
// Nested LIKE
check(
"values a like b like c escape d",
- "(VALUES (ROW((`A` LIKE (`B` LIKE `C` ESCAPE `D`)))))");
+ "VALUES (ROW((`A` LIKE (`B` LIKE `C` ESCAPE `D`))))");
check(
"values a like b like c escape d and false",
- "(VALUES (ROW(((`A` LIKE (`B` LIKE `C` ESCAPE `D`)) AND FALSE))))");
+ "VALUES (ROW(((`A` LIKE (`B` LIKE `C` ESCAPE `D`)) AND FALSE)))");
check(
"values a like b like c like d escape e escape f",
- "(VALUES (ROW((`A` LIKE (`B` LIKE (`C` LIKE `D` ESCAPE `E`) ESCAPE `F`)))))");
+ "VALUES (ROW((`A` LIKE (`B` LIKE (`C` LIKE `D` ESCAPE `E`) ESCAPE `F`))))");
// Mixed LIKE and SIMILAR TO
check(
"values a similar to b like c similar to d escape e escape f",
- "(VALUES (ROW((`A` SIMILAR TO (`B` LIKE (`C` SIMILAR TO `D` ESCAPE `E`) ESCAPE `F`)))))");
+ "VALUES (ROW((`A` SIMILAR TO (`B` LIKE (`C` SIMILAR TO `D` ESCAPE `E`) ESCAPE `F`))))");
// FIXME should fail at "escape"
checkFails(
@@ -585,12 +585,12 @@ public class SqlParserTest {
// LIKE with +
check(
"values a like b + c escape d",
- "(VALUES (ROW((`A` LIKE (`B` + `C`) ESCAPE `D`))))");
+ "VALUES (ROW((`A` LIKE (`B` + `C`) ESCAPE `D`)))");
// LIKE with ||
check(
"values a like b || c escape d",
- "(VALUES (ROW((`A` LIKE (`B` || `C`) ESCAPE `D`))))");
+ "VALUES (ROW((`A` LIKE (`B` || `C`) ESCAPE `D`)))");
// ESCAPE with no expression
// FIXME should fail at "escape"
@@ -627,14 +627,14 @@ public class SqlParserTest {
// Mixed LIKE and SIMILAR TO
check(
"values a similar to b like c similar to d escape e escape f",
- "(VALUES (ROW((`A` SIMILAR TO (`B` LIKE (`C` SIMILAR TO `D` ESCAPE `E`) ESCAPE `F`)))))");
+ "VALUES (ROW((`A` SIMILAR TO (`B` LIKE (`C` SIMILAR TO `D` ESCAPE `E`) ESCAPE `F`))))");
// SIMILAR TO with subquery
check(
"values a similar to (select * from t where a like b escape c) escape d",
- "(VALUES (ROW((`A` SIMILAR TO (SELECT *\n"
+ "VALUES (ROW((`A` SIMILAR TO (SELECT *\n"
+ "FROM `T`\n"
- + "WHERE (`A` LIKE `B` ESCAPE `C`)) ESCAPE `D`))))");
+ + "WHERE (`A` LIKE `B` ESCAPE `C`)) ESCAPE `D`)))");
}
@Test public void testFoo() {
@@ -791,7 +791,7 @@ public class SqlParserTest {
+ "group by grouping sets (deptno, (deptno, gender), ())")
.ok("SELECT `DEPTNO`\n"
+ "FROM `EMP`\n"
- + "GROUP BY (GROUPING_SETS(`DEPTNO`, (ROW(`DEPTNO`, `GENDER`)),))");
+ + "GROUP BY GROUPING SETS(`DEPTNO`, (`DEPTNO`, `GENDER`), ())");
// Grouping sets must have parentheses
sql("select deptno from emp\n"
@@ -807,14 +807,14 @@ public class SqlParserTest {
+ "order by a")
.ok("SELECT `DEPTNO`\n"
+ "FROM `EMP`\n"
- + "GROUP BY (GROUPING_SETS(`DEPTNO`, GROUPING_SETS(`E`, `D`),, CUBE(`X`, `Y`), ROLLUP(`P`, `Q`)))\n"
+ + "GROUP BY GROUPING SETS(`DEPTNO`, GROUPING SETS(`E`, `D`), (), CUBE(`X`, `Y`), ROLLUP(`P`, `Q`))\n"
+ "ORDER BY `A`");
sql("select deptno from emp\n"
+ "group by grouping sets (())")
.ok("SELECT `DEPTNO`\n"
+ "FROM `EMP`\n"
- + "GROUP BY (GROUPING_SETS())");
+ + "GROUP BY GROUPING SETS(())");
}
@Test public void testGroupByCube() {
@@ -822,7 +822,7 @@ public class SqlParserTest {
+ "group by cube ((a, b), (c, d))")
.ok("SELECT `DEPTNO`\n"
+ "FROM `EMP`\n"
- + "GROUP BY (CUBE((ROW(`A`, `B`)), (ROW(`C`, `D`))))");
+ + "GROUP BY CUBE((`A`, `B`), (`C`, `D`))");
}
@Test public void testGroupByCube2() {
@@ -830,7 +830,7 @@ public class SqlParserTest {
+ "group by cube ((a, b), (c, d)) order by a")
.ok("SELECT `DEPTNO`\n"
+ "FROM `EMP`\n"
- + "GROUP BY (CUBE((ROW(`A`, `B`)), (ROW(`C`, `D`))))\n"
+ + "GROUP BY CUBE((`A`, `B`), (`C`, `D`))\n"
+ "ORDER BY `A`");
sql("select deptno from emp\n"
+ "group by cube (^)")
@@ -842,7 +842,7 @@ public class SqlParserTest {
+ "group by rollup (deptno, deptno + 1, gender)")
.ok("SELECT `DEPTNO`\n"
+ "FROM `EMP`\n"
- + "GROUP BY (ROLLUP(`DEPTNO`, (`DEPTNO` + 1), `GENDER`))");
+ + "GROUP BY ROLLUP(`DEPTNO`, (`DEPTNO` + 1), `GENDER`)");
// Nested rollup not ok
sql("select deptno from emp\n"
@@ -855,7 +855,7 @@ public class SqlParserTest {
+ "group by grouping sets (deptno, (deptno, gender), ())")
.ok("SELECT `DEPTNO`, (GROUPING(`DEPTNO`))\n"
+ "FROM `EMP`\n"
- + "GROUP BY (GROUPING_SETS(`DEPTNO`, (ROW(`DEPTNO`, `GENDER`)),))");
+ + "GROUP BY GROUPING SETS(`DEPTNO`, (`DEPTNO`, `GENDER`), ())");
}
@Test public void testWith() {
@@ -864,8 +864,8 @@ public class SqlParserTest {
+ "select deptno from femaleEmps",
"WITH `FEMALEEMPS` AS (SELECT *\n"
+ "FROM `EMPS`\n"
- + "WHERE (`GENDER` = 'F')) SELECT `DEPTNO`\n"
- + "FROM `FEMALEEMPS`");
+ + "WHERE (`GENDER` = 'F')) (SELECT `DEPTNO`\n"
+ + "FROM `FEMALEEMPS`)");
}
@Test public void testWith2() {
@@ -877,8 +877,8 @@ public class SqlParserTest {
+ "FROM `EMPS`\n"
+ "WHERE (`GENDER` = 'F')), `MARRIEDFEMALEEMPS` (`X`, `Y`) AS (SELECT *\n"
+ "FROM `FEMALEEMPS`\n"
- + "WHERE (`MARITASTATUS` = 'M')) SELECT `DEPTNO`\n"
- + "FROM `FEMALEEMPS`");
+ + "WHERE (`MARITASTATUS` = 'M')) (SELECT `DEPTNO`\n"
+ + "FROM `FEMALEEMPS`)");
}
@Test public void testWithFails() {
@@ -891,8 +891,8 @@ public class SqlParserTest {
check(
"with v(i,c) as (values (1, 'a'), (2, 'bb'))\n"
+ "select c, i from v",
- "WITH `V` (`I`, `C`) AS (VALUES (ROW(1, 'a')), (ROW(2, 'bb'))) SELECT `C`, `I`\n"
- + "FROM `V`");
+ "WITH `V` (`I`, `C`) AS (VALUES (ROW(1, 'a')), (ROW(2, 'bb'))) (SELECT `C`, `I`\n"
+ + "FROM `V`)");
}
@Test public void testWithNestedFails() {
@@ -910,9 +910,9 @@ public class SqlParserTest {
+ " with dept2 as (select * from dept)\n"
+ " select 1 as one from empDept)",
"WITH `EMP2` AS (SELECT *\n"
- + "FROM `EMP`) WITH `DEPT2` AS (SELECT *\n"
- + "FROM `DEPT`) SELECT 1 AS `ONE`\n"
- + "FROM `EMPDEPT`");
+ + "FROM `EMP`) (WITH `DEPT2` AS (SELECT *\n"
+ + "FROM `DEPT`) (SELECT 1 AS `ONE`\n"
+ + "FROM `EMPDEPT`))");
}
@Test public void testWithUnion() {
@@ -1627,12 +1627,12 @@ public class SqlParserTest {
// stuff inside comment
check(
"values ( /** 1, 2 + ** */ 3)",
- "(VALUES (ROW(3)))");
+ "VALUES (ROW(3))");
// comment in string is preserved
check(
"values ('a string with /* a comment */ in it')",
- "(VALUES (ROW('a string with /* a comment */ in it')))");
+ "VALUES (ROW('a string with /* a comment */ in it'))");
// SQL:2003, 5.2, syntax rule # 8 "There shall be no <separator>
// separating the <minus sign>s of a <simple comment introducer>".
@@ -1640,12 +1640,12 @@ public class SqlParserTest {
check(
"values (- -1\n"
+ ")",
- "(VALUES (ROW((- -1))))");
+ "VALUES (ROW((- -1)))");
check(
"values (--1+\n"
+ "2)",
- "(VALUES (ROW(2)))");
+ "VALUES (ROW(2))");
// end of multiline commment without start
if (Bug.FRG73_FIXED) {
@@ -1692,29 +1692,29 @@ public class SqlParserTest {
check(
"values (1 + /* comment -- rest of line\n"
+ " rest of comment */ 2)",
- "(VALUES (ROW((1 + 2))))");
+ "VALUES (ROW((1 + 2)))");
// multiline comment inside singleline comment
check(
"values -- rest of line /* a comment */ \n"
+ "(1)",
- "(VALUES (ROW(1)))");
+ "VALUES (ROW(1))");
// non-terminated multiline comment inside singleline comment
check(
"values -- rest of line /* a comment \n"
+ "(1)",
- "(VALUES (ROW(1)))");
+ "VALUES (ROW(1))");
// even if comment abuts the tokens at either end, it becomes a space
check(
"values ('abc'/* a comment*/'def')",
- "(VALUES (ROW('abc'\n'def')))");
+ "VALUES (ROW('abc'\n'def'))");
// comment which starts as soon as it has begun
check(
"values /**/ (1)",
- "(VALUES (ROW(1)))");
+ "VALUES (ROW(1))");
}
// expressions
@@ -1939,11 +1939,11 @@ public class SqlParserTest {
}
@Test public void testValues() {
- check("values(1,'two')", "(VALUES (ROW(1, 'two')))");
+ check("values(1,'two')", "VALUES (ROW(1, 'two'))");
}
@Test public void testValuesExplicitRow() {
- check("values row(1,'two')", "(VALUES (ROW(1, 'two')))");
+ check("values row(1,'two')", "VALUES (ROW(1, 'two'))");
}
@Test public void testFromValues() {
@@ -1980,34 +1980,34 @@ public class SqlParserTest {
@Test public void testTableExtend() {
sql("select * from emp extend (x int, y varchar(10) not null)")
.ok("SELECT *\n"
- + "FROM (EXTEND(`EMP`, `X`, INTEGER, `Y`, VARCHAR(10)))");
+ + "FROM `EMP` EXTEND (`X` INTEGER, `Y` VARCHAR(10))");
sql("select * from emp extend (x int, y varchar(10) not null) where true")
.ok("SELECT *\n"
- + "FROM (EXTEND(`EMP`, `X`, INTEGER, `Y`, VARCHAR(10)))\n"
+ + "FROM `EMP` EXTEND (`X` INTEGER, `Y` VARCHAR(10))\n"
+ "WHERE TRUE");
// with table alias
sql("select * from emp extend (x int, y varchar(10) not null) as t")
.ok("SELECT *\n"
- + "FROM (EXTEND(`EMP`, `X`, INTEGER, `Y`, VARCHAR(10))) AS `T`");
+ + "FROM `EMP` EXTEND (`X` INTEGER, `Y` VARCHAR(10)) AS `T`");
// as previous, without AS
sql("select * from emp extend (x int, y varchar(10) not null) t")
.ok("SELECT *\n"
- + "FROM (EXTEND(`EMP`, `X`, INTEGER, `Y`, VARCHAR(10))) AS `T`");
+ + "FROM `EMP` EXTEND (`X` INTEGER, `Y` VARCHAR(10)) AS `T`");
// with table alias and column alias list
sql("select * from emp extend (x int, y varchar(10) not null) as t(a, b)")
.ok("SELECT *\n"
- + "FROM (EXTEND(`EMP`, `X`, INTEGER, `Y`, VARCHAR(10))) AS `T` (`A`, `B`)");
+ + "FROM `EMP` EXTEND (`X` INTEGER, `Y` VARCHAR(10)) AS `T` (`A`, `B`)");
// as previous, without AS
sql("select * from emp extend (x int, y varchar(10) not null) t(a, b)")
.ok("SELECT *\n"
- + "FROM (EXTEND(`EMP`, `X`, INTEGER, `Y`, VARCHAR(10))) AS `T` (`A`, `B`)");
+ + "FROM `EMP` EXTEND (`X` INTEGER, `Y` VARCHAR(10)) AS `T` (`A`, `B`)");
// omit EXTEND
sql("select * from emp (x int, y varchar(10) not null) t(a, b)")
.ok("SELECT *\n"
- + "FROM (EXTEND(`EMP`, `X`, INTEGER, `Y`, VARCHAR(10))) AS `T` (`A`, `B`)");
+ + "FROM `EMP` EXTEND (`X` INTEGER, `Y` VARCHAR(10)) AS `T` (`A`, `B`)");
sql("select * from emp (x int, y varchar(10) not null) where x = y")
.ok("SELECT *\n"
- + "FROM (EXTEND(`EMP`, `X`, INTEGER, `Y`, VARCHAR(10)))\n"
+ + "FROM `EMP` EXTEND (`X` INTEGER, `Y` VARCHAR(10))\n"
+ "WHERE (`X` = `Y`)");
}
@@ -2188,7 +2188,7 @@ public class SqlParserTest {
@Test public void testUpdate() {
sql("update emps set empno = empno + 1, sal = sal - 1 where empno=12")
- .ok("UPDATE `EMPS` (`EMPNO`, `SAL`) SET `EMPNO` = (`EMPNO` + 1)\n"
+ .ok("UPDATE `EMPS` SET `EMPNO` = (`EMPNO` + 1)\n"
+ ", `SAL` = (`SAL` - 1)\n"
+ "WHERE (`EMPNO` = 12)");
}
@@ -5655,8 +5655,8 @@ public class SqlParserTest {
"values _UTF16'"
+ ConversionUtil.TEST_UNICODE_STRING + "'";
String out1 =
- "(VALUES (ROW(_UTF16'"
- + ConversionUtil.TEST_UNICODE_STRING + "')))";
+ "VALUES (ROW(_UTF16'"
+ + ConversionUtil.TEST_UNICODE_STRING + "'))";
check(in1, out1);
// Without the U& prefix, escapes are left unprocessed
@@ -5664,8 +5664,8 @@ public class SqlParserTest {
"values '"
+ ConversionUtil.TEST_UNICODE_SQL_ESCAPED_LITERAL + "'";
String out2 =
- "(VALUES (ROW('"
- + ConversionUtil.TEST_UNICODE_SQL_ESCAPED_LITERAL + "')))";
+ "VALUES (ROW('"
+ + ConversionUtil.TEST_UNICODE_SQL_ESCAPED_LITERAL + "'))";
check(in2, out2);
// Likewise, even with the U& prefix, if some other escape
@@ -5676,8 +5676,8 @@ public class SqlParserTest {
+ ConversionUtil.TEST_UNICODE_SQL_ESCAPED_LITERAL
+ "' UESCAPE '!'";
String out3 =
- "(VALUES (ROW(_UTF16'"
- + ConversionUtil.TEST_UNICODE_SQL_ESCAPED_LITERAL + "')))";
+ "VALUES (ROW(_UTF16'"
+ + ConversionUtil.TEST_UNICODE_SQL_ESCAPED_LITERAL + "'))";
check(in3, out3);
}
@@ -5689,8 +5689,8 @@ public class SqlParserTest {
"values U&'"
+ ConversionUtil.TEST_UNICODE_SQL_ESCAPED_LITERAL + "'";
String out =
- "(VALUES (ROW(_UTF16'"
- + ConversionUtil.TEST_UNICODE_STRING + "')))";
+ "VALUES (ROW(_UTF16'"
+ + ConversionUtil.TEST_UNICODE_STRING + "'))";
check(in, out);
// Verify that we can override with an explicit escape character
@@ -5915,15 +5915,22 @@ public class SqlParserTest {
// Unparse with no dialect, always parenthesize.
final String actual = sqlNode.toSqlString(null, true).getSql();
- assertEquals(expected, actual);
+ assertEquals(expected, linux(actual));
- // Unparse again in Eigenbase dialect (which we can parse), and
+ // Unparse again in Calcite dialect (which we can parse), and
// minimal parentheses.
final String sql1 =
sqlNode.toSqlString(SqlDialect.CALCITE, false).getSql();
// Parse and unparse again.
- SqlNode sqlNode2 = parseStmtAndHandleEx(sql1);
+ SqlNode sqlNode2;
+ final Quoting q = quoting;
+ try {
+ quoting = Quoting.DOUBLE_QUOTE;
+ sqlNode2 = parseStmtAndHandleEx(sql1);
+ } finally {
+ quoting = q;
+ }
final String sql2 =
sqlNode2.toSqlString(SqlDialect.CALCITE, false).getSql();
@@ -5934,7 +5941,7 @@ public class SqlParserTest {
// If the unparser is not including sufficient parens to override
// precedence, the problem will show up here.
final String actual2 = sqlNode2.toSqlString(null, true).getSql();
- assertEquals(expected, actual2);
+ assertEquals(expected, linux(actual2));
}
public void checkExp(String sql, String expected) {
@@ -5942,15 +5949,22 @@ public class SqlParserTest {
// Unparse with no dialect, always parenthesize.
final String actual = sqlNode.toSqlString(null, true).getSql();
- assertEquals(expected, actual);
+ assertEquals(expected, linux(actual));
- // Unparse again in Eigenbase dialect (which we can parse), and
+ // Unparse again in Calcite dialect (which we can parse), and
// minimal parentheses.
final String sql1 =
sqlNode.toSqlString(SqlDialect.CALCITE, false).getSql();
// Parse and unparse again.
- SqlNode sqlNode2 = parseExpressionAndHandleEx(sql1);
+ SqlNode sqlNode2;
+ final Quoting q = quoting;
+ try {
+ quoting = Quoting.DOUBLE_QUOTE;
+ sqlNode2 = parseExpressionAndHandleEx(sql1);
+ } finally {
+ quoting = q;
+ }
final String sql2 =
sqlNode2.toSqlString(SqlDialect.CALCITE, false).getSql();
@@ -5961,7 +5975,7 @@ public class SqlParserTest {
// If the unparser is not including sufficient parens to override
// precedence, the problem will show up here.
final String actual2 = sqlNode2.toSqlString(null, true).getSql();
- assertEquals(expected, actual2);
+ assertEquals(expected, linux(actual2));
}
public void checkFails(String sql, String expectedMsgPattern) {
@@ -5973,6 +5987,13 @@ public class SqlParserTest {
}
}
+ private String linux(String s) {
+ if (LINUXIFY.get()[0]) {
+ s = Util.toLinux(s);
+ }
+ return s;
+ }
+
/** Helper class for building fluent code such as
* {@code sql("values 1").ok();}. */
private class Sql {
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/057f847b/core/src/test/java/org/apache/calcite/test/CalciteSuite.java
----------------------------------------------------------------------
diff --git a/core/src/test/java/org/apache/calcite/test/CalciteSuite.java b/core/src/test/java/org/apache/calcite/test/CalciteSuite.java
index f50a8f1..09c9aa7 100644
--- a/core/src/test/java/org/apache/calcite/test/CalciteSuite.java
+++ b/core/src/test/java/org/apache/calcite/test/CalciteSuite.java
@@ -18,6 +18,7 @@ package org.apache.calcite.test;
import org.apache.calcite.adapter.clone.ArrayTableTest;
import org.apache.calcite.jdbc.CalciteRemoteDriverTest;
+import org.apache.calcite.plan.RelOptPlanReaderTest;
import org.apache.calcite.plan.RelOptUtilTest;
import org.apache.calcite.plan.RelWriterTest;
import org.apache.calcite.plan.volcano.TraitPropagationTest;
@@ -28,6 +29,7 @@ import org.apache.calcite.rex.RexExecutorTest;
import org.apache.calcite.runtime.BinarySearchTest;
import org.apache.calcite.runtime.EnumerablesTest;
import org.apache.calcite.sql.parser.SqlParserTest;
+import org.apache.calcite.sql.parser.SqlUnParserTest;
import org.apache.calcite.sql.test.SqlAdvisorTest;
import org.apache.calcite.sql.test.SqlOperatorTest;
import org.apache.calcite.sql.test.SqlPrettyWriterTest;
@@ -88,9 +90,11 @@ import org.junit.runners.Suite;
EnumerablesTest.class,
ExceptionMessageTest.class,
InduceGroupingTypeTest.class,
+ RelOptPlanReaderTest.class,
// medium tests (above 0.1s)
SqlParserTest.class,
+ SqlUnParserTest.class,
SqlPrettyWriterTest.class,
SqlValidatorTest.class,
SqlAdvisorTest.class,
[2/8] incubator-calcite git commit: Add H2 integration test
Posted by jh...@apache.org.
Add H2 integration test
The test expects to find either ../calcite-test-dataset or ../../calcite-test-dataset with the test data
Add explicit dependencies (Julian Hyde)
Project: http://git-wip-us.apache.org/repos/asf/incubator-calcite/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-calcite/commit/b2b53b19
Tree: http://git-wip-us.apache.org/repos/asf/incubator-calcite/tree/b2b53b19
Diff: http://git-wip-us.apache.org/repos/asf/incubator-calcite/diff/b2b53b19
Branch: refs/heads/master
Commit: b2b53b19a8a2d4975a447692eeefc572230567dd
Parents: 4c0cf67
Author: Vladimir Sitnikov <si...@gmail.com>
Authored: Tue Feb 10 12:51:21 2015 +0300
Committer: Julian Hyde <jh...@apache.org>
Committed: Thu Apr 16 02:21:43 2015 -0700
----------------------------------------------------------------------
.../calcite/avatica/RemoteDriverTest.java | 3 ++
core/pom.xml | 34 ++++++++++++++++++--
.../java/org/apache/calcite/sql/SqlDialect.java | 4 +++
.../org/apache/calcite/test/CalciteAssert.java | 22 +++++++++++++
pom.xml | 12 +++----
5 files changed, 67 insertions(+), 8 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/b2b53b19/avatica-server/src/test/java/org/apache/calcite/avatica/RemoteDriverTest.java
----------------------------------------------------------------------
diff --git a/avatica-server/src/test/java/org/apache/calcite/avatica/RemoteDriverTest.java b/avatica-server/src/test/java/org/apache/calcite/avatica/RemoteDriverTest.java
index 20c3601..9cc18b4 100644
--- a/avatica-server/src/test/java/org/apache/calcite/avatica/RemoteDriverTest.java
+++ b/avatica-server/src/test/java/org/apache/calcite/avatica/RemoteDriverTest.java
@@ -393,6 +393,9 @@ public class RemoteDriverTest {
}
@Test public void testConnectionIsolation() throws Exception {
+ // Wait 5s for all other tests to finish. (Sorry! Hack!)
+ Thread.sleep(5000);
+
final String sql = "select * from (values (1, 'a'))";
Connection conn1 = ljs();
Connection conn2 = ljs();
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/b2b53b19/core/pom.xml
----------------------------------------------------------------------
diff --git a/core/pom.xml b/core/pom.xml
index 010ed8b..be7343a 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -76,6 +76,11 @@ limitations under the License.
<artifactId>guava</artifactId>
</dependency>
<dependency>
+ <groupId>com.h2database</groupId>
+ <artifactId>h2</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
@@ -136,6 +141,11 @@ limitations under the License.
<groupId>org.pentaho</groupId>
<artifactId>pentaho-aggdesigner-algorithm</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.postgresql</groupId>
+ <artifactId>postgresql</artifactId>
+ <scope>test</scope>
+ </dependency>
</dependencies>
<build>
@@ -229,9 +239,11 @@ limitations under the License.
<failOnWarning>true</failOnWarning>
<!-- ignore "unused but declared" warnings -->
<ignoredUnusedDeclaredDependencies>
+ <ignoredUnusedDeclaredDependency>com.h2database:h2</ignoredUnusedDeclaredDependency>
+ <ignoredUnusedDeclaredDependency>mysql:mysql-connector-java</ignoredUnusedDeclaredDependency>
<ignoredUnusedDeclaredDependency>net.hydromatic:scott-data-hsqldb</ignoredUnusedDeclaredDependency>
<ignoredUnusedDeclaredDependency>net.hydromatic:foodmart-data-hsqldb</ignoredUnusedDeclaredDependency>
- <ignoredUnusedDeclaredDependency>mysql:mysql-connector-java</ignoredUnusedDeclaredDependency>
+ <ignoredUnusedDeclaredDependency>org.postgresql:postgresql</ignoredUnusedDeclaredDependency>
</ignoredUnusedDeclaredDependencies>
</configuration>
</execution>
@@ -266,7 +278,9 @@ limitations under the License.
<executions>
<execution>
<id>failsafe-integration-test</id>
- <phase/>
+ <!-- Disable the integration test inherited from the parent pom
+ so that we can run multiple tests, one per database. -->
+ <phase>none</phase>
</execution>
<execution>
<id>failsafe-test-mysql</id>
@@ -300,6 +314,22 @@ limitations under the License.
</systemPropertyVariables>
</configuration>
</execution>
+ <execution>
+ <id>failsafe-test-h2</id>
+ <goals>
+ <goal>integration-test</goal>
+ </goals>
+ <phase>integration-test</phase>
+ <configuration>
+ <includes>
+ <include>org/apache/calcite/test/JdbcAdapterTest.java</include>
+ <include>org/apache/calcite/test/JdbcTest.java</include>
+ </includes>
+ <systemPropertyVariables>
+ <calcite.test.db>h2</calcite.test.db>
+ </systemPropertyVariables>
+ </configuration>
+ </execution>
</executions>
</plugin>
</plugins>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/b2b53b19/core/src/main/java/org/apache/calcite/sql/SqlDialect.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/SqlDialect.java b/core/src/main/java/org/apache/calcite/sql/SqlDialect.java
index 8f85b1d..586e18f 100644
--- a/core/src/main/java/org/apache/calcite/sql/SqlDialect.java
+++ b/core/src/main/java/org/apache/calcite/sql/SqlDialect.java
@@ -181,6 +181,8 @@ public class SqlDialect {
return DatabaseProduct.TERADATA;
} else if (upperProductName.contains("HSQL")) {
return DatabaseProduct.HSQLDB;
+ } else if (upperProductName.contains("H2")) {
+ return DatabaseProduct.H2;
} else if (upperProductName.contains("VERTICA")) {
return DatabaseProduct.VERTICA;
} else {
@@ -407,6 +409,7 @@ public class SqlDialect {
public boolean supportsCharSet() {
switch (databaseProduct) {
case MYSQL:
+ case H2:
case HSQLDB:
case PHOENIX:
case POSTGRESQL:
@@ -483,6 +486,7 @@ public class SqlDialect {
DERBY("Apache Derby", null),
DB2("IBM DB2", null),
FIREBIRD("Firebird", null),
+ H2("H2", "\""),
HIVE("Apache Hive", null),
INFORMIX("Informix", null),
INGRES("Ingres", null),
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/b2b53b19/core/src/test/java/org/apache/calcite/test/CalciteAssert.java
----------------------------------------------------------------------
diff --git a/core/src/test/java/org/apache/calcite/test/CalciteAssert.java b/core/src/test/java/org/apache/calcite/test/CalciteAssert.java
index 89fbba2..cc4e1f2 100644
--- a/core/src/test/java/org/apache/calcite/test/CalciteAssert.java
+++ b/core/src/test/java/org/apache/calcite/test/CalciteAssert.java
@@ -53,6 +53,7 @@ import com.google.common.collect.Lists;
import net.hydromatic.foodmart.data.hsqldb.FoodmartHsqldb;
import net.hydromatic.scott.data.hsqldb.ScottHsqldb;
+import java.io.File;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.lang.reflect.InvocationTargetException;
@@ -1483,6 +1484,10 @@ public class CalciteAssert {
"org.hsqldb.jdbcDriver"),
new ConnectionSpec(ScottHsqldb.URI, ScottHsqldb.USER,
ScottHsqldb.PASSWORD, "org.hsqldb.jdbcDriver")),
+ H2(
+ new ConnectionSpec("jdbc:h2:" + getDataSetPath()
+ + "/h2/target/foodmart;user=foodmart;password=foodmart",
+ "foodmart", "foodmart", "org.h2.Driver"), null),
MYSQL(
new ConnectionSpec("jdbc:mysql://localhost/foodmart", "foodmart",
"foodmart", "com.mysql.jdbc.Driver"), null),
@@ -1494,6 +1499,23 @@ public class CalciteAssert {
public final ConnectionSpec foodmart;
public final ConnectionSpec scott;
+ private static String getDataSetPath() {
+ String path = System.getProperty("calcite.test.dataset");
+ if (path != null) {
+ return path;
+ }
+ final String[] dirs = {
+ "../calcite-test-dataset",
+ "../../calcite-test-dataset"
+ };
+ for (String s : dirs) {
+ if (new File(s).exists() && new File(s, "vm").exists()) {
+ return s;
+ }
+ }
+ return ".";
+ }
+
DatabaseInstance(ConnectionSpec foodmart, ConnectionSpec scott) {
this.foodmart = foodmart;
this.scott = scott;
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/b2b53b19/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index ea2b0b5..fd79cdb 100644
--- a/pom.xml
+++ b/pom.xml
@@ -157,6 +157,12 @@ limitations under the License.
<version>14.0.1</version>
</dependency>
<dependency>
+ <groupId>com.h2database</groupId>
+ <artifactId>h2</artifactId>
+ <version>1.4.185</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.0.1</version>
@@ -248,12 +254,6 @@ limitations under the License.
<version>2.3.1</version>
</dependency>
<dependency>
- <groupId>com.h2database</groupId>
- <artifactId>h2</artifactId>
- <version>1.4.185</version>
- <scope>test</scope>
- </dependency>
- <dependency>
<groupId>org.incava</groupId>
<artifactId>java-diff</artifactId>
<version>1.1</version>
[3/8] incubator-calcite git commit: Add PostgreSQL integration test
Posted by jh...@apache.org.
Add PostgreSQL integration test
Project: http://git-wip-us.apache.org/repos/asf/incubator-calcite/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-calcite/commit/4c0cf674
Tree: http://git-wip-us.apache.org/repos/asf/incubator-calcite/tree/4c0cf674
Diff: http://git-wip-us.apache.org/repos/asf/incubator-calcite/diff/4c0cf674
Branch: refs/heads/master
Commit: 4c0cf674699ea15f8e3753dce44c83e15eb117d6
Parents: 1fd18a3
Author: Vladimir Sitnikov <si...@gmail.com>
Authored: Tue Feb 10 12:50:15 2015 +0300
Committer: Julian Hyde <jh...@apache.org>
Committed: Thu Apr 16 02:21:43 2015 -0700
----------------------------------------------------------------------
core/pom.xml | 16 ++++++++++++++++
.../org/apache/calcite/adapter/jdbc/JdbcSchema.java | 8 +++++++-
.../java/org/apache/calcite/sql/SqlDialect.java | 1 +
.../java/org/apache/calcite/test/CalciteAssert.java | 13 ++++++++-----
pom.xml | 12 ++++++++++++
5 files changed, 44 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/4c0cf674/core/pom.xml
----------------------------------------------------------------------
diff --git a/core/pom.xml b/core/pom.xml
index 479aca6..010ed8b 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -284,6 +284,22 @@ limitations under the License.
</systemPropertyVariables>
</configuration>
</execution>
+ <execution>
+ <id>failsafe-test-postgresql</id>
+ <goals>
+ <goal>integration-test</goal>
+ </goals>
+ <phase>integration-test</phase>
+ <configuration>
+ <includes>
+ <include>org/apache/calcite/test/JdbcAdapterTest.java</include>
+ <include>org/apache/calcite/test/JdbcTest.java</include>
+ </includes>
+ <systemPropertyVariables>
+ <calcite.test.db>postgresql</calcite.test.db>
+ </systemPropertyVariables>
+ </configuration>
+ </execution>
</executions>
</plugin>
</plugins>
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/4c0cf674/core/src/main/java/org/apache/calcite/adapter/jdbc/JdbcSchema.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/adapter/jdbc/JdbcSchema.java b/core/src/main/java/org/apache/calcite/adapter/jdbc/JdbcSchema.java
index cce3fcc..6d5b3cd 100644
--- a/core/src/main/java/org/apache/calcite/adapter/jdbc/JdbcSchema.java
+++ b/core/src/main/java/org/apache/calcite/adapter/jdbc/JdbcSchema.java
@@ -201,8 +201,14 @@ public class JdbcSchema implements Schema {
// returned by Phoenix among others, maps to TableType.SYSTEM_TABLE.
// We know enum constants are upper-case without spaces, so we can't
// make things worse.
+ // PostgreSQL returns tableTypeName==null for pg_toast* tables
+ // This can happen if you start JdbcSchema off a "public" PG schema
+ // The tables are not designed to be queried by users, however we do
+ // not filter them as we keep all the other table types.
final String tableTypeName2 =
- tableTypeName.toUpperCase().replace(' ', '_');
+ tableTypeName == null
+ ? null
+ : tableTypeName.toUpperCase().replace(' ', '_');
final TableType tableType =
Util.enumVal(TableType.class, tableTypeName2);
final JdbcTable table =
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/4c0cf674/core/src/main/java/org/apache/calcite/sql/SqlDialect.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/sql/SqlDialect.java b/core/src/main/java/org/apache/calcite/sql/SqlDialect.java
index 931d463..8f85b1d 100644
--- a/core/src/main/java/org/apache/calcite/sql/SqlDialect.java
+++ b/core/src/main/java/org/apache/calcite/sql/SqlDialect.java
@@ -409,6 +409,7 @@ public class SqlDialect {
case MYSQL:
case HSQLDB:
case PHOENIX:
+ case POSTGRESQL:
return false;
default:
return true;
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/4c0cf674/core/src/test/java/org/apache/calcite/test/CalciteAssert.java
----------------------------------------------------------------------
diff --git a/core/src/test/java/org/apache/calcite/test/CalciteAssert.java b/core/src/test/java/org/apache/calcite/test/CalciteAssert.java
index f221b02..89fbba2 100644
--- a/core/src/test/java/org/apache/calcite/test/CalciteAssert.java
+++ b/core/src/test/java/org/apache/calcite/test/CalciteAssert.java
@@ -103,10 +103,9 @@ public class CalciteAssert {
* failures. To run against MySQL, specify '-Dcalcite.test.db=mysql' on the
* java command line. */
public static final DatabaseInstance DB =
- Util.first(System.getProperty("calcite.test.db"), "hsqldb")
- .equals("mysql")
- ? DatabaseInstance.MYSQL
- : DatabaseInstance.HSQLDB;
+ DatabaseInstance.valueOf(
+ Util.first(System.getProperty("calcite.test.db"), "HSQLDB")
+ .toUpperCase());
/** Whether to enable slow tests. Default is false. */
public static final boolean ENABLE_SLOW =
@@ -1486,7 +1485,11 @@ public class CalciteAssert {
ScottHsqldb.PASSWORD, "org.hsqldb.jdbcDriver")),
MYSQL(
new ConnectionSpec("jdbc:mysql://localhost/foodmart", "foodmart",
- "foodmart", "com.mysql.jdbc.Driver"), null);
+ "foodmart", "com.mysql.jdbc.Driver"), null),
+ POSTGRESQL(
+ new ConnectionSpec(
+ "jdbc:postgresql://localhost/foodmart?user=foodmart&password=foodmart&searchpath=foodmart",
+ "foodmart", "foodmart", "org.postgresql.Driver"), null);
public final ConnectionSpec foodmart;
public final ConnectionSpec scott;
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/4c0cf674/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 1b9f832..ea2b0b5 100644
--- a/pom.xml
+++ b/pom.xml
@@ -217,6 +217,12 @@ limitations under the License.
<version>3.2</version>
</dependency>
<dependency>
+ <groupId>org.postgresql</groupId>
+ <artifactId>postgresql</artifactId>
+ <version>9.3-1102-jdbc3</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.10.3</version>
@@ -242,6 +248,12 @@ limitations under the License.
<version>2.3.1</version>
</dependency>
<dependency>
+ <groupId>com.h2database</groupId>
+ <artifactId>h2</artifactId>
+ <version>1.4.185</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
<groupId>org.incava</groupId>
<artifactId>java-diff</artifactId>
<version>1.1</version>
[5/8] incubator-calcite git commit: Test case and NPE patch for
[CALCITE-661] Remote fetch in Calcite JDBC driver (Yeong Wei)
Posted by jh...@apache.org.
Test case and NPE patch for [CALCITE-661] Remote fetch in Calcite JDBC driver (Yeong Wei)
Project: http://git-wip-us.apache.org/repos/asf/incubator-calcite/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-calcite/commit/88a6c5bd
Tree: http://git-wip-us.apache.org/repos/asf/incubator-calcite/tree/88a6c5bd
Diff: http://git-wip-us.apache.org/repos/asf/incubator-calcite/diff/88a6c5bd
Branch: refs/heads/master
Commit: 88a6c5bdd2437eecabe05181c892957ff6b1ac5c
Parents: ca871d3
Author: Julian Hyde <jh...@apache.org>
Authored: Wed Apr 15 09:24:55 2015 -0700
Committer: Julian Hyde <jh...@apache.org>
Committed: Thu Apr 16 17:53:21 2015 -0700
----------------------------------------------------------------------
.../java/org/apache/calcite/jdbc/CalciteMetaImpl.java | 5 +++--
.../apache/calcite/jdbc/CalciteRemoteDriverTest.java | 12 ++++++++++++
2 files changed, 15 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/88a6c5bd/core/src/main/java/org/apache/calcite/jdbc/CalciteMetaImpl.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/jdbc/CalciteMetaImpl.java b/core/src/main/java/org/apache/calcite/jdbc/CalciteMetaImpl.java
index 90c711e..455e55e 100644
--- a/core/src/main/java/org/apache/calcite/jdbc/CalciteMetaImpl.java
+++ b/core/src/main/java/org/apache/calcite/jdbc/CalciteMetaImpl.java
@@ -471,8 +471,9 @@ public class CalciteMetaImpl extends MetaImpl {
final StatementHandle h = createStatement(ch);
final CalciteConnectionImpl calciteConnection = getConnection();
CalciteServerStatement statement = calciteConnection.server.getStatement(h);
- calciteConnection.parseQuery(sql, statement.createPrepareContext(),
- maxRowCount);
+ h.signature =
+ calciteConnection.parseQuery(sql, statement.createPrepareContext(),
+ maxRowCount);
return h;
}
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/88a6c5bd/core/src/test/java/org/apache/calcite/jdbc/CalciteRemoteDriverTest.java
----------------------------------------------------------------------
diff --git a/core/src/test/java/org/apache/calcite/jdbc/CalciteRemoteDriverTest.java b/core/src/test/java/org/apache/calcite/jdbc/CalciteRemoteDriverTest.java
index a702657..c19216a 100644
--- a/core/src/test/java/org/apache/calcite/jdbc/CalciteRemoteDriverTest.java
+++ b/core/src/test/java/org/apache/calcite/jdbc/CalciteRemoteDriverTest.java
@@ -408,6 +408,18 @@ public class CalciteRemoteDriverTest {
remoteConnection.createStatement().getMoreResults();
}
+ @Test public void testRemotePreparedStatement() throws Exception {
+ final PreparedStatement preparedStatement =
+ remoteConnection.prepareStatement("select * from \"hr\".\"emps\"");
+ ResultSet resultSet = preparedStatement.executeQuery();
+ int count = 0;
+ while (resultSet.next()) {
+ count += 1;
+ }
+ // TODO: implement remote fetch
+ //assertTrue(count > 0);
+ }
+
/** A bunch of sample values of various types. */
private static final List<Object> SAMPLE_VALUES =
ImmutableList.<Object>of(false, true,
[8/8] incubator-calcite git commit: Disable test for [CALCITE-687]
Make RemoteDriverTest.testStatementLifecycle thread-safe
Posted by jh...@apache.org.
Disable test for [CALCITE-687] Make RemoteDriverTest.testStatementLifecycle thread-safe
Project: http://git-wip-us.apache.org/repos/asf/incubator-calcite/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-calcite/commit/b18a4df3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-calcite/tree/b18a4df3
Diff: http://git-wip-us.apache.org/repos/asf/incubator-calcite/diff/b18a4df3
Branch: refs/heads/master
Commit: b18a4df3dacc63d01aa71e3145049e3fdb2eef5c
Parents: 057f847
Author: Julian Hyde <jh...@apache.org>
Authored: Fri Apr 17 11:08:44 2015 -0700
Committer: Julian Hyde <jh...@apache.org>
Committed: Fri Apr 17 11:08:44 2015 -0700
----------------------------------------------------------------------
.../src/test/java/org/apache/calcite/avatica/RemoteDriverTest.java | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/b18a4df3/avatica-server/src/test/java/org/apache/calcite/avatica/RemoteDriverTest.java
----------------------------------------------------------------------
diff --git a/avatica-server/src/test/java/org/apache/calcite/avatica/RemoteDriverTest.java b/avatica-server/src/test/java/org/apache/calcite/avatica/RemoteDriverTest.java
index 9cc18b4..b2aa2b9 100644
--- a/avatica-server/src/test/java/org/apache/calcite/avatica/RemoteDriverTest.java
+++ b/avatica-server/src/test/java/org/apache/calcite/avatica/RemoteDriverTest.java
@@ -376,6 +376,7 @@ public class RemoteDriverTest {
}
}
+ @Ignore("[CALCITE-687] Make RemoteDriverTest.testStatementLifecycle thread-safe")
@Test public void testStatementLifecycle() throws Exception {
try (AvaticaConnection connection = (AvaticaConnection) ljs()) {
Map<Integer, AvaticaStatement> clientMap = connection.statementMap;
[6/8] incubator-calcite git commit: [CALCITE-661] Remote fetch in
Calcite JDBC driver
Posted by jh...@apache.org.
[CALCITE-661] Remote fetch in Calcite JDBC driver
Project: http://git-wip-us.apache.org/repos/asf/incubator-calcite/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-calcite/commit/01183de8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-calcite/tree/01183de8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-calcite/diff/01183de8
Branch: refs/heads/master
Commit: 01183de8ca99b6e5689a4dcb5b38d3a9788a789f
Parents: 88a6c5b
Author: Julian Hyde <jh...@apache.org>
Authored: Wed Apr 15 09:34:17 2015 -0700
Committer: Julian Hyde <jh...@apache.org>
Committed: Thu Apr 16 17:53:22 2015 -0700
----------------------------------------------------------------------
.../org/apache/calcite/avatica/MetaImpl.java | 13 ++++-
.../calcite/jdbc/CalciteConnectionImpl.java | 26 +++++++--
.../apache/calcite/jdbc/CalciteMetaImpl.java | 58 ++++++++++++++++++++
.../calcite/server/CalciteServerStatement.java | 11 ++++
.../calcite/jdbc/CalciteRemoteDriverTest.java | 21 +++++--
5 files changed, 118 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/01183de8/avatica/src/main/java/org/apache/calcite/avatica/MetaImpl.java
----------------------------------------------------------------------
diff --git a/avatica/src/main/java/org/apache/calcite/avatica/MetaImpl.java b/avatica/src/main/java/org/apache/calcite/avatica/MetaImpl.java
index aa21194..f8f35a7 100644
--- a/avatica/src/main/java/org/apache/calcite/avatica/MetaImpl.java
+++ b/avatica/src/main/java/org/apache/calcite/avatica/MetaImpl.java
@@ -107,8 +107,17 @@ public abstract class MetaImpl implements Meta {
}
public static List<List<Object>> collect(CursorFactory cursorFactory,
- Iterable<Object> iterable,
- List<List<Object>> list) {
+ final Iterator<Object> iterator, List<List<Object>> list) {
+ final Iterable<Object> iterable = new Iterable<Object>() {
+ public Iterator<Object> iterator() {
+ return iterator;
+ }
+ };
+ return collect(cursorFactory, iterable, list);
+ }
+
+ public static List<List<Object>> collect(CursorFactory cursorFactory,
+ Iterable<Object> iterable, List<List<Object>> list) {
switch (cursorFactory.style) {
case OBJECT:
for (Object o : iterable) {
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/01183de8/core/src/main/java/org/apache/calcite/jdbc/CalciteConnectionImpl.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/jdbc/CalciteConnectionImpl.java b/core/src/main/java/org/apache/calcite/jdbc/CalciteConnectionImpl.java
index 88ec54b..94b2bb7 100644
--- a/core/src/main/java/org/apache/calcite/jdbc/CalciteConnectionImpl.java
+++ b/core/src/main/java/org/apache/calcite/jdbc/CalciteConnectionImpl.java
@@ -63,6 +63,7 @@ import com.google.common.collect.Maps;
import java.io.Serializable;
import java.lang.reflect.Type;
import java.sql.SQLException;
+import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Properties;
@@ -157,9 +158,6 @@ abstract class CalciteConnectionImpl
parseQuery(sql, new ContextImpl(this), -1);
return (CalcitePreparedStatement) factory.newPreparedStatement(this, null,
signature, resultSetType, resultSetConcurrency, resultSetHoldability);
- } catch (RuntimeException e) {
- throw Helper.INSTANCE.createException(
- "Error while preparing statement [" + sql + "]", e);
} catch (Exception e) {
throw Helper.INSTANCE.createException(
"Error while preparing statement [" + sql + "]", e);
@@ -196,11 +194,11 @@ abstract class CalciteConnectionImpl
public <T> Queryable<T> createQuery(
Expression expression, Class<T> rowType) {
- return new CalciteQueryable<T>(this, rowType, expression);
+ return new CalciteQueryable<>(this, rowType, expression);
}
public <T> Queryable<T> createQuery(Expression expression, Type rowType) {
- return new CalciteQueryable<T>(this, rowType, expression);
+ return new CalciteQueryable<>(this, rowType, expression);
}
public <T> T execute(Expression expression, Type type) {
@@ -438,6 +436,8 @@ abstract class CalciteConnectionImpl
static class CalciteServerStatementImpl
implements CalciteServerStatement {
private final CalciteConnectionImpl connection;
+ private Iterator<Object> iterator;
+ private Meta.Signature signature;
public CalciteServerStatementImpl(CalciteConnectionImpl connection) {
this.connection = Preconditions.checkNotNull(connection);
@@ -450,6 +450,22 @@ abstract class CalciteConnectionImpl
public CalciteConnection getConnection() {
return connection;
}
+
+ public void setSignature(Meta.Signature signature) {
+ this.signature = signature;
+ }
+
+ public Meta.Signature getSignature() {
+ return signature;
+ }
+
+ public Iterator<Object> getResultSet() {
+ return iterator;
+ }
+
+ public void setResultSet(Iterator<Object> iterator) {
+ this.iterator = iterator;
+ }
}
}
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/01183de8/core/src/main/java/org/apache/calcite/jdbc/CalciteMetaImpl.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/jdbc/CalciteMetaImpl.java b/core/src/main/java/org/apache/calcite/jdbc/CalciteMetaImpl.java
index 455e55e..7054685 100644
--- a/core/src/main/java/org/apache/calcite/jdbc/CalciteMetaImpl.java
+++ b/core/src/main/java/org/apache/calcite/jdbc/CalciteMetaImpl.java
@@ -23,6 +23,7 @@ import org.apache.calcite.avatica.AvaticaParameter;
import org.apache.calcite.avatica.AvaticaStatement;
import org.apache.calcite.avatica.AvaticaUtils;
import org.apache.calcite.avatica.ColumnMetaData;
+import org.apache.calcite.avatica.Meta;
import org.apache.calcite.avatica.MetaImpl;
import org.apache.calcite.linq4j.Enumerable;
import org.apache.calcite.linq4j.Enumerator;
@@ -57,6 +58,7 @@ import java.sql.DatabaseMetaData;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.Collections;
+import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
@@ -474,6 +476,7 @@ public class CalciteMetaImpl extends MetaImpl {
h.signature =
calciteConnection.parseQuery(sql, statement.createPrepareContext(),
maxRowCount);
+ statement.setSignature(h.signature);
return h;
}
@@ -501,6 +504,28 @@ public class CalciteMetaImpl extends MetaImpl {
// TODO: share code with prepare and createIterable
}
+ @Override public Frame fetch(StatementHandle h, List<Object> parameterValues,
+ int offset, int fetchMaxRowCount) {
+ final CalciteConnectionImpl calciteConnection = getConnection();
+ CalciteServerStatement stmt = calciteConnection.server.getStatement(h);
+ final Signature signature = stmt.getSignature();
+ final Iterator<Object> iterator;
+ if (parameterValues != null) {
+ final Iterable<Object> iterable =
+ createIterable(h, signature, Collections.emptyList(), null);
+ iterator = iterable.iterator();
+ stmt.setResultSet(iterator);
+ } else {
+ iterator = stmt.getResultSet();
+ }
+ final List<List<Object>> list = new ArrayList<>();
+ List<List<Object>> rows =
+ MetaImpl.collect(signature.cursorFactory,
+ LimitIterator.of(iterator, fetchMaxRowCount), list);
+ boolean done = fetchMaxRowCount == 0 || list.size() < fetchMaxRowCount;
+ return new Meta.Frame(offset, done, (List<Object>) (List) rows);
+ }
+
/** A trojan-horse method, subject to change without notice. */
@VisibleForTesting
public static DataContext createDataContext(CalciteConnection connection) {
@@ -571,6 +596,39 @@ public class CalciteMetaImpl extends MetaImpl {
};
}
}
+
+ /** Iterator that returns at most {@code limit} rows from an underlying
+ * {@link Iterator}. */
+ private static class LimitIterator<E> implements Iterator<E> {
+ private final Iterator<E> iterator;
+ private final int limit;
+ int i = 0;
+
+ private LimitIterator(Iterator<E> iterator, int limit) {
+ this.iterator = iterator;
+ this.limit = limit;
+ }
+
+ static <E> Iterator<E> of(Iterator<E> iterator, int limit) {
+ if (limit <= 0) {
+ return iterator;
+ }
+ return new LimitIterator<>(iterator, limit);
+ }
+
+ public boolean hasNext() {
+ return iterator.hasNext() && i < limit;
+ }
+
+ public E next() {
+ ++i;
+ return iterator.next();
+ }
+
+ public void remove() {
+ throw new UnsupportedOperationException();
+ }
+ }
}
// End CalciteMetaImpl.java
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/01183de8/core/src/main/java/org/apache/calcite/server/CalciteServerStatement.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/calcite/server/CalciteServerStatement.java b/core/src/main/java/org/apache/calcite/server/CalciteServerStatement.java
index 61ecf88..244879d 100644
--- a/core/src/main/java/org/apache/calcite/server/CalciteServerStatement.java
+++ b/core/src/main/java/org/apache/calcite/server/CalciteServerStatement.java
@@ -16,9 +16,12 @@
*/
package org.apache.calcite.server;
+import org.apache.calcite.avatica.Meta;
import org.apache.calcite.jdbc.CalciteConnection;
import org.apache.calcite.jdbc.CalcitePrepare;
+import java.util.Iterator;
+
/**
* Statement within a Calcite server.
*/
@@ -28,6 +31,14 @@ public interface CalciteServerStatement {
/** Returns the connection. */
CalciteConnection getConnection();
+
+ void setSignature(Meta.Signature signature);
+
+ Meta.Signature getSignature();
+
+ Iterator<Object> getResultSet();
+
+ void setResultSet(Iterator<Object> resultSet);
}
// End CalciteServerStatement.java
http://git-wip-us.apache.org/repos/asf/incubator-calcite/blob/01183de8/core/src/test/java/org/apache/calcite/jdbc/CalciteRemoteDriverTest.java
----------------------------------------------------------------------
diff --git a/core/src/test/java/org/apache/calcite/jdbc/CalciteRemoteDriverTest.java b/core/src/test/java/org/apache/calcite/jdbc/CalciteRemoteDriverTest.java
index c19216a..f9a32ac 100644
--- a/core/src/test/java/org/apache/calcite/jdbc/CalciteRemoteDriverTest.java
+++ b/core/src/test/java/org/apache/calcite/jdbc/CalciteRemoteDriverTest.java
@@ -408,16 +408,29 @@ public class CalciteRemoteDriverTest {
remoteConnection.createStatement().getMoreResults();
}
- @Test public void testRemotePreparedStatement() throws Exception {
+ @Test public void testRemoteExecute() throws Exception {
+ ResultSet resultSet =
+ remoteConnection.createStatement().executeQuery(
+ "select * from \"hr\".\"emps\"");
+ int count = 0;
+ while (resultSet.next()) {
+ ++count;
+ }
+ assertTrue(count > 0);
+ }
+
+ /** Test case for
+ * <a href="https://issues.apache.org/jira/browse/CALCITE-661">[CALCITE-661]
+ * Remote fetch in Calcite JDBC driver</a>. */
+ @Test public void testRemotePrepareExecute() throws Exception {
final PreparedStatement preparedStatement =
remoteConnection.prepareStatement("select * from \"hr\".\"emps\"");
ResultSet resultSet = preparedStatement.executeQuery();
int count = 0;
while (resultSet.next()) {
- count += 1;
+ ++count;
}
- // TODO: implement remote fetch
- //assertTrue(count > 0);
+ assertTrue(count > 0);
}
/** A bunch of sample values of various types. */