You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by al...@apache.org on 2024/10/03 19:47:51 UTC
(datafusion) branch main updated: Add IMDB(JOB) Benchmark [2/N] (imdb queries) (#12529)
This is an automated email from the ASF dual-hosted git repository.
alamb pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/main by this push:
new 77f330c6a2 Add IMDB(JOB) Benchmark [2/N] (imdb queries) (#12529)
77f330c6a2 is described below
commit 77f330c6a2b26f2d1d4d4bf11d456fad466316b4
Author: Austin Liu <au...@gmail.com>
AuthorDate: Fri Oct 4 03:47:46 2024 +0800
Add IMDB(JOB) Benchmark [2/N] (imdb queries) (#12529)
* imdb dataset
* cargo fmt
* Add 113 queries for IMDB(JOB)
Signed-off-by: Austin Liu <au...@gmail.com>
* Add `get_query_sql` from `query_id` string
Signed-off-by: Austin Liu <au...@gmail.com>
* Fix CSV reader & Remove Parquet partition
Signed-off-by: Austin Liu <au...@gmail.com>
* Add benchmark IMDB runner
Signed-off-by: Austin Liu <au...@gmail.com>
* Add `run_imdb` script
Signed-off-by: Austin Liu <au...@gmail.com>
* Add checker for imdb option
Signed-off-by: Austin Liu <au...@gmail.com>
* Add SLT for IMDB
Signed-off-by: Austin Liu <au...@gmail.com>
* Fix `get_query_sql()` for CI roundtrip test
Signed-off-by: Austin Liu <au...@gmail.com>
Fix `get_query_sql()` for CI roundtrip test
Signed-off-by: Austin Liu <au...@gmail.com>
Fix `get_query_sql()` for CI roundtrip test
Signed-off-by: Austin Liu <au...@gmail.com>
* Clean up
Signed-off-by: Austin Liu <au...@gmail.com>
* Add missing license
Signed-off-by: Austin Liu <au...@gmail.com>
* Add IMDB(JOB) queries `2b` to `5c`
Signed-off-by: Austin Liu <au...@gmail.com>
* Add `INCLUDE_IMDB` in CI verify-benchmark-results
Signed-off-by: Austin Liu <au...@gmail.com>
* Prepare IMDB dataset
Signed-off-by: Austin Liu <au...@gmail.com>
Prepare IMDB dataset
Signed-off-by: Austin Liu <au...@gmail.com>
* use uint as id type
* format
* Seperate `tpch` and `imdb` benchmarking CI jobs
Signed-off-by: Austin Liu <au...@gmail.com>
Fix path
Signed-off-by: Austin Liu <au...@gmail.com>
Fix path
Signed-off-by: Austin Liu <au...@gmail.com>
Remove `tpch` in `imdb` benchmark
Signed-off-by: Austin Liu <au...@gmail.com>
* Remove IMDB(JOB) slt in CI
Signed-off-by: Austin Liu <au...@gmail.com>
Remove IMDB(JOB) slt in CI
Signed-off-by: Austin Liu <au...@gmail.com>
---------
Signed-off-by: Austin Liu <au...@gmail.com>
Co-authored-by: DouPache <do...@gmail.com>
---
benchmarks/bench.sh | 14 +
benchmarks/queries/imdb/10a.sql | 1 +
benchmarks/queries/imdb/10b.sql | 1 +
benchmarks/queries/imdb/10c.sql | 1 +
benchmarks/queries/imdb/11a.sql | 1 +
benchmarks/queries/imdb/11b.sql | 1 +
benchmarks/queries/imdb/11c.sql | 1 +
benchmarks/queries/imdb/11d.sql | 1 +
benchmarks/queries/imdb/12a.sql | 1 +
benchmarks/queries/imdb/12b.sql | 1 +
benchmarks/queries/imdb/12c.sql | 1 +
benchmarks/queries/imdb/13a.sql | 1 +
benchmarks/queries/imdb/13b.sql | 1 +
benchmarks/queries/imdb/13c.sql | 1 +
benchmarks/queries/imdb/13d.sql | 1 +
benchmarks/queries/imdb/14a.sql | 1 +
benchmarks/queries/imdb/14b.sql | 1 +
benchmarks/queries/imdb/14c.sql | 1 +
benchmarks/queries/imdb/15a.sql | 1 +
benchmarks/queries/imdb/15b.sql | 1 +
benchmarks/queries/imdb/15c.sql | 1 +
benchmarks/queries/imdb/15d.sql | 1 +
benchmarks/queries/imdb/16a.sql | 1 +
benchmarks/queries/imdb/16b.sql | 1 +
benchmarks/queries/imdb/16c.sql | 1 +
benchmarks/queries/imdb/16d.sql | 1 +
benchmarks/queries/imdb/17a.sql | 1 +
benchmarks/queries/imdb/17b.sql | 1 +
benchmarks/queries/imdb/17c.sql | 1 +
benchmarks/queries/imdb/17d.sql | 1 +
benchmarks/queries/imdb/17e.sql | 1 +
benchmarks/queries/imdb/17f.sql | 1 +
benchmarks/queries/imdb/18a.sql | 1 +
benchmarks/queries/imdb/18b.sql | 1 +
benchmarks/queries/imdb/18c.sql | 1 +
benchmarks/queries/imdb/19a.sql | 1 +
benchmarks/queries/imdb/19b.sql | 1 +
benchmarks/queries/imdb/19c.sql | 1 +
benchmarks/queries/imdb/19d.sql | 1 +
benchmarks/queries/imdb/1a.sql | 1 +
benchmarks/queries/imdb/1b.sql | 1 +
benchmarks/queries/imdb/1c.sql | 1 +
benchmarks/queries/imdb/1d.sql | 1 +
benchmarks/queries/imdb/20a.sql | 1 +
benchmarks/queries/imdb/20b.sql | 1 +
benchmarks/queries/imdb/20c.sql | 1 +
benchmarks/queries/imdb/21a.sql | 1 +
benchmarks/queries/imdb/21b.sql | 1 +
benchmarks/queries/imdb/21c.sql | 1 +
benchmarks/queries/imdb/22a.sql | 1 +
benchmarks/queries/imdb/22b.sql | 1 +
benchmarks/queries/imdb/22c.sql | 1 +
benchmarks/queries/imdb/22d.sql | 1 +
benchmarks/queries/imdb/23a.sql | 1 +
benchmarks/queries/imdb/23b.sql | 1 +
benchmarks/queries/imdb/23c.sql | 1 +
benchmarks/queries/imdb/24a.sql | 1 +
benchmarks/queries/imdb/24b.sql | 1 +
benchmarks/queries/imdb/25a.sql | 1 +
benchmarks/queries/imdb/25b.sql | 1 +
benchmarks/queries/imdb/25c.sql | 1 +
benchmarks/queries/imdb/26a.sql | 1 +
benchmarks/queries/imdb/26b.sql | 1 +
benchmarks/queries/imdb/26c.sql | 1 +
benchmarks/queries/imdb/27a.sql | 1 +
benchmarks/queries/imdb/27b.sql | 1 +
benchmarks/queries/imdb/27c.sql | 1 +
benchmarks/queries/imdb/28a.sql | 1 +
benchmarks/queries/imdb/28b.sql | 1 +
benchmarks/queries/imdb/28c.sql | 1 +
benchmarks/queries/imdb/29a.sql | 1 +
benchmarks/queries/imdb/29b.sql | 1 +
benchmarks/queries/imdb/29c.sql | 1 +
benchmarks/queries/imdb/2a.sql | 1 +
benchmarks/queries/imdb/2b.sql | 1 +
benchmarks/queries/imdb/2c.sql | 1 +
benchmarks/queries/imdb/2d.sql | 1 +
benchmarks/queries/imdb/30a.sql | 1 +
benchmarks/queries/imdb/30b.sql | 1 +
benchmarks/queries/imdb/30c.sql | 1 +
benchmarks/queries/imdb/31a.sql | 1 +
benchmarks/queries/imdb/31b.sql | 1 +
benchmarks/queries/imdb/31c.sql | 1 +
benchmarks/queries/imdb/32a.sql | 1 +
benchmarks/queries/imdb/32b.sql | 1 +
benchmarks/queries/imdb/33a.sql | 1 +
benchmarks/queries/imdb/33b.sql | 1 +
benchmarks/queries/imdb/33c.sql | 1 +
benchmarks/queries/imdb/3a.sql | 1 +
benchmarks/queries/imdb/3b.sql | 1 +
benchmarks/queries/imdb/3c.sql | 1 +
benchmarks/queries/imdb/4a.sql | 1 +
benchmarks/queries/imdb/4b.sql | 1 +
benchmarks/queries/imdb/4c.sql | 1 +
benchmarks/queries/imdb/5a.sql | 1 +
benchmarks/queries/imdb/5b.sql | 1 +
benchmarks/queries/imdb/5c.sql | 1 +
benchmarks/queries/imdb/6a.sql | 1 +
benchmarks/queries/imdb/6b.sql | 1 +
benchmarks/queries/imdb/6c.sql | 1 +
benchmarks/queries/imdb/6d.sql | 1 +
benchmarks/queries/imdb/6e.sql | 1 +
benchmarks/queries/imdb/6f.sql | 1 +
benchmarks/queries/imdb/7a.sql | 1 +
benchmarks/queries/imdb/7b.sql | 1 +
benchmarks/queries/imdb/7c.sql | 1 +
benchmarks/queries/imdb/8a.sql | 1 +
benchmarks/queries/imdb/8b.sql | 1 +
benchmarks/queries/imdb/8c.sql | 1 +
benchmarks/queries/imdb/8d.sql | 1 +
benchmarks/queries/imdb/9a.sql | 1 +
benchmarks/queries/imdb/9b.sql | 1 +
benchmarks/queries/imdb/9c.sql | 1 +
benchmarks/queries/imdb/9d.sql | 1 +
benchmarks/src/bin/dfbench.rs | 4 +-
benchmarks/src/bin/imdb.rs | 11 +
benchmarks/src/imdb/convert.rs | 6 +-
benchmarks/src/imdb/mod.rs | 75 ++--
benchmarks/src/imdb/run.rs | 827 ++++++++++++++++++++++++++++++++++++++++
119 files changed, 1023 insertions(+), 27 deletions(-)
diff --git a/benchmarks/bench.sh b/benchmarks/bench.sh
index 24efab6c6c..70faa9ef2b 100755
--- a/benchmarks/bench.sh
+++ b/benchmarks/bench.sh
@@ -211,6 +211,7 @@ main() {
run_clickbench_1
run_clickbench_partitioned
run_clickbench_extended
+ run_imdb
;;
tpch)
run_tpch "1"
@@ -239,6 +240,9 @@ main() {
clickbench_extended)
run_clickbench_extended
;;
+ imdb)
+ run_imdb
+ ;;
*)
echo "Error: unknown benchmark '$BENCHMARK' for run"
usage
@@ -510,6 +514,16 @@ data_imdb() {
fi
}
+# Runs the imdb benchmark
+run_imdb() {
+ IMDB_DIR="${DATA_DIR}/imdb"
+
+ RESULTS_FILE="${RESULTS_DIR}/imdb.json"
+ echo "RESULTS_FILE: ${RESULTS_FILE}"
+ echo "Running imdb benchmark..."
+ $CARGO_COMMAND --bin imdb -- benchmark datafusion --iterations 5 --path "${IMDB_DIR}" --prefer_hash_join "${PREFER_HASH_JOIN}" --format parquet -o "${RESULTS_FILE}"
+}
+
diff --git a/benchmarks/queries/imdb/10a.sql b/benchmarks/queries/imdb/10a.sql
new file mode 100644
index 0000000000..95b049b774
--- /dev/null
+++ b/benchmarks/queries/imdb/10a.sql
@@ -0,0 +1 @@
+SELECT MIN(chn.name) AS uncredited_voiced_character, MIN(t.title) AS russian_movie FROM char_name AS chn, cast_info AS ci, company_name AS cn, company_type AS ct, movie_companies AS mc, role_type AS rt, title AS t WHERE ci.note like '%(voice)%' and ci.note like '%(uncredited)%' AND cn.country_code = '[ru]' AND rt.role = 'actor' AND t.production_year > 2005 AND t.id = mc.movie_id AND t.id = ci.movie_id AND ci.movie_id = mc.movie_id AND chn.id = ci.person_role_id AND rt.id = ci.role_id [...]
diff --git a/benchmarks/queries/imdb/10b.sql b/benchmarks/queries/imdb/10b.sql
new file mode 100644
index 0000000000..c321536314
--- /dev/null
+++ b/benchmarks/queries/imdb/10b.sql
@@ -0,0 +1 @@
+SELECT MIN(chn.name) AS character, MIN(t.title) AS russian_mov_with_actor_producer FROM char_name AS chn, cast_info AS ci, company_name AS cn, company_type AS ct, movie_companies AS mc, role_type AS rt, title AS t WHERE ci.note like '%(producer)%' AND cn.country_code = '[ru]' AND rt.role = 'actor' AND t.production_year > 2010 AND t.id = mc.movie_id AND t.id = ci.movie_id AND ci.movie_id = mc.movie_id AND chn.id = ci.person_role_id AND rt.id = ci.role_id AND cn.id = mc.company_id AND c [...]
diff --git a/benchmarks/queries/imdb/10c.sql b/benchmarks/queries/imdb/10c.sql
new file mode 100644
index 0000000000..b862cf4fa7
--- /dev/null
+++ b/benchmarks/queries/imdb/10c.sql
@@ -0,0 +1 @@
+SELECT MIN(chn.name) AS character, MIN(t.title) AS movie_with_american_producer FROM char_name AS chn, cast_info AS ci, company_name AS cn, company_type AS ct, movie_companies AS mc, role_type AS rt, title AS t WHERE ci.note like '%(producer)%' AND cn.country_code = '[us]' AND t.production_year > 1990 AND t.id = mc.movie_id AND t.id = ci.movie_id AND ci.movie_id = mc.movie_id AND chn.id = ci.person_role_id AND rt.id = ci.role_id AND cn.id = mc.company_id AND ct.id = mc.company_type_id;
diff --git a/benchmarks/queries/imdb/11a.sql b/benchmarks/queries/imdb/11a.sql
new file mode 100644
index 0000000000..f835968e90
--- /dev/null
+++ b/benchmarks/queries/imdb/11a.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS from_company, MIN(lt.link) AS movie_link_type, MIN(t.title) AS non_polish_sequel_movie FROM company_name AS cn, company_type AS ct, keyword AS k, link_type AS lt, movie_companies AS mc, movie_keyword AS mk, movie_link AS ml, title AS t WHERE cn.country_code !='[pl]' AND (cn.name LIKE '%Film%' OR cn.name LIKE '%Warner%') AND ct.kind ='production companies' AND k.keyword ='sequel' AND lt.link LIKE '%follow%' AND mc.note IS NULL AND t.production_year BETWEEN 1950 AND [...]
diff --git a/benchmarks/queries/imdb/11b.sql b/benchmarks/queries/imdb/11b.sql
new file mode 100644
index 0000000000..2411e19ea6
--- /dev/null
+++ b/benchmarks/queries/imdb/11b.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS from_company, MIN(lt.link) AS movie_link_type, MIN(t.title) AS sequel_movie FROM company_name AS cn, company_type AS ct, keyword AS k, link_type AS lt, movie_companies AS mc, movie_keyword AS mk, movie_link AS ml, title AS t WHERE cn.country_code !='[pl]' AND (cn.name LIKE '%Film%' OR cn.name LIKE '%Warner%') AND ct.kind ='production companies' AND k.keyword ='sequel' AND lt.link LIKE '%follows%' AND mc.note IS NULL AND t.production_year = 1998 and t.title like '% [...]
diff --git a/benchmarks/queries/imdb/11c.sql b/benchmarks/queries/imdb/11c.sql
new file mode 100644
index 0000000000..3bf7946789
--- /dev/null
+++ b/benchmarks/queries/imdb/11c.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS from_company, MIN(mc.note) AS production_note, MIN(t.title) AS movie_based_on_book FROM company_name AS cn, company_type AS ct, keyword AS k, link_type AS lt, movie_companies AS mc, movie_keyword AS mk, movie_link AS ml, title AS t WHERE cn.country_code !='[pl]' and (cn.name like '20th Century Fox%' or cn.name like 'Twentieth Century Fox%') AND ct.kind != 'production companies' and ct.kind is not NULL AND k.keyword in ('sequel', 'revenge', 'based-on-novel') AND [...]
diff --git a/benchmarks/queries/imdb/11d.sql b/benchmarks/queries/imdb/11d.sql
new file mode 100644
index 0000000000..0bc33e1d6e
--- /dev/null
+++ b/benchmarks/queries/imdb/11d.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS from_company, MIN(mc.note) AS production_note, MIN(t.title) AS movie_based_on_book FROM company_name AS cn, company_type AS ct, keyword AS k, link_type AS lt, movie_companies AS mc, movie_keyword AS mk, movie_link AS ml, title AS t WHERE cn.country_code !='[pl]' AND ct.kind != 'production companies' and ct.kind is not NULL AND k.keyword in ('sequel', 'revenge', 'based-on-novel') AND mc.note is not NULL AND t.production_year > 1950 AND lt.id = ml.link_type_id A [...]
diff --git a/benchmarks/queries/imdb/12a.sql b/benchmarks/queries/imdb/12a.sql
new file mode 100644
index 0000000000..22add74bd5
--- /dev/null
+++ b/benchmarks/queries/imdb/12a.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS movie_company, MIN(mi_idx.info) AS rating, MIN(t.title) AS drama_horror_movie FROM company_name AS cn, company_type AS ct, info_type AS it1, info_type AS it2, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, title AS t WHERE cn.country_code = '[us]' AND ct.kind = 'production companies' AND it1.info = 'genres' AND it2.info = 'rating' AND mi.info in ('Drama', 'Horror') AND mi_idx.info > '8.0' AND t.production_year between 2005 and 2008 AND t.id [...]
diff --git a/benchmarks/queries/imdb/12b.sql b/benchmarks/queries/imdb/12b.sql
new file mode 100644
index 0000000000..fc30ad550d
--- /dev/null
+++ b/benchmarks/queries/imdb/12b.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS budget, MIN(t.title) AS unsuccsessful_movie FROM company_name AS cn, company_type AS ct, info_type AS it1, info_type AS it2, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, title AS t WHERE cn.country_code ='[us]' AND ct.kind is not NULL and (ct.kind ='production companies' or ct.kind = 'distributors') AND it1.info ='budget' AND it2.info ='bottom 10 rank' AND t.production_year >2000 AND (t.title LIKE 'Birdemic%' OR t.title LIKE '%Movie%') AND t. [...]
diff --git a/benchmarks/queries/imdb/12c.sql b/benchmarks/queries/imdb/12c.sql
new file mode 100644
index 0000000000..64a340b238
--- /dev/null
+++ b/benchmarks/queries/imdb/12c.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS movie_company, MIN(mi_idx.info) AS rating, MIN(t.title) AS mainstream_movie FROM company_name AS cn, company_type AS ct, info_type AS it1, info_type AS it2, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, title AS t WHERE cn.country_code = '[us]' AND ct.kind = 'production companies' AND it1.info = 'genres' AND it2.info = 'rating' AND mi.info in ('Drama', 'Horror', 'Western', 'Family') AND mi_idx.info > '7.0' AND t.production_year between 200 [...]
diff --git a/benchmarks/queries/imdb/13a.sql b/benchmarks/queries/imdb/13a.sql
new file mode 100644
index 0000000000..95eb439d1e
--- /dev/null
+++ b/benchmarks/queries/imdb/13a.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS release_date, MIN(miidx.info) AS rating, MIN(t.title) AS german_movie FROM company_name AS cn, company_type AS ct, info_type AS it, info_type AS it2, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_info_idx AS miidx, title AS t WHERE cn.country_code ='[de]' AND ct.kind ='production companies' AND it.info ='rating' AND it2.info ='release dates' AND kt.kind ='movie' AND mi.movie_id = t.id AND it2.id = mi.info_type_id AND kt.id = t.kind_id AND mc.movie [...]
diff --git a/benchmarks/queries/imdb/13b.sql b/benchmarks/queries/imdb/13b.sql
new file mode 100644
index 0000000000..4b6f75ab0a
--- /dev/null
+++ b/benchmarks/queries/imdb/13b.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS producing_company, MIN(miidx.info) AS rating, MIN(t.title) AS movie_about_winning FROM company_name AS cn, company_type AS ct, info_type AS it, info_type AS it2, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_info_idx AS miidx, title AS t WHERE cn.country_code ='[us]' AND ct.kind ='production companies' AND it.info ='rating' AND it2.info ='release dates' AND kt.kind ='movie' AND t.title != '' AND (t.title LIKE '%Champion%' OR t.title LIKE '%Loser% [...]
diff --git a/benchmarks/queries/imdb/13c.sql b/benchmarks/queries/imdb/13c.sql
new file mode 100644
index 0000000000..9e8c92327b
--- /dev/null
+++ b/benchmarks/queries/imdb/13c.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS producing_company, MIN(miidx.info) AS rating, MIN(t.title) AS movie_about_winning FROM company_name AS cn, company_type AS ct, info_type AS it, info_type AS it2, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_info_idx AS miidx, title AS t WHERE cn.country_code ='[us]' AND ct.kind ='production companies' AND it.info ='rating' AND it2.info ='release dates' AND kt.kind ='movie' AND t.title != '' AND (t.title LIKE 'Champion%' OR t.title LIKE 'Loser%') [...]
diff --git a/benchmarks/queries/imdb/13d.sql b/benchmarks/queries/imdb/13d.sql
new file mode 100644
index 0000000000..a8bc567cab
--- /dev/null
+++ b/benchmarks/queries/imdb/13d.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS producing_company, MIN(miidx.info) AS rating, MIN(t.title) AS movie FROM company_name AS cn, company_type AS ct, info_type AS it, info_type AS it2, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_info_idx AS miidx, title AS t WHERE cn.country_code ='[us]' AND ct.kind ='production companies' AND it.info ='rating' AND it2.info ='release dates' AND kt.kind ='movie' AND mi.movie_id = t.id AND it2.id = mi.info_type_id AND kt.id = t.kind_id AND mc.movie_i [...]
diff --git a/benchmarks/queries/imdb/14a.sql b/benchmarks/queries/imdb/14a.sql
new file mode 100644
index 0000000000..af1a7c8983
--- /dev/null
+++ b/benchmarks/queries/imdb/14a.sql
@@ -0,0 +1 @@
+SELECT MIN(mi_idx.info) AS rating, MIN(t.title) AS northern_dark_movie FROM info_type AS it1, info_type AS it2, keyword AS k, kind_type AS kt, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE it1.info = 'countries' AND it2.info = 'rating' AND k.keyword in ('murder', 'murder-in-title', 'blood', 'violence') AND kt.kind = 'movie' AND mi.info IN ('Sweden', 'Norway', 'Germany', 'Denmark', 'Swedish', 'Denish', 'Norwegian', 'German', 'USA', 'American') AND m [...]
diff --git a/benchmarks/queries/imdb/14b.sql b/benchmarks/queries/imdb/14b.sql
new file mode 100644
index 0000000000..c606ebc73d
--- /dev/null
+++ b/benchmarks/queries/imdb/14b.sql
@@ -0,0 +1 @@
+SELECT MIN(mi_idx.info) AS rating, MIN(t.title) AS western_dark_production FROM info_type AS it1, info_type AS it2, keyword AS k, kind_type AS kt, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE it1.info = 'countries' AND it2.info = 'rating' AND k.keyword in ('murder', 'murder-in-title') AND kt.kind = 'movie' AND mi.info IN ('Sweden', 'Norway', 'Germany', 'Denmark', 'Swedish', 'Denish', 'Norwegian', 'German', 'USA', 'American') AND mi_idx.info > '6. [...]
diff --git a/benchmarks/queries/imdb/14c.sql b/benchmarks/queries/imdb/14c.sql
new file mode 100644
index 0000000000..2a6dffde26
--- /dev/null
+++ b/benchmarks/queries/imdb/14c.sql
@@ -0,0 +1 @@
+SELECT MIN(mi_idx.info) AS rating, MIN(t.title) AS north_european_dark_production FROM info_type AS it1, info_type AS it2, keyword AS k, kind_type AS kt, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE it1.info = 'countries' AND it2.info = 'rating' AND k.keyword is not null and k.keyword in ('murder', 'murder-in-title', 'blood', 'violence') AND kt.kind in ('movie', 'episode') AND mi.info IN ('Sweden', 'Norway', 'Germany', 'Denmark', 'Swedish', 'Danis [...]
diff --git a/benchmarks/queries/imdb/15a.sql b/benchmarks/queries/imdb/15a.sql
new file mode 100644
index 0000000000..1d052f0044
--- /dev/null
+++ b/benchmarks/queries/imdb/15a.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS release_date, MIN(t.title) AS internet_movie FROM aka_title AS at, company_name AS cn, company_type AS ct, info_type AS it1, keyword AS k, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, title AS t WHERE cn.country_code = '[us]' AND it1.info = 'release dates' AND mc.note like '%(200%)%' and mc.note like '%(worldwide)%' AND mi.note like '%internet%' AND mi.info like 'USA:% 200%' AND t.production_year > 2000 AND t.id = at.movie_id AND t.id = mi.mo [...]
diff --git a/benchmarks/queries/imdb/15b.sql b/benchmarks/queries/imdb/15b.sql
new file mode 100644
index 0000000000..21c81358fa
--- /dev/null
+++ b/benchmarks/queries/imdb/15b.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS release_date, MIN(t.title) AS youtube_movie FROM aka_title AS at, company_name AS cn, company_type AS ct, info_type AS it1, keyword AS k, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, title AS t WHERE cn.country_code = '[us]' and cn.name = 'YouTube' AND it1.info = 'release dates' AND mc.note like '%(200%)%' and mc.note like '%(worldwide)%' AND mi.note like '%internet%' AND mi.info like 'USA:% 200%' AND t.production_year between 2005 and 2010 A [...]
diff --git a/benchmarks/queries/imdb/15c.sql b/benchmarks/queries/imdb/15c.sql
new file mode 100644
index 0000000000..2d08c52039
--- /dev/null
+++ b/benchmarks/queries/imdb/15c.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS release_date, MIN(t.title) AS modern_american_internet_movie FROM aka_title AS at, company_name AS cn, company_type AS ct, info_type AS it1, keyword AS k, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, title AS t WHERE cn.country_code = '[us]' AND it1.info = 'release dates' AND mi.note like '%internet%' AND mi.info is not NULL and (mi.info like 'USA:% 199%' or mi.info like 'USA:% 200%') AND t.production_year > 1990 AND t.id = at.movie_id AND t.i [...]
diff --git a/benchmarks/queries/imdb/15d.sql b/benchmarks/queries/imdb/15d.sql
new file mode 100644
index 0000000000..040e9815d8
--- /dev/null
+++ b/benchmarks/queries/imdb/15d.sql
@@ -0,0 +1 @@
+SELECT MIN(at.title) AS aka_title, MIN(t.title) AS internet_movie_title FROM aka_title AS at, company_name AS cn, company_type AS ct, info_type AS it1, keyword AS k, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, title AS t WHERE cn.country_code = '[us]' AND it1.info = 'release dates' AND mi.note like '%internet%' AND t.production_year > 1990 AND t.id = at.movie_id AND t.id = mi.movie_id AND t.id = mk.movie_id AND t.id = mc.movie_id AND mk.movie_id = mi.movie_id AND mk [...]
diff --git a/benchmarks/queries/imdb/16a.sql b/benchmarks/queries/imdb/16a.sql
new file mode 100644
index 0000000000..aaa0020269
--- /dev/null
+++ b/benchmarks/queries/imdb/16a.sql
@@ -0,0 +1 @@
+SELECT MIN(an.name) AS cool_actor_pseudonym, MIN(t.title) AS series_named_after_char FROM aka_name AS an, cast_info AS ci, company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, name AS n, title AS t WHERE cn.country_code ='[us]' AND k.keyword ='character-name-in-title' AND t.episode_nr >= 50 AND t.episode_nr < 100 AND an.person_id = n.id AND n.id = ci.person_id AND ci.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND t.id = mc.movie_id AND mc.com [...]
diff --git a/benchmarks/queries/imdb/16b.sql b/benchmarks/queries/imdb/16b.sql
new file mode 100644
index 0000000000..c6c0bef319
--- /dev/null
+++ b/benchmarks/queries/imdb/16b.sql
@@ -0,0 +1 @@
+SELECT MIN(an.name) AS cool_actor_pseudonym, MIN(t.title) AS series_named_after_char FROM aka_name AS an, cast_info AS ci, company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, name AS n, title AS t WHERE cn.country_code ='[us]' AND k.keyword ='character-name-in-title' AND an.person_id = n.id AND n.id = ci.person_id AND ci.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND t.id = mc.movie_id AND mc.company_id = cn.id AND an.person_id = ci.person_i [...]
diff --git a/benchmarks/queries/imdb/16c.sql b/benchmarks/queries/imdb/16c.sql
new file mode 100644
index 0000000000..5c3b357521
--- /dev/null
+++ b/benchmarks/queries/imdb/16c.sql
@@ -0,0 +1 @@
+SELECT MIN(an.name) AS cool_actor_pseudonym, MIN(t.title) AS series_named_after_char FROM aka_name AS an, cast_info AS ci, company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, name AS n, title AS t WHERE cn.country_code ='[us]' AND k.keyword ='character-name-in-title' AND t.episode_nr < 100 AND an.person_id = n.id AND n.id = ci.person_id AND ci.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND t.id = mc.movie_id AND mc.company_id = cn.id AND an. [...]
diff --git a/benchmarks/queries/imdb/16d.sql b/benchmarks/queries/imdb/16d.sql
new file mode 100644
index 0000000000..c9e1b5f25c
--- /dev/null
+++ b/benchmarks/queries/imdb/16d.sql
@@ -0,0 +1 @@
+SELECT MIN(an.name) AS cool_actor_pseudonym, MIN(t.title) AS series_named_after_char FROM aka_name AS an, cast_info AS ci, company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, name AS n, title AS t WHERE cn.country_code ='[us]' AND k.keyword ='character-name-in-title' AND t.episode_nr >= 5 AND t.episode_nr < 100 AND an.person_id = n.id AND n.id = ci.person_id AND ci.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND t.id = mc.movie_id AND mc.comp [...]
diff --git a/benchmarks/queries/imdb/17a.sql b/benchmarks/queries/imdb/17a.sql
new file mode 100644
index 0000000000..e854a957e4
--- /dev/null
+++ b/benchmarks/queries/imdb/17a.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS member_in_charnamed_american_movie, MIN(n.name) AS a1 FROM cast_info AS ci, company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, name AS n, title AS t WHERE cn.country_code ='[us]' AND k.keyword ='character-name-in-title' AND n.name LIKE 'B%' AND n.id = ci.person_id AND ci.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND t.id = mc.movie_id AND mc.company_id = cn.id AND ci.movie_id = mc.movie_id AND ci.movie_id = mk.movie_ [...]
diff --git a/benchmarks/queries/imdb/17b.sql b/benchmarks/queries/imdb/17b.sql
new file mode 100644
index 0000000000..903f2196b2
--- /dev/null
+++ b/benchmarks/queries/imdb/17b.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS member_in_charnamed_movie, MIN(n.name) AS a1 FROM cast_info AS ci, company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, name AS n, title AS t WHERE k.keyword ='character-name-in-title' AND n.name LIKE 'Z%' AND n.id = ci.person_id AND ci.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND t.id = mc.movie_id AND mc.company_id = cn.id AND ci.movie_id = mc.movie_id AND ci.movie_id = mk.movie_id AND mc.movie_id = mk.movie_id;
diff --git a/benchmarks/queries/imdb/17c.sql b/benchmarks/queries/imdb/17c.sql
new file mode 100644
index 0000000000..a96faa0b43
--- /dev/null
+++ b/benchmarks/queries/imdb/17c.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS member_in_charnamed_movie, MIN(n.name) AS a1 FROM cast_info AS ci, company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, name AS n, title AS t WHERE k.keyword ='character-name-in-title' AND n.name LIKE 'X%' AND n.id = ci.person_id AND ci.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND t.id = mc.movie_id AND mc.company_id = cn.id AND ci.movie_id = mc.movie_id AND ci.movie_id = mk.movie_id AND mc.movie_id = mk.movie_id;
diff --git a/benchmarks/queries/imdb/17d.sql b/benchmarks/queries/imdb/17d.sql
new file mode 100644
index 0000000000..73e1f2c309
--- /dev/null
+++ b/benchmarks/queries/imdb/17d.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS member_in_charnamed_movie FROM cast_info AS ci, company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, name AS n, title AS t WHERE k.keyword ='character-name-in-title' AND n.name LIKE '%Bert%' AND n.id = ci.person_id AND ci.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND t.id = mc.movie_id AND mc.company_id = cn.id AND ci.movie_id = mc.movie_id AND ci.movie_id = mk.movie_id AND mc.movie_id = mk.movie_id;
diff --git a/benchmarks/queries/imdb/17e.sql b/benchmarks/queries/imdb/17e.sql
new file mode 100644
index 0000000000..65ea73ed05
--- /dev/null
+++ b/benchmarks/queries/imdb/17e.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS member_in_charnamed_movie FROM cast_info AS ci, company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, name AS n, title AS t WHERE cn.country_code ='[us]' AND k.keyword ='character-name-in-title' AND n.id = ci.person_id AND ci.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND t.id = mc.movie_id AND mc.company_id = cn.id AND ci.movie_id = mc.movie_id AND ci.movie_id = mk.movie_id AND mc.movie_id = mk.movie_id;
diff --git a/benchmarks/queries/imdb/17f.sql b/benchmarks/queries/imdb/17f.sql
new file mode 100644
index 0000000000..542233d63e
--- /dev/null
+++ b/benchmarks/queries/imdb/17f.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS member_in_charnamed_movie FROM cast_info AS ci, company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, name AS n, title AS t WHERE k.keyword ='character-name-in-title' AND n.name LIKE '%B%' AND n.id = ci.person_id AND ci.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND t.id = mc.movie_id AND mc.company_id = cn.id AND ci.movie_id = mc.movie_id AND ci.movie_id = mk.movie_id AND mc.movie_id = mk.movie_id;
diff --git a/benchmarks/queries/imdb/18a.sql b/benchmarks/queries/imdb/18a.sql
new file mode 100644
index 0000000000..275e04bdb1
--- /dev/null
+++ b/benchmarks/queries/imdb/18a.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS movie_budget, MIN(mi_idx.info) AS movie_votes, MIN(t.title) AS movie_title FROM cast_info AS ci, info_type AS it1, info_type AS it2, movie_info AS mi, movie_info_idx AS mi_idx, name AS n, title AS t WHERE ci.note in ('(producer)', '(executive producer)') AND it1.info = 'budget' AND it2.info = 'votes' AND n.gender = 'm' and n.name like '%Tim%' AND t.id = mi.movie_id AND t.id = mi_idx.movie_id AND t.id = ci.movie_id AND ci.movie_id = mi.movie_id AND ci.movie_id = [...]
diff --git a/benchmarks/queries/imdb/18b.sql b/benchmarks/queries/imdb/18b.sql
new file mode 100644
index 0000000000..3ae40ed93d
--- /dev/null
+++ b/benchmarks/queries/imdb/18b.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS movie_budget, MIN(mi_idx.info) AS movie_votes, MIN(t.title) AS movie_title FROM cast_info AS ci, info_type AS it1, info_type AS it2, movie_info AS mi, movie_info_idx AS mi_idx, name AS n, title AS t WHERE ci.note in ('(writer)', '(head writer)', '(written by)', '(story)', '(story editor)') AND it1.info = 'genres' AND it2.info = 'rating' AND mi.info in ('Horror', 'Thriller') and mi.note is NULL AND mi_idx.info > '8.0' AND n.gender is not null and n.gender = 'f [...]
diff --git a/benchmarks/queries/imdb/18c.sql b/benchmarks/queries/imdb/18c.sql
new file mode 100644
index 0000000000..01f28ea527
--- /dev/null
+++ b/benchmarks/queries/imdb/18c.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS movie_budget, MIN(mi_idx.info) AS movie_votes, MIN(t.title) AS movie_title FROM cast_info AS ci, info_type AS it1, info_type AS it2, movie_info AS mi, movie_info_idx AS mi_idx, name AS n, title AS t WHERE ci.note in ('(writer)', '(head writer)', '(written by)', '(story)', '(story editor)') AND it1.info = 'genres' AND it2.info = 'votes' AND mi.info in ('Horror', 'Action', 'Sci-Fi', 'Thriller', 'Crime', 'War') AND n.gender = 'm' AND t.id = mi.movie_id AND t.id = [...]
diff --git a/benchmarks/queries/imdb/19a.sql b/benchmarks/queries/imdb/19a.sql
new file mode 100644
index 0000000000..ceaae671fd
--- /dev/null
+++ b/benchmarks/queries/imdb/19a.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS voicing_actress, MIN(t.title) AS voiced_movie FROM aka_name AS an, char_name AS chn, cast_info AS ci, company_name AS cn, info_type AS it, movie_companies AS mc, movie_info AS mi, name AS n, role_type AS rt, title AS t WHERE ci.note in ('(voice)', '(voice: Japanese version)', '(voice) (uncredited)', '(voice: English version)') AND cn.country_code ='[us]' AND it.info = 'release dates' AND mc.note is not NULL and (mc.note like '%(USA)%' or mc.note like '%(worldwide [...]
diff --git a/benchmarks/queries/imdb/19b.sql b/benchmarks/queries/imdb/19b.sql
new file mode 100644
index 0000000000..62e852ba3e
--- /dev/null
+++ b/benchmarks/queries/imdb/19b.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS voicing_actress, MIN(t.title) AS kung_fu_panda FROM aka_name AS an, char_name AS chn, cast_info AS ci, company_name AS cn, info_type AS it, movie_companies AS mc, movie_info AS mi, name AS n, role_type AS rt, title AS t WHERE ci.note = '(voice)' AND cn.country_code ='[us]' AND it.info = 'release dates' AND mc.note like '%(200%)%' and (mc.note like '%(USA)%' or mc.note like '%(worldwide)%') AND mi.info is not null and (mi.info like 'Japan:%2007%' or mi.info like [...]
diff --git a/benchmarks/queries/imdb/19c.sql b/benchmarks/queries/imdb/19c.sql
new file mode 100644
index 0000000000..6885af5012
--- /dev/null
+++ b/benchmarks/queries/imdb/19c.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS voicing_actress, MIN(t.title) AS jap_engl_voiced_movie FROM aka_name AS an, char_name AS chn, cast_info AS ci, company_name AS cn, info_type AS it, movie_companies AS mc, movie_info AS mi, name AS n, role_type AS rt, title AS t WHERE ci.note in ('(voice)', '(voice: Japanese version)', '(voice) (uncredited)', '(voice: English version)') AND cn.country_code ='[us]' AND it.info = 'release dates' AND mi.info is not null and (mi.info like 'Japan:%200%' or mi.info like [...]
diff --git a/benchmarks/queries/imdb/19d.sql b/benchmarks/queries/imdb/19d.sql
new file mode 100644
index 0000000000..06fcc76ba7
--- /dev/null
+++ b/benchmarks/queries/imdb/19d.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS voicing_actress, MIN(t.title) AS jap_engl_voiced_movie FROM aka_name AS an, char_name AS chn, cast_info AS ci, company_name AS cn, info_type AS it, movie_companies AS mc, movie_info AS mi, name AS n, role_type AS rt, title AS t WHERE ci.note in ('(voice)', '(voice: Japanese version)', '(voice) (uncredited)', '(voice: English version)') AND cn.country_code ='[us]' AND it.info = 'release dates' AND n.gender ='f' AND rt.role ='actress' AND t.production_year > 2000 A [...]
diff --git a/benchmarks/queries/imdb/1a.sql b/benchmarks/queries/imdb/1a.sql
new file mode 100644
index 0000000000..07b3516388
--- /dev/null
+++ b/benchmarks/queries/imdb/1a.sql
@@ -0,0 +1 @@
+SELECT MIN(mc.note) AS production_note, MIN(t.title) AS movie_title, MIN(t.production_year) AS movie_year FROM company_type AS ct, info_type AS it, movie_companies AS mc, movie_info_idx AS mi_idx, title AS t WHERE ct.kind = 'production companies' AND it.info = 'top 250 rank' AND mc.note not like '%(as Metro-Goldwyn-Mayer Pictures)%' and (mc.note like '%(co-production)%' or mc.note like '%(presents)%') AND ct.id = mc.company_type_id AND t.id = mc.movie_id AND t.id = mi_idx.movie_id AND m [...]
diff --git a/benchmarks/queries/imdb/1b.sql b/benchmarks/queries/imdb/1b.sql
new file mode 100644
index 0000000000..f2901e8b52
--- /dev/null
+++ b/benchmarks/queries/imdb/1b.sql
@@ -0,0 +1 @@
+SELECT MIN(mc.note) AS production_note, MIN(t.title) AS movie_title, MIN(t.production_year) AS movie_year FROM company_type AS ct, info_type AS it, movie_companies AS mc, movie_info_idx AS mi_idx, title AS t WHERE ct.kind = 'production companies' AND it.info = 'bottom 10 rank' AND mc.note not like '%(as Metro-Goldwyn-Mayer Pictures)%' AND t.production_year between 2005 and 2010 AND ct.id = mc.company_type_id AND t.id = mc.movie_id AND t.id = mi_idx.movie_id AND mc.movie_id = mi_idx.movi [...]
diff --git a/benchmarks/queries/imdb/1c.sql b/benchmarks/queries/imdb/1c.sql
new file mode 100644
index 0000000000..94e66c30aa
--- /dev/null
+++ b/benchmarks/queries/imdb/1c.sql
@@ -0,0 +1 @@
+SELECT MIN(mc.note) AS production_note, MIN(t.title) AS movie_title, MIN(t.production_year) AS movie_year FROM company_type AS ct, info_type AS it, movie_companies AS mc, movie_info_idx AS mi_idx, title AS t WHERE ct.kind = 'production companies' AND it.info = 'top 250 rank' AND mc.note not like '%(as Metro-Goldwyn-Mayer Pictures)%' and (mc.note like '%(co-production)%') AND t.production_year >2010 AND ct.id = mc.company_type_id AND t.id = mc.movie_id AND t.id = mi_idx.movie_id AND mc.m [...]
diff --git a/benchmarks/queries/imdb/1d.sql b/benchmarks/queries/imdb/1d.sql
new file mode 100644
index 0000000000..52f58e80c8
--- /dev/null
+++ b/benchmarks/queries/imdb/1d.sql
@@ -0,0 +1 @@
+SELECT MIN(mc.note) AS production_note, MIN(t.title) AS movie_title, MIN(t.production_year) AS movie_year FROM company_type AS ct, info_type AS it, movie_companies AS mc, movie_info_idx AS mi_idx, title AS t WHERE ct.kind = 'production companies' AND it.info = 'bottom 10 rank' AND mc.note not like '%(as Metro-Goldwyn-Mayer Pictures)%' AND t.production_year >2000 AND ct.id = mc.company_type_id AND t.id = mc.movie_id AND t.id = mi_idx.movie_id AND mc.movie_id = mi_idx.movie_id AND it.id = [...]
diff --git a/benchmarks/queries/imdb/20a.sql b/benchmarks/queries/imdb/20a.sql
new file mode 100644
index 0000000000..2a1c269d6a
--- /dev/null
+++ b/benchmarks/queries/imdb/20a.sql
@@ -0,0 +1 @@
+SELECT MIN(t.title) AS complete_downey_ironman_movie FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, char_name AS chn, cast_info AS ci, keyword AS k, kind_type AS kt, movie_keyword AS mk, name AS n, title AS t WHERE cct1.kind = 'cast' AND cct2.kind like '%complete%' AND chn.name not like '%Sherlock%' and (chn.name like '%Tony%Stark%' or chn.name like '%Iron%Man%') AND k.keyword in ('superhero', 'sequel', 'second-part', 'marvel-comics', 'based-on-comic', 'tv- [...]
diff --git a/benchmarks/queries/imdb/20b.sql b/benchmarks/queries/imdb/20b.sql
new file mode 100644
index 0000000000..4c2455a52e
--- /dev/null
+++ b/benchmarks/queries/imdb/20b.sql
@@ -0,0 +1 @@
+SELECT MIN(t.title) AS complete_downey_ironman_movie FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, char_name AS chn, cast_info AS ci, keyword AS k, kind_type AS kt, movie_keyword AS mk, name AS n, title AS t WHERE cct1.kind = 'cast' AND cct2.kind like '%complete%' AND chn.name not like '%Sherlock%' and (chn.name like '%Tony%Stark%' or chn.name like '%Iron%Man%') AND k.keyword in ('superhero', 'sequel', 'second-part', 'marvel-comics', 'based-on-comic', 'tv- [...]
diff --git a/benchmarks/queries/imdb/20c.sql b/benchmarks/queries/imdb/20c.sql
new file mode 100644
index 0000000000..b85b22f6b4
--- /dev/null
+++ b/benchmarks/queries/imdb/20c.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS cast_member, MIN(t.title) AS complete_dynamic_hero_movie FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, char_name AS chn, cast_info AS ci, keyword AS k, kind_type AS kt, movie_keyword AS mk, name AS n, title AS t WHERE cct1.kind = 'cast' AND cct2.kind like '%complete%' AND chn.name is not NULL and (chn.name like '%man%' or chn.name like '%Man%') AND k.keyword in ('superhero', 'marvel-comics', 'based-on-comic', 'tv-special', 'fight', 'v [...]
diff --git a/benchmarks/queries/imdb/21a.sql b/benchmarks/queries/imdb/21a.sql
new file mode 100644
index 0000000000..8a66a00be6
--- /dev/null
+++ b/benchmarks/queries/imdb/21a.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS company_name, MIN(lt.link) AS link_type, MIN(t.title) AS western_follow_up FROM company_name AS cn, company_type AS ct, keyword AS k, link_type AS lt, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, movie_link AS ml, title AS t WHERE cn.country_code !='[pl]' AND (cn.name LIKE '%Film%' OR cn.name LIKE '%Warner%') AND ct.kind ='production companies' AND k.keyword ='sequel' AND lt.link LIKE '%follow%' AND mc.note IS NULL AND mi.info IN ('Sweden', 'Norway [...]
diff --git a/benchmarks/queries/imdb/21b.sql b/benchmarks/queries/imdb/21b.sql
new file mode 100644
index 0000000000..90d3a5a4c0
--- /dev/null
+++ b/benchmarks/queries/imdb/21b.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS company_name, MIN(lt.link) AS link_type, MIN(t.title) AS german_follow_up FROM company_name AS cn, company_type AS ct, keyword AS k, link_type AS lt, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, movie_link AS ml, title AS t WHERE cn.country_code !='[pl]' AND (cn.name LIKE '%Film%' OR cn.name LIKE '%Warner%') AND ct.kind ='production companies' AND k.keyword ='sequel' AND lt.link LIKE '%follow%' AND mc.note IS NULL AND mi.info IN ('Germany', 'German [...]
diff --git a/benchmarks/queries/imdb/21c.sql b/benchmarks/queries/imdb/21c.sql
new file mode 100644
index 0000000000..16a42ae6f4
--- /dev/null
+++ b/benchmarks/queries/imdb/21c.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS company_name, MIN(lt.link) AS link_type, MIN(t.title) AS western_follow_up FROM company_name AS cn, company_type AS ct, keyword AS k, link_type AS lt, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, movie_link AS ml, title AS t WHERE cn.country_code !='[pl]' AND (cn.name LIKE '%Film%' OR cn.name LIKE '%Warner%') AND ct.kind ='production companies' AND k.keyword ='sequel' AND lt.link LIKE '%follow%' AND mc.note IS NULL AND mi.info IN ('Sweden', 'Norway [...]
diff --git a/benchmarks/queries/imdb/22a.sql b/benchmarks/queries/imdb/22a.sql
new file mode 100644
index 0000000000..e513799698
--- /dev/null
+++ b/benchmarks/queries/imdb/22a.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS movie_company, MIN(mi_idx.info) AS rating, MIN(t.title) AS western_violent_movie FROM company_name AS cn, company_type AS ct, info_type AS it1, info_type AS it2, keyword AS k, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE cn.country_code != '[us]' AND it1.info = 'countries' AND it2.info = 'rating' AND k.keyword in ('murder', 'murder-in-title', 'blood', 'violence') AND kt.kind in ('movi [...]
diff --git a/benchmarks/queries/imdb/22b.sql b/benchmarks/queries/imdb/22b.sql
new file mode 100644
index 0000000000..f98d0ea809
--- /dev/null
+++ b/benchmarks/queries/imdb/22b.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS movie_company, MIN(mi_idx.info) AS rating, MIN(t.title) AS western_violent_movie FROM company_name AS cn, company_type AS ct, info_type AS it1, info_type AS it2, keyword AS k, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE cn.country_code != '[us]' AND it1.info = 'countries' AND it2.info = 'rating' AND k.keyword in ('murder', 'murder-in-title', 'blood', 'violence') AND kt.kind in ('movi [...]
diff --git a/benchmarks/queries/imdb/22c.sql b/benchmarks/queries/imdb/22c.sql
new file mode 100644
index 0000000000..cf757956e0
--- /dev/null
+++ b/benchmarks/queries/imdb/22c.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS movie_company, MIN(mi_idx.info) AS rating, MIN(t.title) AS western_violent_movie FROM company_name AS cn, company_type AS ct, info_type AS it1, info_type AS it2, keyword AS k, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE cn.country_code != '[us]' AND it1.info = 'countries' AND it2.info = 'rating' AND k.keyword in ('murder', 'murder-in-title', 'blood', 'violence') AND kt.kind in ('movi [...]
diff --git a/benchmarks/queries/imdb/22d.sql b/benchmarks/queries/imdb/22d.sql
new file mode 100644
index 0000000000..a47feeb051
--- /dev/null
+++ b/benchmarks/queries/imdb/22d.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS movie_company, MIN(mi_idx.info) AS rating, MIN(t.title) AS western_violent_movie FROM company_name AS cn, company_type AS ct, info_type AS it1, info_type AS it2, keyword AS k, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE cn.country_code != '[us]' AND it1.info = 'countries' AND it2.info = 'rating' AND k.keyword in ('murder', 'murder-in-title', 'blood', 'violence') AND kt.kind in ('movi [...]
diff --git a/benchmarks/queries/imdb/23a.sql b/benchmarks/queries/imdb/23a.sql
new file mode 100644
index 0000000000..724da913b5
--- /dev/null
+++ b/benchmarks/queries/imdb/23a.sql
@@ -0,0 +1 @@
+SELECT MIN(kt.kind) AS movie_kind, MIN(t.title) AS complete_us_internet_movie FROM complete_cast AS cc, comp_cast_type AS cct1, company_name AS cn, company_type AS ct, info_type AS it1, keyword AS k, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, title AS t WHERE cct1.kind = 'complete+verified' AND cn.country_code = '[us]' AND it1.info = 'release dates' AND kt.kind in ('movie') AND mi.note like '%internet%' AND mi.info is not NULL and (mi.info like ' [...]
diff --git a/benchmarks/queries/imdb/23b.sql b/benchmarks/queries/imdb/23b.sql
new file mode 100644
index 0000000000..e39f0ecc28
--- /dev/null
+++ b/benchmarks/queries/imdb/23b.sql
@@ -0,0 +1 @@
+SELECT MIN(kt.kind) AS movie_kind, MIN(t.title) AS complete_nerdy_internet_movie FROM complete_cast AS cc, comp_cast_type AS cct1, company_name AS cn, company_type AS ct, info_type AS it1, keyword AS k, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, title AS t WHERE cct1.kind = 'complete+verified' AND cn.country_code = '[us]' AND it1.info = 'release dates' AND k.keyword in ('nerd', 'loner', 'alienation', 'dignity') AND kt.kind in ('movie') AND mi.note [...]
diff --git a/benchmarks/queries/imdb/23c.sql b/benchmarks/queries/imdb/23c.sql
new file mode 100644
index 0000000000..839d762d05
--- /dev/null
+++ b/benchmarks/queries/imdb/23c.sql
@@ -0,0 +1 @@
+SELECT MIN(kt.kind) AS movie_kind, MIN(t.title) AS complete_us_internet_movie FROM complete_cast AS cc, comp_cast_type AS cct1, company_name AS cn, company_type AS ct, info_type AS it1, keyword AS k, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, title AS t WHERE cct1.kind = 'complete+verified' AND cn.country_code = '[us]' AND it1.info = 'release dates' AND kt.kind in ('movie', 'tv movie', 'video movie', 'video game') AND mi.note like '%internet%' AND [...]
diff --git a/benchmarks/queries/imdb/24a.sql b/benchmarks/queries/imdb/24a.sql
new file mode 100644
index 0000000000..8f10621e02
--- /dev/null
+++ b/benchmarks/queries/imdb/24a.sql
@@ -0,0 +1 @@
+SELECT MIN(chn.name) AS voiced_char_name, MIN(n.name) AS voicing_actress_name, MIN(t.title) AS voiced_action_movie_jap_eng FROM aka_name AS an, char_name AS chn, cast_info AS ci, company_name AS cn, info_type AS it, keyword AS k, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, name AS n, role_type AS rt, title AS t WHERE ci.note in ('(voice)', '(voice: Japanese version)', '(voice) (uncredited)', '(voice: English version)') AND cn.country_code ='[us]' AND it.info = 'releas [...]
diff --git a/benchmarks/queries/imdb/24b.sql b/benchmarks/queries/imdb/24b.sql
new file mode 100644
index 0000000000..d8a2836000
--- /dev/null
+++ b/benchmarks/queries/imdb/24b.sql
@@ -0,0 +1 @@
+SELECT MIN(chn.name) AS voiced_char_name, MIN(n.name) AS voicing_actress_name, MIN(t.title) AS kung_fu_panda FROM aka_name AS an, char_name AS chn, cast_info AS ci, company_name AS cn, info_type AS it, keyword AS k, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, name AS n, role_type AS rt, title AS t WHERE ci.note in ('(voice)', '(voice: Japanese version)', '(voice) (uncredited)', '(voice: English version)') AND cn.country_code ='[us]' AND cn.name = 'DreamWorks Animation [...]
diff --git a/benchmarks/queries/imdb/25a.sql b/benchmarks/queries/imdb/25a.sql
new file mode 100644
index 0000000000..bc55cc01d2
--- /dev/null
+++ b/benchmarks/queries/imdb/25a.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS movie_budget, MIN(mi_idx.info) AS movie_votes, MIN(n.name) AS male_writer, MIN(t.title) AS violent_movie_title FROM cast_info AS ci, info_type AS it1, info_type AS it2, keyword AS k, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, name AS n, title AS t WHERE ci.note in ('(writer)', '(head writer)', '(written by)', '(story)', '(story editor)') AND it1.info = 'genres' AND it2.info = 'votes' AND k.keyword in ('murder', 'blood', 'gore', 'death', 'f [...]
diff --git a/benchmarks/queries/imdb/25b.sql b/benchmarks/queries/imdb/25b.sql
new file mode 100644
index 0000000000..3457655bb9
--- /dev/null
+++ b/benchmarks/queries/imdb/25b.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS movie_budget, MIN(mi_idx.info) AS movie_votes, MIN(n.name) AS male_writer, MIN(t.title) AS violent_movie_title FROM cast_info AS ci, info_type AS it1, info_type AS it2, keyword AS k, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, name AS n, title AS t WHERE ci.note in ('(writer)', '(head writer)', '(written by)', '(story)', '(story editor)') AND it1.info = 'genres' AND it2.info = 'votes' AND k.keyword in ('murder', 'blood', 'gore', 'death', 'f [...]
diff --git a/benchmarks/queries/imdb/25c.sql b/benchmarks/queries/imdb/25c.sql
new file mode 100644
index 0000000000..cf56a313d8
--- /dev/null
+++ b/benchmarks/queries/imdb/25c.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS movie_budget, MIN(mi_idx.info) AS movie_votes, MIN(n.name) AS male_writer, MIN(t.title) AS violent_movie_title FROM cast_info AS ci, info_type AS it1, info_type AS it2, keyword AS k, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, name AS n, title AS t WHERE ci.note in ('(writer)', '(head writer)', '(written by)', '(story)', '(story editor)') AND it1.info = 'genres' AND it2.info = 'votes' AND k.keyword in ('murder', 'violence', 'blood', 'gore', [...]
diff --git a/benchmarks/queries/imdb/26a.sql b/benchmarks/queries/imdb/26a.sql
new file mode 100644
index 0000000000..b431f204c6
--- /dev/null
+++ b/benchmarks/queries/imdb/26a.sql
@@ -0,0 +1 @@
+SELECT MIN(chn.name) AS character_name, MIN(mi_idx.info) AS rating, MIN(n.name) AS playing_actor, MIN(t.title) AS complete_hero_movie FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, char_name AS chn, cast_info AS ci, info_type AS it2, keyword AS k, kind_type AS kt, movie_info_idx AS mi_idx, movie_keyword AS mk, name AS n, title AS t WHERE cct1.kind = 'cast' AND cct2.kind like '%complete%' AND chn.name is not NULL and (chn.name like '%man%' or chn.name like '% [...]
diff --git a/benchmarks/queries/imdb/26b.sql b/benchmarks/queries/imdb/26b.sql
new file mode 100644
index 0000000000..882d234d77
--- /dev/null
+++ b/benchmarks/queries/imdb/26b.sql
@@ -0,0 +1 @@
+SELECT MIN(chn.name) AS character_name, MIN(mi_idx.info) AS rating, MIN(t.title) AS complete_hero_movie FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, char_name AS chn, cast_info AS ci, info_type AS it2, keyword AS k, kind_type AS kt, movie_info_idx AS mi_idx, movie_keyword AS mk, name AS n, title AS t WHERE cct1.kind = 'cast' AND cct2.kind like '%complete%' AND chn.name is not NULL and (chn.name like '%man%' or chn.name like '%Man%') AND it2.info = 'rating [...]
diff --git a/benchmarks/queries/imdb/26c.sql b/benchmarks/queries/imdb/26c.sql
new file mode 100644
index 0000000000..4b9eae0b76
--- /dev/null
+++ b/benchmarks/queries/imdb/26c.sql
@@ -0,0 +1 @@
+SELECT MIN(chn.name) AS character_name, MIN(mi_idx.info) AS rating, MIN(t.title) AS complete_hero_movie FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, char_name AS chn, cast_info AS ci, info_type AS it2, keyword AS k, kind_type AS kt, movie_info_idx AS mi_idx, movie_keyword AS mk, name AS n, title AS t WHERE cct1.kind = 'cast' AND cct2.kind like '%complete%' AND chn.name is not NULL and (chn.name like '%man%' or chn.name like '%Man%') AND it2.info = 'rating [...]
diff --git a/benchmarks/queries/imdb/27a.sql b/benchmarks/queries/imdb/27a.sql
new file mode 100644
index 0000000000..239673cd81
--- /dev/null
+++ b/benchmarks/queries/imdb/27a.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS producing_company, MIN(lt.link) AS link_type, MIN(t.title) AS complete_western_sequel FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, company_name AS cn, company_type AS ct, keyword AS k, link_type AS lt, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, movie_link AS ml, title AS t WHERE cct1.kind in ('cast', 'crew') AND cct2.kind = 'complete' AND cn.country_code !='[pl]' AND (cn.name LIKE '%Film%' OR cn.name LIKE '%Warner%' [...]
diff --git a/benchmarks/queries/imdb/27b.sql b/benchmarks/queries/imdb/27b.sql
new file mode 100644
index 0000000000..4bf85260f2
--- /dev/null
+++ b/benchmarks/queries/imdb/27b.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS producing_company, MIN(lt.link) AS link_type, MIN(t.title) AS complete_western_sequel FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, company_name AS cn, company_type AS ct, keyword AS k, link_type AS lt, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, movie_link AS ml, title AS t WHERE cct1.kind in ('cast', 'crew') AND cct2.kind = 'complete' AND cn.country_code !='[pl]' AND (cn.name LIKE '%Film%' OR cn.name LIKE '%Warner%' [...]
diff --git a/benchmarks/queries/imdb/27c.sql b/benchmarks/queries/imdb/27c.sql
new file mode 100644
index 0000000000..dc26ebff68
--- /dev/null
+++ b/benchmarks/queries/imdb/27c.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS producing_company, MIN(lt.link) AS link_type, MIN(t.title) AS complete_western_sequel FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, company_name AS cn, company_type AS ct, keyword AS k, link_type AS lt, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, movie_link AS ml, title AS t WHERE cct1.kind = 'cast' AND cct2.kind like 'complete%' AND cn.country_code !='[pl]' AND (cn.name LIKE '%Film%' OR cn.name LIKE '%Warner%') AND c [...]
diff --git a/benchmarks/queries/imdb/28a.sql b/benchmarks/queries/imdb/28a.sql
new file mode 100644
index 0000000000..8cb1177386
--- /dev/null
+++ b/benchmarks/queries/imdb/28a.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS movie_company, MIN(mi_idx.info) AS rating, MIN(t.title) AS complete_euro_dark_movie FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, company_name AS cn, company_type AS ct, info_type AS it1, info_type AS it2, keyword AS k, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE cct1.kind = 'crew' AND cct2.kind != 'complete+verified' AND cn.country_code != '[us]' AND it1.in [...]
diff --git a/benchmarks/queries/imdb/28b.sql b/benchmarks/queries/imdb/28b.sql
new file mode 100644
index 0000000000..10f43c8982
--- /dev/null
+++ b/benchmarks/queries/imdb/28b.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS movie_company, MIN(mi_idx.info) AS rating, MIN(t.title) AS complete_euro_dark_movie FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, company_name AS cn, company_type AS ct, info_type AS it1, info_type AS it2, keyword AS k, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE cct1.kind = 'crew' AND cct2.kind != 'complete+verified' AND cn.country_code != '[us]' AND it1.in [...]
diff --git a/benchmarks/queries/imdb/28c.sql b/benchmarks/queries/imdb/28c.sql
new file mode 100644
index 0000000000..6b2e4047ae
--- /dev/null
+++ b/benchmarks/queries/imdb/28c.sql
@@ -0,0 +1 @@
+SELECT MIN(cn.name) AS movie_company, MIN(mi_idx.info) AS rating, MIN(t.title) AS complete_euro_dark_movie FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, company_name AS cn, company_type AS ct, info_type AS it1, info_type AS it2, keyword AS k, kind_type AS kt, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE cct1.kind = 'cast' AND cct2.kind = 'complete' AND cn.country_code != '[us]' AND it1.info = 'cou [...]
diff --git a/benchmarks/queries/imdb/29a.sql b/benchmarks/queries/imdb/29a.sql
new file mode 100644
index 0000000000..3033acbe6c
--- /dev/null
+++ b/benchmarks/queries/imdb/29a.sql
@@ -0,0 +1 @@
+SELECT MIN(chn.name) AS voiced_char, MIN(n.name) AS voicing_actress, MIN(t.title) AS voiced_animation FROM aka_name AS an, complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, char_name AS chn, cast_info AS ci, company_name AS cn, info_type AS it, info_type AS it3, keyword AS k, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, name AS n, person_info AS pi, role_type AS rt, title AS t WHERE cct1.kind ='cast' AND cct2.kind ='complete+verified' AND chn.name = [...]
diff --git a/benchmarks/queries/imdb/29b.sql b/benchmarks/queries/imdb/29b.sql
new file mode 100644
index 0000000000..88d50fc7b7
--- /dev/null
+++ b/benchmarks/queries/imdb/29b.sql
@@ -0,0 +1 @@
+SELECT MIN(chn.name) AS voiced_char, MIN(n.name) AS voicing_actress, MIN(t.title) AS voiced_animation FROM aka_name AS an, complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, char_name AS chn, cast_info AS ci, company_name AS cn, info_type AS it, info_type AS it3, keyword AS k, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, name AS n, person_info AS pi, role_type AS rt, title AS t WHERE cct1.kind ='cast' AND cct2.kind ='complete+verified' AND chn.name = [...]
diff --git a/benchmarks/queries/imdb/29c.sql b/benchmarks/queries/imdb/29c.sql
new file mode 100644
index 0000000000..cb95178182
--- /dev/null
+++ b/benchmarks/queries/imdb/29c.sql
@@ -0,0 +1 @@
+SELECT MIN(chn.name) AS voiced_char, MIN(n.name) AS voicing_actress, MIN(t.title) AS voiced_animation FROM aka_name AS an, complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, char_name AS chn, cast_info AS ci, company_name AS cn, info_type AS it, info_type AS it3, keyword AS k, movie_companies AS mc, movie_info AS mi, movie_keyword AS mk, name AS n, person_info AS pi, role_type AS rt, title AS t WHERE cct1.kind ='cast' AND cct2.kind ='complete+verified' AND ci.note in [...]
diff --git a/benchmarks/queries/imdb/2a.sql b/benchmarks/queries/imdb/2a.sql
new file mode 100644
index 0000000000..f3ef4db75f
--- /dev/null
+++ b/benchmarks/queries/imdb/2a.sql
@@ -0,0 +1 @@
+SELECT MIN(t.title) AS movie_title FROM company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, title AS t WHERE cn.country_code ='[de]' AND k.keyword ='character-name-in-title' AND cn.id = mc.company_id AND mc.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND mc.movie_id = mk.movie_id;
diff --git a/benchmarks/queries/imdb/2b.sql b/benchmarks/queries/imdb/2b.sql
new file mode 100644
index 0000000000..82b2123fbc
--- /dev/null
+++ b/benchmarks/queries/imdb/2b.sql
@@ -0,0 +1 @@
+SELECT MIN(t.title) AS movie_title FROM company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, title AS t WHERE cn.country_code ='[nl]' AND k.keyword ='character-name-in-title' AND cn.id = mc.company_id AND mc.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND mc.movie_id = mk.movie_id;
diff --git a/benchmarks/queries/imdb/2c.sql b/benchmarks/queries/imdb/2c.sql
new file mode 100644
index 0000000000..b5f9b75dd6
--- /dev/null
+++ b/benchmarks/queries/imdb/2c.sql
@@ -0,0 +1 @@
+SELECT MIN(t.title) AS movie_title FROM company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, title AS t WHERE cn.country_code ='[sm]' AND k.keyword ='character-name-in-title' AND cn.id = mc.company_id AND mc.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND mc.movie_id = mk.movie_id;
diff --git a/benchmarks/queries/imdb/2d.sql b/benchmarks/queries/imdb/2d.sql
new file mode 100644
index 0000000000..4a27919465
--- /dev/null
+++ b/benchmarks/queries/imdb/2d.sql
@@ -0,0 +1 @@
+SELECT MIN(t.title) AS movie_title FROM company_name AS cn, keyword AS k, movie_companies AS mc, movie_keyword AS mk, title AS t WHERE cn.country_code ='[us]' AND k.keyword ='character-name-in-title' AND cn.id = mc.company_id AND mc.movie_id = t.id AND t.id = mk.movie_id AND mk.keyword_id = k.id AND mc.movie_id = mk.movie_id;
diff --git a/benchmarks/queries/imdb/30a.sql b/benchmarks/queries/imdb/30a.sql
new file mode 100644
index 0000000000..698872fa83
--- /dev/null
+++ b/benchmarks/queries/imdb/30a.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS movie_budget, MIN(mi_idx.info) AS movie_votes, MIN(n.name) AS writer, MIN(t.title) AS complete_violent_movie FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, cast_info AS ci, info_type AS it1, info_type AS it2, keyword AS k, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, name AS n, title AS t WHERE cct1.kind in ('cast', 'crew') AND cct2.kind ='complete+verified' AND ci.note in ('(writer)', '(head writer)', '(written by) [...]
diff --git a/benchmarks/queries/imdb/30b.sql b/benchmarks/queries/imdb/30b.sql
new file mode 100644
index 0000000000..5fdb849349
--- /dev/null
+++ b/benchmarks/queries/imdb/30b.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS movie_budget, MIN(mi_idx.info) AS movie_votes, MIN(n.name) AS writer, MIN(t.title) AS complete_gore_movie FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, cast_info AS ci, info_type AS it1, info_type AS it2, keyword AS k, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, name AS n, title AS t WHERE cct1.kind in ('cast', 'crew') AND cct2.kind ='complete+verified' AND ci.note in ('(writer)', '(head writer)', '(written by)', [...]
diff --git a/benchmarks/queries/imdb/30c.sql b/benchmarks/queries/imdb/30c.sql
new file mode 100644
index 0000000000..a18087e392
--- /dev/null
+++ b/benchmarks/queries/imdb/30c.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS movie_budget, MIN(mi_idx.info) AS movie_votes, MIN(n.name) AS writer, MIN(t.title) AS complete_violent_movie FROM complete_cast AS cc, comp_cast_type AS cct1, comp_cast_type AS cct2, cast_info AS ci, info_type AS it1, info_type AS it2, keyword AS k, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, name AS n, title AS t WHERE cct1.kind = 'cast' AND cct2.kind ='complete+verified' AND ci.note in ('(writer)', '(head writer)', '(written by)', '(story) [...]
diff --git a/benchmarks/queries/imdb/31a.sql b/benchmarks/queries/imdb/31a.sql
new file mode 100644
index 0000000000..7dd855011f
--- /dev/null
+++ b/benchmarks/queries/imdb/31a.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS movie_budget, MIN(mi_idx.info) AS movie_votes, MIN(n.name) AS writer, MIN(t.title) AS violent_liongate_movie FROM cast_info AS ci, company_name AS cn, info_type AS it1, info_type AS it2, keyword AS k, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, name AS n, title AS t WHERE ci.note in ('(writer)', '(head writer)', '(written by)', '(story)', '(story editor)') AND cn.name like 'Lionsgate%' AND it1.info = 'genres' AND it2.i [...]
diff --git a/benchmarks/queries/imdb/31b.sql b/benchmarks/queries/imdb/31b.sql
new file mode 100644
index 0000000000..3be5680f7d
--- /dev/null
+++ b/benchmarks/queries/imdb/31b.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS movie_budget, MIN(mi_idx.info) AS movie_votes, MIN(n.name) AS writer, MIN(t.title) AS violent_liongate_movie FROM cast_info AS ci, company_name AS cn, info_type AS it1, info_type AS it2, keyword AS k, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, name AS n, title AS t WHERE ci.note in ('(writer)', '(head writer)', '(written by)', '(story)', '(story editor)') AND cn.name like 'Lionsgate%' AND it1.info = 'genres' AND it2.i [...]
diff --git a/benchmarks/queries/imdb/31c.sql b/benchmarks/queries/imdb/31c.sql
new file mode 100644
index 0000000000..156ea2d5ee
--- /dev/null
+++ b/benchmarks/queries/imdb/31c.sql
@@ -0,0 +1 @@
+SELECT MIN(mi.info) AS movie_budget, MIN(mi_idx.info) AS movie_votes, MIN(n.name) AS writer, MIN(t.title) AS violent_liongate_movie FROM cast_info AS ci, company_name AS cn, info_type AS it1, info_type AS it2, keyword AS k, movie_companies AS mc, movie_info AS mi, movie_info_idx AS mi_idx, movie_keyword AS mk, name AS n, title AS t WHERE ci.note in ('(writer)', '(head writer)', '(written by)', '(story)', '(story editor)') AND cn.name like 'Lionsgate%' AND it1.info = 'genres' AND it2.i [...]
diff --git a/benchmarks/queries/imdb/32a.sql b/benchmarks/queries/imdb/32a.sql
new file mode 100644
index 0000000000..9647fb7106
--- /dev/null
+++ b/benchmarks/queries/imdb/32a.sql
@@ -0,0 +1 @@
+SELECT MIN(lt.link) AS link_type, MIN(t1.title) AS first_movie, MIN(t2.title) AS second_movie FROM keyword AS k, link_type AS lt, movie_keyword AS mk, movie_link AS ml, title AS t1, title AS t2 WHERE k.keyword ='10,000-mile-club' AND mk.keyword_id = k.id AND t1.id = mk.movie_id AND ml.movie_id = t1.id AND ml.linked_movie_id = t2.id AND lt.id = ml.link_type_id AND mk.movie_id = t1.id;
diff --git a/benchmarks/queries/imdb/32b.sql b/benchmarks/queries/imdb/32b.sql
new file mode 100644
index 0000000000..6d096ab434
--- /dev/null
+++ b/benchmarks/queries/imdb/32b.sql
@@ -0,0 +1 @@
+SELECT MIN(lt.link) AS link_type, MIN(t1.title) AS first_movie, MIN(t2.title) AS second_movie FROM keyword AS k, link_type AS lt, movie_keyword AS mk, movie_link AS ml, title AS t1, title AS t2 WHERE k.keyword ='character-name-in-title' AND mk.keyword_id = k.id AND t1.id = mk.movie_id AND ml.movie_id = t1.id AND ml.linked_movie_id = t2.id AND lt.id = ml.link_type_id AND mk.movie_id = t1.id;
diff --git a/benchmarks/queries/imdb/33a.sql b/benchmarks/queries/imdb/33a.sql
new file mode 100644
index 0000000000..24aac4e207
--- /dev/null
+++ b/benchmarks/queries/imdb/33a.sql
@@ -0,0 +1 @@
+SELECT MIN(cn1.name) AS first_company, MIN(cn2.name) AS second_company, MIN(mi_idx1.info) AS first_rating, MIN(mi_idx2.info) AS second_rating, MIN(t1.title) AS first_movie, MIN(t2.title) AS second_movie FROM company_name AS cn1, company_name AS cn2, info_type AS it1, info_type AS it2, kind_type AS kt1, kind_type AS kt2, link_type AS lt, movie_companies AS mc1, movie_companies AS mc2, movie_info_idx AS mi_idx1, movie_info_idx AS mi_idx2, movie_link AS ml, title AS t1, title AS t2 WHERE cn [...]
diff --git a/benchmarks/queries/imdb/33b.sql b/benchmarks/queries/imdb/33b.sql
new file mode 100644
index 0000000000..fe6fd75a69
--- /dev/null
+++ b/benchmarks/queries/imdb/33b.sql
@@ -0,0 +1 @@
+SELECT MIN(cn1.name) AS first_company, MIN(cn2.name) AS second_company, MIN(mi_idx1.info) AS first_rating, MIN(mi_idx2.info) AS second_rating, MIN(t1.title) AS first_movie, MIN(t2.title) AS second_movie FROM company_name AS cn1, company_name AS cn2, info_type AS it1, info_type AS it2, kind_type AS kt1, kind_type AS kt2, link_type AS lt, movie_companies AS mc1, movie_companies AS mc2, movie_info_idx AS mi_idx1, movie_info_idx AS mi_idx2, movie_link AS ml, title AS t1, title AS t2 WHERE cn [...]
diff --git a/benchmarks/queries/imdb/33c.sql b/benchmarks/queries/imdb/33c.sql
new file mode 100644
index 0000000000..c9f0907d3f
--- /dev/null
+++ b/benchmarks/queries/imdb/33c.sql
@@ -0,0 +1 @@
+SELECT MIN(cn1.name) AS first_company, MIN(cn2.name) AS second_company, MIN(mi_idx1.info) AS first_rating, MIN(mi_idx2.info) AS second_rating, MIN(t1.title) AS first_movie, MIN(t2.title) AS second_movie FROM company_name AS cn1, company_name AS cn2, info_type AS it1, info_type AS it2, kind_type AS kt1, kind_type AS kt2, link_type AS lt, movie_companies AS mc1, movie_companies AS mc2, movie_info_idx AS mi_idx1, movie_info_idx AS mi_idx2, movie_link AS ml, title AS t1, title AS t2 WHERE cn [...]
diff --git a/benchmarks/queries/imdb/3a.sql b/benchmarks/queries/imdb/3a.sql
new file mode 100644
index 0000000000..231c957be2
--- /dev/null
+++ b/benchmarks/queries/imdb/3a.sql
@@ -0,0 +1 @@
+SELECT MIN(t.title) AS movie_title FROM keyword AS k, movie_info AS mi, movie_keyword AS mk, title AS t WHERE k.keyword like '%sequel%' AND mi.info IN ('Sweden', 'Norway', 'Germany', 'Denmark', 'Swedish', 'Denish', 'Norwegian', 'German') AND t.production_year > 2005 AND t.id = mi.movie_id AND t.id = mk.movie_id AND mk.movie_id = mi.movie_id AND k.id = mk.keyword_id;
diff --git a/benchmarks/queries/imdb/3b.sql b/benchmarks/queries/imdb/3b.sql
new file mode 100644
index 0000000000..fd21efc810
--- /dev/null
+++ b/benchmarks/queries/imdb/3b.sql
@@ -0,0 +1 @@
+SELECT MIN(t.title) AS movie_title FROM keyword AS k, movie_info AS mi, movie_keyword AS mk, title AS t WHERE k.keyword like '%sequel%' AND mi.info IN ('Bulgaria') AND t.production_year > 2010 AND t.id = mi.movie_id AND t.id = mk.movie_id AND mk.movie_id = mi.movie_id AND k.id = mk.keyword_id;
diff --git a/benchmarks/queries/imdb/3c.sql b/benchmarks/queries/imdb/3c.sql
new file mode 100644
index 0000000000..5f34232a2e
--- /dev/null
+++ b/benchmarks/queries/imdb/3c.sql
@@ -0,0 +1 @@
+SELECT MIN(t.title) AS movie_title FROM keyword AS k, movie_info AS mi, movie_keyword AS mk, title AS t WHERE k.keyword like '%sequel%' AND mi.info IN ('Sweden', 'Norway', 'Germany', 'Denmark', 'Swedish', 'Denish', 'Norwegian', 'German', 'USA', 'American') AND t.production_year > 1990 AND t.id = mi.movie_id AND t.id = mk.movie_id AND mk.movie_id = mi.movie_id AND k.id = mk.keyword_id;
diff --git a/benchmarks/queries/imdb/4a.sql b/benchmarks/queries/imdb/4a.sql
new file mode 100644
index 0000000000..636afab02c
--- /dev/null
+++ b/benchmarks/queries/imdb/4a.sql
@@ -0,0 +1 @@
+SELECT MIN(mi_idx.info) AS rating, MIN(t.title) AS movie_title FROM info_type AS it, keyword AS k, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE it.info ='rating' AND k.keyword like '%sequel%' AND mi_idx.info > '5.0' AND t.production_year > 2005 AND t.id = mi_idx.movie_id AND t.id = mk.movie_id AND mk.movie_id = mi_idx.movie_id AND k.id = mk.keyword_id AND it.id = mi_idx.info_type_id;
diff --git a/benchmarks/queries/imdb/4b.sql b/benchmarks/queries/imdb/4b.sql
new file mode 100644
index 0000000000..ebd3e89920
--- /dev/null
+++ b/benchmarks/queries/imdb/4b.sql
@@ -0,0 +1 @@
+SELECT MIN(mi_idx.info) AS rating, MIN(t.title) AS movie_title FROM info_type AS it, keyword AS k, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE it.info ='rating' AND k.keyword like '%sequel%' AND mi_idx.info > '9.0' AND t.production_year > 2010 AND t.id = mi_idx.movie_id AND t.id = mk.movie_id AND mk.movie_id = mi_idx.movie_id AND k.id = mk.keyword_id AND it.id = mi_idx.info_type_id;
diff --git a/benchmarks/queries/imdb/4c.sql b/benchmarks/queries/imdb/4c.sql
new file mode 100644
index 0000000000..309281200f
--- /dev/null
+++ b/benchmarks/queries/imdb/4c.sql
@@ -0,0 +1 @@
+SELECT MIN(mi_idx.info) AS rating, MIN(t.title) AS movie_title FROM info_type AS it, keyword AS k, movie_info_idx AS mi_idx, movie_keyword AS mk, title AS t WHERE it.info ='rating' AND k.keyword like '%sequel%' AND mi_idx.info > '2.0' AND t.production_year > 1990 AND t.id = mi_idx.movie_id AND t.id = mk.movie_id AND mk.movie_id = mi_idx.movie_id AND k.id = mk.keyword_id AND it.id = mi_idx.info_type_id;
diff --git a/benchmarks/queries/imdb/5a.sql b/benchmarks/queries/imdb/5a.sql
new file mode 100644
index 0000000000..04aae9881f
--- /dev/null
+++ b/benchmarks/queries/imdb/5a.sql
@@ -0,0 +1 @@
+SELECT MIN(t.title) AS typical_european_movie FROM company_type AS ct, info_type AS it, movie_companies AS mc, movie_info AS mi, title AS t WHERE ct.kind = 'production companies' AND mc.note like '%(theatrical)%' and mc.note like '%(France)%' AND mi.info IN ('Sweden', 'Norway', 'Germany', 'Denmark', 'Swedish', 'Denish', 'Norwegian', 'German') AND t.production_year > 2005 AND t.id = mi.movie_id AND t.id = mc.movie_id AND mc.movie_id = mi.movie_id AND ct.id = mc.company_type_id AND it.i [...]
diff --git a/benchmarks/queries/imdb/5b.sql b/benchmarks/queries/imdb/5b.sql
new file mode 100644
index 0000000000..f03a519d61
--- /dev/null
+++ b/benchmarks/queries/imdb/5b.sql
@@ -0,0 +1 @@
+SELECT MIN(t.title) AS american_vhs_movie FROM company_type AS ct, info_type AS it, movie_companies AS mc, movie_info AS mi, title AS t WHERE ct.kind = 'production companies' AND mc.note like '%(VHS)%' and mc.note like '%(USA)%' and mc.note like '%(1994)%' AND mi.info IN ('USA', 'America') AND t.production_year > 2010 AND t.id = mi.movie_id AND t.id = mc.movie_id AND mc.movie_id = mi.movie_id AND ct.id = mc.company_type_id AND it.id = mi.info_type_id;
diff --git a/benchmarks/queries/imdb/5c.sql b/benchmarks/queries/imdb/5c.sql
new file mode 100644
index 0000000000..2705e7e2c7
--- /dev/null
+++ b/benchmarks/queries/imdb/5c.sql
@@ -0,0 +1 @@
+SELECT MIN(t.title) AS american_movie FROM company_type AS ct, info_type AS it, movie_companies AS mc, movie_info AS mi, title AS t WHERE ct.kind = 'production companies' AND mc.note not like '%(TV)%' and mc.note like '%(USA)%' AND mi.info IN ('Sweden', 'Norway', 'Germany', 'Denmark', 'Swedish', 'Denish', 'Norwegian', 'German', 'USA', 'American') AND t.production_year > 1990 AND t.id = mi.movie_id AND t.id = mc.movie_id AND mc.movie_id = mi.movie_id AND ct.id = mc.company_type_id AND [...]
diff --git a/benchmarks/queries/imdb/6a.sql b/benchmarks/queries/imdb/6a.sql
new file mode 100644
index 0000000000..34b3a6da5f
--- /dev/null
+++ b/benchmarks/queries/imdb/6a.sql
@@ -0,0 +1 @@
+SELECT MIN(k.keyword) AS movie_keyword, MIN(n.name) AS actor_name, MIN(t.title) AS marvel_movie FROM cast_info AS ci, keyword AS k, movie_keyword AS mk, name AS n, title AS t WHERE k.keyword = 'marvel-cinematic-universe' AND n.name LIKE '%Downey%Robert%' AND t.production_year > 2010 AND k.id = mk.keyword_id AND t.id = mk.movie_id AND t.id = ci.movie_id AND ci.movie_id = mk.movie_id AND n.id = ci.person_id;
diff --git a/benchmarks/queries/imdb/6b.sql b/benchmarks/queries/imdb/6b.sql
new file mode 100644
index 0000000000..1233c41e66
--- /dev/null
+++ b/benchmarks/queries/imdb/6b.sql
@@ -0,0 +1 @@
+SELECT MIN(k.keyword) AS movie_keyword, MIN(n.name) AS actor_name, MIN(t.title) AS hero_movie FROM cast_info AS ci, keyword AS k, movie_keyword AS mk, name AS n, title AS t WHERE k.keyword in ('superhero', 'sequel', 'second-part', 'marvel-comics', 'based-on-comic', 'tv-special', 'fight', 'violence') AND n.name LIKE '%Downey%Robert%' AND t.production_year > 2014 AND k.id = mk.keyword_id AND t.id = mk.movie_id AND t.id = ci.movie_id AND ci.movie_id = mk.movie_id AND n.id = ci.person_id;
diff --git a/benchmarks/queries/imdb/6c.sql b/benchmarks/queries/imdb/6c.sql
new file mode 100644
index 0000000000..d1f97746e1
--- /dev/null
+++ b/benchmarks/queries/imdb/6c.sql
@@ -0,0 +1 @@
+SELECT MIN(k.keyword) AS movie_keyword, MIN(n.name) AS actor_name, MIN(t.title) AS marvel_movie FROM cast_info AS ci, keyword AS k, movie_keyword AS mk, name AS n, title AS t WHERE k.keyword = 'marvel-cinematic-universe' AND n.name LIKE '%Downey%Robert%' AND t.production_year > 2014 AND k.id = mk.keyword_id AND t.id = mk.movie_id AND t.id = ci.movie_id AND ci.movie_id = mk.movie_id AND n.id = ci.person_id;
diff --git a/benchmarks/queries/imdb/6d.sql b/benchmarks/queries/imdb/6d.sql
new file mode 100644
index 0000000000..07729510a4
--- /dev/null
+++ b/benchmarks/queries/imdb/6d.sql
@@ -0,0 +1 @@
+SELECT MIN(k.keyword) AS movie_keyword, MIN(n.name) AS actor_name, MIN(t.title) AS hero_movie FROM cast_info AS ci, keyword AS k, movie_keyword AS mk, name AS n, title AS t WHERE k.keyword in ('superhero', 'sequel', 'second-part', 'marvel-comics', 'based-on-comic', 'tv-special', 'fight', 'violence') AND n.name LIKE '%Downey%Robert%' AND t.production_year > 2000 AND k.id = mk.keyword_id AND t.id = mk.movie_id AND t.id = ci.movie_id AND ci.movie_id = mk.movie_id AND n.id = ci.person_id;
diff --git a/benchmarks/queries/imdb/6e.sql b/benchmarks/queries/imdb/6e.sql
new file mode 100644
index 0000000000..2e77873fd8
--- /dev/null
+++ b/benchmarks/queries/imdb/6e.sql
@@ -0,0 +1 @@
+SELECT MIN(k.keyword) AS movie_keyword, MIN(n.name) AS actor_name, MIN(t.title) AS marvel_movie FROM cast_info AS ci, keyword AS k, movie_keyword AS mk, name AS n, title AS t WHERE k.keyword = 'marvel-cinematic-universe' AND n.name LIKE '%Downey%Robert%' AND t.production_year > 2000 AND k.id = mk.keyword_id AND t.id = mk.movie_id AND t.id = ci.movie_id AND ci.movie_id = mk.movie_id AND n.id = ci.person_id;
diff --git a/benchmarks/queries/imdb/6f.sql b/benchmarks/queries/imdb/6f.sql
new file mode 100644
index 0000000000..6039011291
--- /dev/null
+++ b/benchmarks/queries/imdb/6f.sql
@@ -0,0 +1 @@
+SELECT MIN(k.keyword) AS movie_keyword, MIN(n.name) AS actor_name, MIN(t.title) AS hero_movie FROM cast_info AS ci, keyword AS k, movie_keyword AS mk, name AS n, title AS t WHERE k.keyword in ('superhero', 'sequel', 'second-part', 'marvel-comics', 'based-on-comic', 'tv-special', 'fight', 'violence') AND t.production_year > 2000 AND k.id = mk.keyword_id AND t.id = mk.movie_id AND t.id = ci.movie_id AND ci.movie_id = mk.movie_id AND n.id = ci.person_id;
diff --git a/benchmarks/queries/imdb/7a.sql b/benchmarks/queries/imdb/7a.sql
new file mode 100644
index 0000000000..c6b26ce36f
--- /dev/null
+++ b/benchmarks/queries/imdb/7a.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS of_person, MIN(t.title) AS biography_movie FROM aka_name AS an, cast_info AS ci, info_type AS it, link_type AS lt, movie_link AS ml, name AS n, person_info AS pi, title AS t WHERE an.name LIKE '%a%' AND it.info ='mini biography' AND lt.link ='features' AND n.name_pcode_cf BETWEEN 'A' AND 'F' AND (n.gender='m' OR (n.gender = 'f' AND n.name LIKE 'B%')) AND pi.note ='Volker Boehm' AND t.production_year BETWEEN 1980 AND 1995 AND n.id = an.person_id AND n.id = pi.person_ [...]
diff --git a/benchmarks/queries/imdb/7b.sql b/benchmarks/queries/imdb/7b.sql
new file mode 100644
index 0000000000..4e4f6e7615
--- /dev/null
+++ b/benchmarks/queries/imdb/7b.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS of_person, MIN(t.title) AS biography_movie FROM aka_name AS an, cast_info AS ci, info_type AS it, link_type AS lt, movie_link AS ml, name AS n, person_info AS pi, title AS t WHERE an.name LIKE '%a%' AND it.info ='mini biography' AND lt.link ='features' AND n.name_pcode_cf LIKE 'D%' AND n.gender='m' AND pi.note ='Volker Boehm' AND t.production_year BETWEEN 1980 AND 1984 AND n.id = an.person_id AND n.id = pi.person_id AND ci.person_id = n.id AND t.id = ci.movie_id AND [...]
diff --git a/benchmarks/queries/imdb/7c.sql b/benchmarks/queries/imdb/7c.sql
new file mode 100644
index 0000000000..a399342fae
--- /dev/null
+++ b/benchmarks/queries/imdb/7c.sql
@@ -0,0 +1 @@
+SELECT MIN(n.name) AS cast_member_name, MIN(pi.info) AS cast_member_info FROM aka_name AS an, cast_info AS ci, info_type AS it, link_type AS lt, movie_link AS ml, name AS n, person_info AS pi, title AS t WHERE an.name is not NULL and (an.name LIKE '%a%' or an.name LIKE 'A%') AND it.info ='mini biography' AND lt.link in ('references', 'referenced in', 'features', 'featured in') AND n.name_pcode_cf BETWEEN 'A' AND 'F' AND (n.gender='m' OR (n.gender = 'f' AND n.name LIKE 'A%')) AND pi.not [...]
diff --git a/benchmarks/queries/imdb/8a.sql b/benchmarks/queries/imdb/8a.sql
new file mode 100644
index 0000000000..66ed05880d
--- /dev/null
+++ b/benchmarks/queries/imdb/8a.sql
@@ -0,0 +1 @@
+SELECT MIN(an1.name) AS actress_pseudonym, MIN(t.title) AS japanese_movie_dubbed FROM aka_name AS an1, cast_info AS ci, company_name AS cn, movie_companies AS mc, name AS n1, role_type AS rt, title AS t WHERE ci.note ='(voice: English version)' AND cn.country_code ='[jp]' AND mc.note like '%(Japan)%' and mc.note not like '%(USA)%' AND n1.name like '%Yo%' and n1.name not like '%Yu%' AND rt.role ='actress' AND an1.person_id = n1.id AND n1.id = ci.person_id AND ci.movie_id = t.id AND t.id = [...]
diff --git a/benchmarks/queries/imdb/8b.sql b/benchmarks/queries/imdb/8b.sql
new file mode 100644
index 0000000000..044b5f8e86
--- /dev/null
+++ b/benchmarks/queries/imdb/8b.sql
@@ -0,0 +1 @@
+SELECT MIN(an.name) AS acress_pseudonym, MIN(t.title) AS japanese_anime_movie FROM aka_name AS an, cast_info AS ci, company_name AS cn, movie_companies AS mc, name AS n, role_type AS rt, title AS t WHERE ci.note ='(voice: English version)' AND cn.country_code ='[jp]' AND mc.note like '%(Japan)%' and mc.note not like '%(USA)%' and (mc.note like '%(2006)%' or mc.note like '%(2007)%') AND n.name like '%Yo%' and n.name not like '%Yu%' AND rt.role ='actress' AND t.production_year between 2006 [...]
diff --git a/benchmarks/queries/imdb/8c.sql b/benchmarks/queries/imdb/8c.sql
new file mode 100644
index 0000000000..d02b74c02c
--- /dev/null
+++ b/benchmarks/queries/imdb/8c.sql
@@ -0,0 +1 @@
+SELECT MIN(a1.name) AS writer_pseudo_name, MIN(t.title) AS movie_title FROM aka_name AS a1, cast_info AS ci, company_name AS cn, movie_companies AS mc, name AS n1, role_type AS rt, title AS t WHERE cn.country_code ='[us]' AND rt.role ='writer' AND a1.person_id = n1.id AND n1.id = ci.person_id AND ci.movie_id = t.id AND t.id = mc.movie_id AND mc.company_id = cn.id AND ci.role_id = rt.id AND a1.person_id = ci.person_id AND ci.movie_id = mc.movie_id;
diff --git a/benchmarks/queries/imdb/8d.sql b/benchmarks/queries/imdb/8d.sql
new file mode 100644
index 0000000000..0834c0ff5c
--- /dev/null
+++ b/benchmarks/queries/imdb/8d.sql
@@ -0,0 +1 @@
+SELECT MIN(an1.name) AS costume_designer_pseudo, MIN(t.title) AS movie_with_costumes FROM aka_name AS an1, cast_info AS ci, company_name AS cn, movie_companies AS mc, name AS n1, role_type AS rt, title AS t WHERE cn.country_code ='[us]' AND rt.role ='costume designer' AND an1.person_id = n1.id AND n1.id = ci.person_id AND ci.movie_id = t.id AND t.id = mc.movie_id AND mc.company_id = cn.id AND ci.role_id = rt.id AND an1.person_id = ci.person_id AND ci.movie_id = mc.movie_id;
diff --git a/benchmarks/queries/imdb/9a.sql b/benchmarks/queries/imdb/9a.sql
new file mode 100644
index 0000000000..593b16213b
--- /dev/null
+++ b/benchmarks/queries/imdb/9a.sql
@@ -0,0 +1 @@
+SELECT MIN(an.name) AS alternative_name, MIN(chn.name) AS character_name, MIN(t.title) AS movie FROM aka_name AS an, char_name AS chn, cast_info AS ci, company_name AS cn, movie_companies AS mc, name AS n, role_type AS rt, title AS t WHERE ci.note in ('(voice)', '(voice: Japanese version)', '(voice) (uncredited)', '(voice: English version)') AND cn.country_code ='[us]' AND mc.note is not NULL and (mc.note like '%(USA)%' or mc.note like '%(worldwide)%') AND n.gender ='f' and n.name like [...]
diff --git a/benchmarks/queries/imdb/9b.sql b/benchmarks/queries/imdb/9b.sql
new file mode 100644
index 0000000000..a4933fd685
--- /dev/null
+++ b/benchmarks/queries/imdb/9b.sql
@@ -0,0 +1 @@
+SELECT MIN(an.name) AS alternative_name, MIN(chn.name) AS voiced_character, MIN(n.name) AS voicing_actress, MIN(t.title) AS american_movie FROM aka_name AS an, char_name AS chn, cast_info AS ci, company_name AS cn, movie_companies AS mc, name AS n, role_type AS rt, title AS t WHERE ci.note = '(voice)' AND cn.country_code ='[us]' AND mc.note like '%(200%)%' and (mc.note like '%(USA)%' or mc.note like '%(worldwide)%') AND n.gender ='f' and n.name like '%Angel%' AND rt.role ='actress' AND [...]
diff --git a/benchmarks/queries/imdb/9c.sql b/benchmarks/queries/imdb/9c.sql
new file mode 100644
index 0000000000..0be511810c
--- /dev/null
+++ b/benchmarks/queries/imdb/9c.sql
@@ -0,0 +1 @@
+SELECT MIN(an.name) AS alternative_name, MIN(chn.name) AS voiced_character_name, MIN(n.name) AS voicing_actress, MIN(t.title) AS american_movie FROM aka_name AS an, char_name AS chn, cast_info AS ci, company_name AS cn, movie_companies AS mc, name AS n, role_type AS rt, title AS t WHERE ci.note in ('(voice)', '(voice: Japanese version)', '(voice) (uncredited)', '(voice: English version)') AND cn.country_code ='[us]' AND n.gender ='f' and n.name like '%An%' AND rt.role ='actress' AND ci. [...]
diff --git a/benchmarks/queries/imdb/9d.sql b/benchmarks/queries/imdb/9d.sql
new file mode 100644
index 0000000000..51262ca5eb
--- /dev/null
+++ b/benchmarks/queries/imdb/9d.sql
@@ -0,0 +1 @@
+SELECT MIN(an.name) AS alternative_name, MIN(chn.name) AS voiced_char_name, MIN(n.name) AS voicing_actress, MIN(t.title) AS american_movie FROM aka_name AS an, char_name AS chn, cast_info AS ci, company_name AS cn, movie_companies AS mc, name AS n, role_type AS rt, title AS t WHERE ci.note in ('(voice)', '(voice: Japanese version)', '(voice) (uncredited)', '(voice: English version)') AND cn.country_code ='[us]' AND n.gender ='f' AND rt.role ='actress' AND ci.movie_id = t.id AND t.id = m [...]
diff --git a/benchmarks/src/bin/dfbench.rs b/benchmarks/src/bin/dfbench.rs
index 9ce6848a06..f7b84116e7 100644
--- a/benchmarks/src/bin/dfbench.rs
+++ b/benchmarks/src/bin/dfbench.rs
@@ -33,7 +33,7 @@ static ALLOC: snmalloc_rs::SnMalloc = snmalloc_rs::SnMalloc;
#[global_allocator]
static ALLOC: mimalloc::MiMalloc = mimalloc::MiMalloc;
-use datafusion_benchmarks::{clickbench, parquet_filter, sort, tpch};
+use datafusion_benchmarks::{clickbench, imdb, parquet_filter, sort, tpch};
#[derive(Debug, StructOpt)]
#[structopt(about = "benchmark command")]
@@ -43,6 +43,7 @@ enum Options {
Clickbench(clickbench::RunOpt),
ParquetFilter(parquet_filter::RunOpt),
Sort(sort::RunOpt),
+ Imdb(imdb::RunOpt),
}
// Main benchmark runner entrypoint
@@ -56,5 +57,6 @@ pub async fn main() -> Result<()> {
Options::Clickbench(opt) => opt.run().await,
Options::ParquetFilter(opt) => opt.run().await,
Options::Sort(opt) => opt.run().await,
+ Options::Imdb(opt) => opt.run().await,
}
}
diff --git a/benchmarks/src/bin/imdb.rs b/benchmarks/src/bin/imdb.rs
index 40efb84b05..13421f8a89 100644
--- a/benchmarks/src/bin/imdb.rs
+++ b/benchmarks/src/bin/imdb.rs
@@ -34,9 +34,17 @@ static ALLOC: snmalloc_rs::SnMalloc = snmalloc_rs::SnMalloc;
#[global_allocator]
static ALLOC: mimalloc::MiMalloc = mimalloc::MiMalloc;
+#[derive(Debug, StructOpt)]
+#[structopt(about = "benchmark command")]
+enum BenchmarkSubCommandOpt {
+ #[structopt(name = "datafusion")]
+ DataFusionBenchmark(imdb::RunOpt),
+}
+
#[derive(Debug, StructOpt)]
#[structopt(name = "IMDB", about = "IMDB Dataset Processing.")]
enum ImdbOpt {
+ Benchmark(BenchmarkSubCommandOpt),
Convert(imdb::ConvertOpt),
}
@@ -44,6 +52,9 @@ enum ImdbOpt {
pub async fn main() -> Result<()> {
env_logger::init();
match ImdbOpt::from_args() {
+ ImdbOpt::Benchmark(BenchmarkSubCommandOpt::DataFusionBenchmark(opt)) => {
+ opt.run().await
+ }
ImdbOpt::Convert(opt) => opt.run().await,
}
}
diff --git a/benchmarks/src/imdb/convert.rs b/benchmarks/src/imdb/convert.rs
index c95f7f8bf5..4e470d711d 100644
--- a/benchmarks/src/imdb/convert.rs
+++ b/benchmarks/src/imdb/convert.rs
@@ -51,11 +51,12 @@ impl ConvertOpt {
pub async fn run(self) -> Result<()> {
let input_path = self.input_path.to_str().unwrap();
let output_path = self.output_path.to_str().unwrap();
+ let config = SessionConfig::new().with_batch_size(self.batch_size);
+ let ctx = SessionContext::new_with_config(config);
for table in IMDB_TABLES {
let start = Instant::now();
let schema = get_imdb_table_schema(table);
-
let input_path = format!("{input_path}/{table}.csv");
let output_path = format!("{output_path}/{table}.parquet");
let options = CsvReadOptions::new()
@@ -65,9 +66,6 @@ impl ConvertOpt {
.escape(b'\\')
.file_extension(".csv");
- let config = SessionConfig::new().with_batch_size(self.batch_size);
- let ctx = SessionContext::new_with_config(config);
-
let mut csv = ctx.read_csv(&input_path, options).await?;
// Select all apart from the padding column
diff --git a/benchmarks/src/imdb/mod.rs b/benchmarks/src/imdb/mod.rs
index 8e2977c038..6a45242e6f 100644
--- a/benchmarks/src/imdb/mod.rs
+++ b/benchmarks/src/imdb/mod.rs
@@ -17,10 +17,18 @@
//! Benchmark derived from IMDB dataset.
-use datafusion::arrow::datatypes::{DataType, Field, Schema};
+use datafusion::{
+ arrow::datatypes::{DataType, Field, Schema},
+ common::plan_err,
+ error::Result,
+};
mod convert;
pub use convert::ConvertOpt;
+use std::fs;
+mod run;
+pub use run::RunOpt;
+
// we have 21 tables in the IMDB dataset
pub const IMDB_TABLES: &[&str] = &[
"aka_name",
@@ -51,7 +59,7 @@ pub const IMDB_TABLES: &[&str] = &[
pub fn get_imdb_table_schema(table: &str) -> Schema {
match table {
"aka_name" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("person_id", DataType::Int32, false),
Field::new("name", DataType::Utf8, true),
Field::new("imdb_index", DataType::Utf8, true),
@@ -61,7 +69,7 @@ pub fn get_imdb_table_schema(table: &str) -> Schema {
Field::new("md5sum", DataType::Utf8, true),
]),
"aka_title" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("movie_id", DataType::Int32, false),
Field::new("title", DataType::Utf8, true),
Field::new("imdb_index", DataType::Utf8, true),
@@ -75,7 +83,7 @@ pub fn get_imdb_table_schema(table: &str) -> Schema {
Field::new("md5sum", DataType::Utf8, true),
]),
"cast_info" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("person_id", DataType::Int32, false),
Field::new("movie_id", DataType::Int32, false),
Field::new("person_role_id", DataType::Int32, true),
@@ -84,7 +92,7 @@ pub fn get_imdb_table_schema(table: &str) -> Schema {
Field::new("role_id", DataType::Int32, false),
]),
"char_name" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("name", DataType::Utf8, false),
Field::new("imdb_index", DataType::Utf8, true),
Field::new("imdb_id", DataType::Int32, true),
@@ -93,11 +101,11 @@ pub fn get_imdb_table_schema(table: &str) -> Schema {
Field::new("md5sum", DataType::Utf8, true),
]),
"comp_cast_type" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("kind", DataType::Utf8, false),
]),
"company_name" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("name", DataType::Utf8, false),
Field::new("country_code", DataType::Utf8, true),
Field::new("imdb_id", DataType::Int32, true),
@@ -106,59 +114,59 @@ pub fn get_imdb_table_schema(table: &str) -> Schema {
Field::new("md5sum", DataType::Utf8, true),
]),
"company_type" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("kind", DataType::Utf8, true),
]),
"complete_cast" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("movie_id", DataType::Int32, true),
Field::new("subject_id", DataType::Int32, false),
Field::new("status_id", DataType::Int32, false),
]),
"info_type" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("info", DataType::Utf8, false),
]),
"keyword" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("keyword", DataType::Utf8, false),
Field::new("phonetic_code", DataType::Utf8, true),
]),
"kind_type" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("kind", DataType::Utf8, true),
]),
"link_type" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("link", DataType::Utf8, false),
]),
"movie_companies" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("movie_id", DataType::Int32, false),
Field::new("company_id", DataType::Int32, false),
Field::new("company_type_id", DataType::Int32, false),
Field::new("note", DataType::Utf8, true),
]),
"movie_info_idx" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("movie_id", DataType::Int32, false),
Field::new("info_type_id", DataType::Int32, false),
Field::new("info", DataType::Utf8, false),
Field::new("note", DataType::Utf8, true),
]),
"movie_keyword" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("movie_id", DataType::Int32, false),
Field::new("keyword_id", DataType::Int32, false),
]),
"movie_link" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("movie_id", DataType::Int32, false),
Field::new("linked_movie_id", DataType::Int32, false),
Field::new("link_type_id", DataType::Int32, false),
]),
"name" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("name", DataType::Utf8, false),
Field::new("imdb_index", DataType::Utf8, true),
Field::new("imdb_id", DataType::Int32, true),
@@ -169,11 +177,11 @@ pub fn get_imdb_table_schema(table: &str) -> Schema {
Field::new("md5sum", DataType::Utf8, true),
]),
"role_type" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("role", DataType::Utf8, false),
]),
"title" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("title", DataType::Utf8, false),
Field::new("imdb_index", DataType::Utf8, true),
Field::new("kind_id", DataType::Int32, false),
@@ -187,14 +195,14 @@ pub fn get_imdb_table_schema(table: &str) -> Schema {
Field::new("md5sum", DataType::Utf8, true),
]),
"movie_info" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("movie_id", DataType::Int32, false),
Field::new("info_type_id", DataType::Int32, false),
Field::new("info", DataType::Utf8, false),
Field::new("note", DataType::Utf8, true),
]),
"person_info" => Schema::new(vec![
- Field::new("id", DataType::Int32, false),
+ Field::new("id", DataType::UInt32, false),
Field::new("person_id", DataType::Int32, false),
Field::new("info_type_id", DataType::Int32, false),
Field::new("info", DataType::Utf8, false),
@@ -203,3 +211,26 @@ pub fn get_imdb_table_schema(table: &str) -> Schema {
_ => unimplemented!("Schema for table {} is not implemented", table),
}
}
+
+/// Get the SQL statements from the specified query file
+pub fn get_query_sql(query: &str) -> Result<Vec<String>> {
+ let possibilities = vec![
+ format!("queries/imdb/{query}.sql"),
+ format!("benchmarks/queries/imdb/{query}.sql"),
+ ];
+ let mut errors = vec![];
+ for filename in possibilities {
+ match fs::read_to_string(&filename) {
+ Ok(contents) => {
+ return Ok(contents
+ .split(';')
+ .map(|s| s.trim())
+ .filter(|s| !s.is_empty())
+ .map(|s| s.to_string())
+ .collect());
+ }
+ Err(e) => errors.push(format!("{filename}: {e}")),
+ };
+ }
+ plan_err!("invalid query. Could not find query: {:?}", errors)
+}
diff --git a/benchmarks/src/imdb/run.rs b/benchmarks/src/imdb/run.rs
new file mode 100644
index 0000000000..697c79dc89
--- /dev/null
+++ b/benchmarks/src/imdb/run.rs
@@ -0,0 +1,827 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::path::PathBuf;
+use std::sync::Arc;
+
+use super::{get_imdb_table_schema, get_query_sql, IMDB_TABLES};
+use crate::{BenchmarkRun, CommonOpt};
+
+use arrow::record_batch::RecordBatch;
+use arrow::util::pretty::{self, pretty_format_batches};
+use datafusion::datasource::file_format::csv::CsvFormat;
+use datafusion::datasource::file_format::parquet::ParquetFormat;
+use datafusion::datasource::file_format::FileFormat;
+use datafusion::datasource::listing::{
+ ListingOptions, ListingTable, ListingTableConfig, ListingTableUrl,
+};
+use datafusion::datasource::{MemTable, TableProvider};
+use datafusion::error::Result;
+use datafusion::physical_plan::display::DisplayableExecutionPlan;
+use datafusion::physical_plan::{collect, displayable};
+use datafusion::prelude::*;
+use datafusion_common::instant::Instant;
+use datafusion_common::{DEFAULT_CSV_EXTENSION, DEFAULT_PARQUET_EXTENSION};
+
+use log::info;
+use structopt::StructOpt;
+
+// hack to avoid `default_value is meaningless for bool` errors
+type BoolDefaultTrue = bool;
+
+/// Run the imdb benchmark (a.k.a. JOB).
+///
+/// This benchmarks is derived from the [Join Order Benchmark / JOB] proposed in paper [How Good Are Query Optimizers, Really?][1].
+/// The data and answers are downloaded from
+/// [2] and [3].
+///
+/// [1]: https://www.vldb.org/pvldb/vol9/p204-leis.pdf
+/// [2]: http://homepages.cwi.nl/~boncz/job/imdb.tgz
+/// [3]: https://db.in.tum.de/~leis/qo/job.tgz
+
+#[derive(Debug, StructOpt, Clone)]
+#[structopt(verbatim_doc_comment)]
+pub struct RunOpt {
+ /// Query number. If not specified, runs all queries
+ #[structopt(short, long)]
+ query: Option<usize>,
+
+ /// Common options
+ #[structopt(flatten)]
+ common: CommonOpt,
+
+ /// Path to data files
+ #[structopt(parse(from_os_str), required = true, short = "p", long = "path")]
+ path: PathBuf,
+
+ /// File format: `csv` or `parquet`
+ #[structopt(short = "f", long = "format", default_value = "csv")]
+ file_format: String,
+
+ /// Load the data into a MemTable before executing the query
+ #[structopt(short = "m", long = "mem-table")]
+ mem_table: bool,
+
+ /// Path to machine readable output file
+ #[structopt(parse(from_os_str), short = "o", long = "output")]
+ output_path: Option<PathBuf>,
+
+ /// Whether to disable collection of statistics (and cost based optimizations) or not.
+ #[structopt(short = "S", long = "disable-statistics")]
+ disable_statistics: bool,
+
+ /// If true then hash join used, if false then sort merge join
+ /// True by default.
+ #[structopt(short = "j", long = "prefer_hash_join", default_value = "true")]
+ prefer_hash_join: BoolDefaultTrue,
+}
+
+const IMDB_QUERY_START_ID: usize = 1;
+const IMDB_QUERY_END_ID: usize = 113;
+
+fn map_query_id_to_str(query_id: usize) -> &'static str {
+ match query_id {
+ // 1
+ 1 => "1a",
+ 2 => "1b",
+ 3 => "1c",
+ 4 => "1d",
+
+ // 2
+ 5 => "2a",
+ 6 => "2b",
+ 7 => "2c",
+ 8 => "2d",
+
+ // 3
+ 9 => "3a",
+ 10 => "3b",
+ 11 => "3c",
+
+ // 4
+ 12 => "4a",
+ 13 => "4b",
+ 14 => "4c",
+
+ // 5
+ 15 => "5a",
+ 16 => "5b",
+ 17 => "5c",
+
+ // 6
+ 18 => "6a",
+ 19 => "6b",
+ 20 => "6c",
+ 21 => "6d",
+ 22 => "6e",
+ 23 => "6f",
+
+ // 7
+ 24 => "7a",
+ 25 => "7b",
+ 26 => "7c",
+
+ // 8
+ 27 => "8a",
+ 28 => "8b",
+ 29 => "8c",
+ 30 => "8d",
+
+ // 9
+ 31 => "9a",
+ 32 => "9b",
+ 33 => "9c",
+ 34 => "9d",
+
+ // 10
+ 35 => "10a",
+ 36 => "10b",
+ 37 => "10c",
+
+ // 11
+ 38 => "11a",
+ 39 => "11b",
+ 40 => "11c",
+ 41 => "11d",
+
+ // 12
+ 42 => "12a",
+ 43 => "12b",
+ 44 => "12c",
+
+ // 13
+ 45 => "13a",
+ 46 => "13b",
+ 47 => "13c",
+ 48 => "13d",
+
+ // 14
+ 49 => "14a",
+ 50 => "14b",
+ 51 => "14c",
+
+ // 15
+ 52 => "15a",
+ 53 => "15b",
+ 54 => "15c",
+ 55 => "15d",
+
+ // 16
+ 56 => "16a",
+ 57 => "16b",
+ 58 => "16c",
+ 59 => "16d",
+
+ // 17
+ 60 => "17a",
+ 61 => "17b",
+ 62 => "17c",
+ 63 => "17d",
+ 64 => "17e",
+ 65 => "17f",
+
+ // 18
+ 66 => "18a",
+ 67 => "18b",
+ 68 => "18c",
+
+ // 19
+ 69 => "19a",
+ 70 => "19b",
+ 71 => "19c",
+ 72 => "19d",
+
+ // 20
+ 73 => "20a",
+ 74 => "20b",
+ 75 => "20c",
+
+ // 21
+ 76 => "21a",
+ 77 => "21b",
+ 78 => "21c",
+
+ // 22
+ 79 => "22a",
+ 80 => "22b",
+ 81 => "22c",
+ 82 => "22d",
+
+ // 23
+ 83 => "23a",
+ 84 => "23b",
+ 85 => "23c",
+
+ // 24
+ 86 => "24a",
+ 87 => "24b",
+
+ // 25
+ 88 => "25a",
+ 89 => "25b",
+ 90 => "25c",
+
+ // 26
+ 91 => "26a",
+ 92 => "26b",
+ 93 => "26c",
+
+ // 27
+ 94 => "27a",
+ 95 => "27b",
+ 96 => "27c",
+
+ // 28
+ 97 => "28a",
+ 98 => "28b",
+ 99 => "28c",
+
+ // 29
+ 100 => "29a",
+ 101 => "29b",
+ 102 => "29c",
+
+ // 30
+ 103 => "30a",
+ 104 => "30b",
+ 105 => "30c",
+
+ // 31
+ 106 => "31a",
+ 107 => "31b",
+ 108 => "31c",
+
+ // 32
+ 109 => "32a",
+ 110 => "32b",
+
+ // 33
+ 111 => "33a",
+ 112 => "33b",
+ 113 => "33c",
+
+ // Fallback for unknown query_id
+ _ => "unknown",
+ }
+}
+
+impl RunOpt {
+ pub async fn run(self) -> Result<()> {
+ println!("Running benchmarks with the following options: {self:?}");
+ let query_range = match self.query {
+ Some(query_id) => query_id..=query_id,
+ None => IMDB_QUERY_START_ID..=IMDB_QUERY_END_ID,
+ };
+
+ let mut benchmark_run = BenchmarkRun::new();
+ for query_id in query_range {
+ benchmark_run.start_new_case(&format!("Query {query_id}"));
+ let query_run = self.benchmark_query(query_id).await?;
+ for iter in query_run {
+ benchmark_run.write_iter(iter.elapsed, iter.row_count);
+ }
+ }
+ benchmark_run.maybe_write_json(self.output_path.as_ref())?;
+ Ok(())
+ }
+
+ async fn benchmark_query(&self, query_id: usize) -> Result<Vec<QueryResult>> {
+ let mut config = self
+ .common
+ .config()
+ .with_collect_statistics(!self.disable_statistics);
+ config.options_mut().optimizer.prefer_hash_join = self.prefer_hash_join;
+ config
+ .options_mut()
+ .execution
+ .parquet
+ .schema_force_view_types = self.common.force_view_types;
+ let ctx = SessionContext::new_with_config(config);
+
+ // register tables
+ self.register_tables(&ctx).await?;
+
+ let mut millis = vec![];
+ // run benchmark
+ let mut query_results = vec![];
+ for i in 0..self.iterations() {
+ let start = Instant::now();
+
+ let query_id_str = map_query_id_to_str(query_id);
+ let sql = &get_query_sql(query_id_str)?;
+
+ let mut result = vec![];
+
+ for query in sql {
+ result = self.execute_query(&ctx, query).await?;
+ }
+
+ let elapsed = start.elapsed(); //.as_secs_f64() * 1000.0;
+ let ms = elapsed.as_secs_f64() * 1000.0;
+ millis.push(ms);
+ info!("output:\n\n{}\n\n", pretty_format_batches(&result)?);
+ let row_count = result.iter().map(|b| b.num_rows()).sum();
+ println!(
+ "Query {query_id} iteration {i} took {ms:.1} ms and returned {row_count} rows"
+ );
+ query_results.push(QueryResult { elapsed, row_count });
+ }
+
+ let avg = millis.iter().sum::<f64>() / millis.len() as f64;
+ println!("Query {query_id} avg time: {avg:.2} ms");
+
+ Ok(query_results)
+ }
+
+ async fn register_tables(&self, ctx: &SessionContext) -> Result<()> {
+ for table in IMDB_TABLES {
+ let table_provider = { self.get_table(ctx, table).await? };
+
+ if self.mem_table {
+ println!("Loading table '{table}' into memory");
+ let start = Instant::now();
+ let memtable =
+ MemTable::load(table_provider, Some(self.partitions()), &ctx.state())
+ .await?;
+ println!(
+ "Loaded table '{}' into memory in {} ms",
+ table,
+ start.elapsed().as_millis()
+ );
+ ctx.register_table(*table, Arc::new(memtable))?;
+ } else {
+ ctx.register_table(*table, table_provider)?;
+ }
+ }
+ Ok(())
+ }
+
+ async fn execute_query(
+ &self,
+ ctx: &SessionContext,
+ sql: &str,
+ ) -> Result<Vec<RecordBatch>> {
+ let debug = self.common.debug;
+ let plan = ctx.sql(sql).await?;
+ let (state, plan) = plan.into_parts();
+
+ if debug {
+ println!("=== Logical plan ===\n{plan}\n");
+ }
+
+ let plan = state.optimize(&plan)?;
+ if debug {
+ println!("=== Optimized logical plan ===\n{plan}\n");
+ }
+ let physical_plan = state.create_physical_plan(&plan).await?;
+ if debug {
+ println!(
+ "=== Physical plan ===\n{}\n",
+ displayable(physical_plan.as_ref()).indent(true)
+ );
+ }
+ let result = collect(physical_plan.clone(), state.task_ctx()).await?;
+ if debug {
+ println!(
+ "=== Physical plan with metrics ===\n{}\n",
+ DisplayableExecutionPlan::with_metrics(physical_plan.as_ref())
+ .indent(true)
+ );
+ if !result.is_empty() {
+ // do not call print_batches if there are no batches as the result is confusing
+ // and makes it look like there is a batch with no columns
+ pretty::print_batches(&result)?;
+ }
+ }
+ Ok(result)
+ }
+
+ async fn get_table(
+ &self,
+ ctx: &SessionContext,
+ table: &str,
+ ) -> Result<Arc<dyn TableProvider>> {
+ let path = self.path.to_str().unwrap();
+ let table_format = self.file_format.as_str();
+
+ // Obtain a snapshot of the SessionState
+ let state = ctx.state();
+ let (format, path, extension): (Arc<dyn FileFormat>, String, &'static str) =
+ match table_format {
+ // dbgen creates .tbl ('|' delimited) files without header
+ "tbl" => {
+ let path = format!("{path}/{table}.tbl");
+
+ let format = CsvFormat::default()
+ .with_delimiter(b'|')
+ .with_has_header(false);
+
+ (Arc::new(format), path, ".tbl")
+ }
+ "csv" => {
+ let path = format!("{path}/{table}.csv");
+ let format = CsvFormat::default()
+ .with_delimiter(b',')
+ .with_escape(Some(b'\\'))
+ .with_has_header(false);
+
+ (Arc::new(format), path, DEFAULT_CSV_EXTENSION)
+ }
+ "parquet" => {
+ let path = format!("{path}/{table}.parquet");
+ let format = ParquetFormat::default()
+ .with_options(ctx.state().table_options().parquet.clone());
+ (Arc::new(format), path, DEFAULT_PARQUET_EXTENSION)
+ }
+ other => {
+ unimplemented!("Invalid file format '{}'", other);
+ }
+ };
+
+ let options = ListingOptions::new(format)
+ .with_file_extension(extension)
+ .with_collect_stat(state.config().collect_statistics());
+
+ let table_path = ListingTableUrl::parse(path)?;
+ let config = ListingTableConfig::new(table_path).with_listing_options(options);
+ let config = match table_format {
+ "parquet" => config.with_schema(Arc::new(get_imdb_table_schema(table))),
+ "csv" => config.with_schema(Arc::new(get_imdb_table_schema(table))),
+ _ => unreachable!(),
+ };
+
+ Ok(Arc::new(ListingTable::try_new(config)?))
+ }
+
+ fn iterations(&self) -> usize {
+ self.common.iterations
+ }
+
+ fn partitions(&self) -> usize {
+ self.common.partitions.unwrap_or(num_cpus::get())
+ }
+}
+
+struct QueryResult {
+ elapsed: std::time::Duration,
+ row_count: usize,
+}
+
+#[cfg(test)]
+// Only run with "ci" mode when we have the data
+#[cfg(feature = "ci")]
+mod tests {
+ use std::path::Path;
+
+ use super::*;
+
+ use datafusion::common::exec_err;
+ use datafusion::error::Result;
+ use datafusion_proto::bytes::{
+ logical_plan_from_bytes, logical_plan_to_bytes, physical_plan_from_bytes,
+ physical_plan_to_bytes,
+ };
+
+ fn get_imdb_data_path() -> Result<String> {
+ let path =
+ std::env::var("IMDB_DATA").unwrap_or_else(|_| "benchmarks/data".to_string());
+ if !Path::new(&path).exists() {
+ return exec_err!(
+ "Benchmark data not found (set IMDB_DATA env var to override): {}",
+ path
+ );
+ }
+ Ok(path)
+ }
+
+ async fn round_trip_logical_plan(query: usize) -> Result<()> {
+ let ctx = SessionContext::default();
+ let path = get_imdb_data_path()?;
+ let common = CommonOpt {
+ iterations: 1,
+ partitions: Some(2),
+ batch_size: 8192,
+ debug: false,
+ force_view_types: false,
+ };
+ let opt = RunOpt {
+ query: Some(query),
+ common,
+ path: PathBuf::from(path.to_string()),
+ file_format: "parquet".to_string(),
+ mem_table: false,
+ output_path: None,
+ disable_statistics: false,
+ prefer_hash_join: true,
+ };
+ opt.register_tables(&ctx).await?;
+ let queries = get_query_sql(map_query_id_to_str(query))?;
+ for query in queries {
+ let plan = ctx.sql(&query).await?;
+ let plan = plan.into_optimized_plan()?;
+ let bytes = logical_plan_to_bytes(&plan)?;
+ let plan2 = logical_plan_from_bytes(&bytes, &ctx)?;
+ let plan_formatted = format!("{}", plan.display_indent());
+ let plan2_formatted = format!("{}", plan2.display_indent());
+ assert_eq!(plan_formatted, plan2_formatted);
+ }
+ Ok(())
+ }
+
+ async fn round_trip_physical_plan(query: usize) -> Result<()> {
+ let ctx = SessionContext::default();
+ let path = get_imdb_data_path()?;
+ let common = CommonOpt {
+ iterations: 1,
+ partitions: Some(2),
+ batch_size: 8192,
+ debug: false,
+ force_view_types: false,
+ };
+ let opt = RunOpt {
+ query: Some(query),
+ common,
+ path: PathBuf::from(path.to_string()),
+ file_format: "parquet".to_string(),
+ mem_table: false,
+ output_path: None,
+ disable_statistics: false,
+ prefer_hash_join: true,
+ };
+ opt.register_tables(&ctx).await?;
+ let queries = get_query_sql(map_query_id_to_str(query))?;
+ for query in queries {
+ let plan = ctx.sql(&query).await?;
+ let plan = plan.create_physical_plan().await?;
+ let bytes = physical_plan_to_bytes(plan.clone())?;
+ let plan2 = physical_plan_from_bytes(&bytes, &ctx)?;
+ let plan_formatted = format!("{}", displayable(plan.as_ref()).indent(false));
+ let plan2_formatted =
+ format!("{}", displayable(plan2.as_ref()).indent(false));
+ assert_eq!(plan_formatted, plan2_formatted);
+ }
+ Ok(())
+ }
+
+ macro_rules! test_round_trip_logical {
+ ($tn:ident, $query:expr) => {
+ #[tokio::test]
+ async fn $tn() -> Result<()> {
+ round_trip_logical_plan($query).await
+ }
+ };
+ }
+
+ macro_rules! test_round_trip_physical {
+ ($tn:ident, $query:expr) => {
+ #[tokio::test]
+ async fn $tn() -> Result<()> {
+ round_trip_physical_plan($query).await
+ }
+ };
+ }
+
+ // logical plan tests
+ test_round_trip_logical!(round_trip_logical_plan_1a, 1);
+ test_round_trip_logical!(round_trip_logical_plan_1b, 2);
+ test_round_trip_logical!(round_trip_logical_plan_1c, 3);
+ test_round_trip_logical!(round_trip_logical_plan_1d, 4);
+ test_round_trip_logical!(round_trip_logical_plan_2a, 5);
+ test_round_trip_logical!(round_trip_logical_plan_2b, 6);
+ test_round_trip_logical!(round_trip_logical_plan_2c, 7);
+ test_round_trip_logical!(round_trip_logical_plan_2d, 8);
+ test_round_trip_logical!(round_trip_logical_plan_3a, 9);
+ test_round_trip_logical!(round_trip_logical_plan_3b, 10);
+ test_round_trip_logical!(round_trip_logical_plan_3c, 11);
+ test_round_trip_logical!(round_trip_logical_plan_4a, 12);
+ test_round_trip_logical!(round_trip_logical_plan_4b, 13);
+ test_round_trip_logical!(round_trip_logical_plan_4c, 14);
+ test_round_trip_logical!(round_trip_logical_plan_5a, 15);
+ test_round_trip_logical!(round_trip_logical_plan_5b, 16);
+ test_round_trip_logical!(round_trip_logical_plan_5c, 17);
+ test_round_trip_logical!(round_trip_logical_plan_6a, 18);
+ test_round_trip_logical!(round_trip_logical_plan_6b, 19);
+ test_round_trip_logical!(round_trip_logical_plan_6c, 20);
+ test_round_trip_logical!(round_trip_logical_plan_6d, 21);
+ test_round_trip_logical!(round_trip_logical_plan_6e, 22);
+ test_round_trip_logical!(round_trip_logical_plan_6f, 23);
+ test_round_trip_logical!(round_trip_logical_plan_7a, 24);
+ test_round_trip_logical!(round_trip_logical_plan_7b, 25);
+ test_round_trip_logical!(round_trip_logical_plan_7c, 26);
+ test_round_trip_logical!(round_trip_logical_plan_8a, 27);
+ test_round_trip_logical!(round_trip_logical_plan_8b, 28);
+ test_round_trip_logical!(round_trip_logical_plan_8c, 29);
+ test_round_trip_logical!(round_trip_logical_plan_8d, 30);
+ test_round_trip_logical!(round_trip_logical_plan_9a, 31);
+ test_round_trip_logical!(round_trip_logical_plan_9b, 32);
+ test_round_trip_logical!(round_trip_logical_plan_9c, 33);
+ test_round_trip_logical!(round_trip_logical_plan_9d, 34);
+ test_round_trip_logical!(round_trip_logical_plan_10a, 35);
+ test_round_trip_logical!(round_trip_logical_plan_10b, 36);
+ test_round_trip_logical!(round_trip_logical_plan_10c, 37);
+ test_round_trip_logical!(round_trip_logical_plan_11a, 38);
+ test_round_trip_logical!(round_trip_logical_plan_11b, 39);
+ test_round_trip_logical!(round_trip_logical_plan_11c, 40);
+ test_round_trip_logical!(round_trip_logical_plan_11d, 41);
+ test_round_trip_logical!(round_trip_logical_plan_12a, 42);
+ test_round_trip_logical!(round_trip_logical_plan_12b, 43);
+ test_round_trip_logical!(round_trip_logical_plan_12c, 44);
+ test_round_trip_logical!(round_trip_logical_plan_13a, 45);
+ test_round_trip_logical!(round_trip_logical_plan_13b, 46);
+ test_round_trip_logical!(round_trip_logical_plan_13c, 47);
+ test_round_trip_logical!(round_trip_logical_plan_13d, 48);
+ test_round_trip_logical!(round_trip_logical_plan_14a, 49);
+ test_round_trip_logical!(round_trip_logical_plan_14b, 50);
+ test_round_trip_logical!(round_trip_logical_plan_14c, 51);
+ test_round_trip_logical!(round_trip_logical_plan_15a, 52);
+ test_round_trip_logical!(round_trip_logical_plan_15b, 53);
+ test_round_trip_logical!(round_trip_logical_plan_15c, 54);
+ test_round_trip_logical!(round_trip_logical_plan_15d, 55);
+ test_round_trip_logical!(round_trip_logical_plan_16a, 56);
+ test_round_trip_logical!(round_trip_logical_plan_16b, 57);
+ test_round_trip_logical!(round_trip_logical_plan_16c, 58);
+ test_round_trip_logical!(round_trip_logical_plan_16d, 59);
+ test_round_trip_logical!(round_trip_logical_plan_17a, 60);
+ test_round_trip_logical!(round_trip_logical_plan_17b, 61);
+ test_round_trip_logical!(round_trip_logical_plan_17c, 62);
+ test_round_trip_logical!(round_trip_logical_plan_17d, 63);
+ test_round_trip_logical!(round_trip_logical_plan_17e, 64);
+ test_round_trip_logical!(round_trip_logical_plan_17f, 65);
+ test_round_trip_logical!(round_trip_logical_plan_18a, 66);
+ test_round_trip_logical!(round_trip_logical_plan_18b, 67);
+ test_round_trip_logical!(round_trip_logical_plan_18c, 68);
+ test_round_trip_logical!(round_trip_logical_plan_19a, 69);
+ test_round_trip_logical!(round_trip_logical_plan_19b, 70);
+ test_round_trip_logical!(round_trip_logical_plan_19c, 71);
+ test_round_trip_logical!(round_trip_logical_plan_19d, 72);
+ test_round_trip_logical!(round_trip_logical_plan_20a, 73);
+ test_round_trip_logical!(round_trip_logical_plan_20b, 74);
+ test_round_trip_logical!(round_trip_logical_plan_20c, 75);
+ test_round_trip_logical!(round_trip_logical_plan_21a, 76);
+ test_round_trip_logical!(round_trip_logical_plan_21b, 77);
+ test_round_trip_logical!(round_trip_logical_plan_21c, 78);
+ test_round_trip_logical!(round_trip_logical_plan_22a, 79);
+ test_round_trip_logical!(round_trip_logical_plan_22b, 80);
+ test_round_trip_logical!(round_trip_logical_plan_22c, 81);
+ test_round_trip_logical!(round_trip_logical_plan_22d, 82);
+ test_round_trip_logical!(round_trip_logical_plan_23a, 83);
+ test_round_trip_logical!(round_trip_logical_plan_23b, 84);
+ test_round_trip_logical!(round_trip_logical_plan_23c, 85);
+ test_round_trip_logical!(round_trip_logical_plan_24a, 86);
+ test_round_trip_logical!(round_trip_logical_plan_24b, 87);
+ test_round_trip_logical!(round_trip_logical_plan_25a, 88);
+ test_round_trip_logical!(round_trip_logical_plan_25b, 89);
+ test_round_trip_logical!(round_trip_logical_plan_25c, 90);
+ test_round_trip_logical!(round_trip_logical_plan_26a, 91);
+ test_round_trip_logical!(round_trip_logical_plan_26b, 92);
+ test_round_trip_logical!(round_trip_logical_plan_26c, 93);
+ test_round_trip_logical!(round_trip_logical_plan_27a, 94);
+ test_round_trip_logical!(round_trip_logical_plan_27b, 95);
+ test_round_trip_logical!(round_trip_logical_plan_27c, 96);
+ test_round_trip_logical!(round_trip_logical_plan_28a, 97);
+ test_round_trip_logical!(round_trip_logical_plan_28b, 98);
+ test_round_trip_logical!(round_trip_logical_plan_28c, 99);
+ test_round_trip_logical!(round_trip_logical_plan_29a, 100);
+ test_round_trip_logical!(round_trip_logical_plan_29b, 101);
+ test_round_trip_logical!(round_trip_logical_plan_29c, 102);
+ test_round_trip_logical!(round_trip_logical_plan_30a, 103);
+ test_round_trip_logical!(round_trip_logical_plan_30b, 104);
+ test_round_trip_logical!(round_trip_logical_plan_30c, 105);
+ test_round_trip_logical!(round_trip_logical_plan_31a, 106);
+ test_round_trip_logical!(round_trip_logical_plan_31b, 107);
+ test_round_trip_logical!(round_trip_logical_plan_31c, 108);
+ test_round_trip_logical!(round_trip_logical_plan_32a, 109);
+ test_round_trip_logical!(round_trip_logical_plan_32b, 110);
+ test_round_trip_logical!(round_trip_logical_plan_33a, 111);
+ test_round_trip_logical!(round_trip_logical_plan_33b, 112);
+ test_round_trip_logical!(round_trip_logical_plan_33c, 113);
+
+ // physical plan tests
+ test_round_trip_physical!(round_trip_physical_plan_1a, 1);
+ test_round_trip_physical!(round_trip_physical_plan_1b, 2);
+ test_round_trip_physical!(round_trip_physical_plan_1c, 3);
+ test_round_trip_physical!(round_trip_physical_plan_1d, 4);
+ test_round_trip_physical!(round_trip_physical_plan_2a, 5);
+ test_round_trip_physical!(round_trip_physical_plan_2b, 6);
+ test_round_trip_physical!(round_trip_physical_plan_2c, 7);
+ test_round_trip_physical!(round_trip_physical_plan_2d, 8);
+ test_round_trip_physical!(round_trip_physical_plan_3a, 9);
+ test_round_trip_physical!(round_trip_physical_plan_3b, 10);
+ test_round_trip_physical!(round_trip_physical_plan_3c, 11);
+ test_round_trip_physical!(round_trip_physical_plan_4a, 12);
+ test_round_trip_physical!(round_trip_physical_plan_4b, 13);
+ test_round_trip_physical!(round_trip_physical_plan_4c, 14);
+ test_round_trip_physical!(round_trip_physical_plan_5a, 15);
+ test_round_trip_physical!(round_trip_physical_plan_5b, 16);
+ test_round_trip_physical!(round_trip_physical_plan_5c, 17);
+ test_round_trip_physical!(round_trip_physical_plan_6a, 18);
+ test_round_trip_physical!(round_trip_physical_plan_6b, 19);
+ test_round_trip_physical!(round_trip_physical_plan_6c, 20);
+ test_round_trip_physical!(round_trip_physical_plan_6d, 21);
+ test_round_trip_physical!(round_trip_physical_plan_6e, 22);
+ test_round_trip_physical!(round_trip_physical_plan_6f, 23);
+ test_round_trip_physical!(round_trip_physical_plan_7a, 24);
+ test_round_trip_physical!(round_trip_physical_plan_7b, 25);
+ test_round_trip_physical!(round_trip_physical_plan_7c, 26);
+ test_round_trip_physical!(round_trip_physical_plan_8a, 27);
+ test_round_trip_physical!(round_trip_physical_plan_8b, 28);
+ test_round_trip_physical!(round_trip_physical_plan_8c, 29);
+ test_round_trip_physical!(round_trip_physical_plan_8d, 30);
+ test_round_trip_physical!(round_trip_physical_plan_9a, 31);
+ test_round_trip_physical!(round_trip_physical_plan_9b, 32);
+ test_round_trip_physical!(round_trip_physical_plan_9c, 33);
+ test_round_trip_physical!(round_trip_physical_plan_9d, 34);
+ test_round_trip_physical!(round_trip_physical_plan_10a, 35);
+ test_round_trip_physical!(round_trip_physical_plan_10b, 36);
+ test_round_trip_physical!(round_trip_physical_plan_10c, 37);
+ test_round_trip_physical!(round_trip_physical_plan_11a, 38);
+ test_round_trip_physical!(round_trip_physical_plan_11b, 39);
+ test_round_trip_physical!(round_trip_physical_plan_11c, 40);
+ test_round_trip_physical!(round_trip_physical_plan_11d, 41);
+ test_round_trip_physical!(round_trip_physical_plan_12a, 42);
+ test_round_trip_physical!(round_trip_physical_plan_12b, 43);
+ test_round_trip_physical!(round_trip_physical_plan_12c, 44);
+ test_round_trip_physical!(round_trip_physical_plan_13a, 45);
+ test_round_trip_physical!(round_trip_physical_plan_13b, 46);
+ test_round_trip_physical!(round_trip_physical_plan_13c, 47);
+ test_round_trip_physical!(round_trip_physical_plan_13d, 48);
+ test_round_trip_physical!(round_trip_physical_plan_14a, 49);
+ test_round_trip_physical!(round_trip_physical_plan_14b, 50);
+ test_round_trip_physical!(round_trip_physical_plan_14c, 51);
+ test_round_trip_physical!(round_trip_physical_plan_15a, 52);
+ test_round_trip_physical!(round_trip_physical_plan_15b, 53);
+ test_round_trip_physical!(round_trip_physical_plan_15c, 54);
+ test_round_trip_physical!(round_trip_physical_plan_15d, 55);
+ test_round_trip_physical!(round_trip_physical_plan_16a, 56);
+ test_round_trip_physical!(round_trip_physical_plan_16b, 57);
+ test_round_trip_physical!(round_trip_physical_plan_16c, 58);
+ test_round_trip_physical!(round_trip_physical_plan_16d, 59);
+ test_round_trip_physical!(round_trip_physical_plan_17a, 60);
+ test_round_trip_physical!(round_trip_physical_plan_17b, 61);
+ test_round_trip_physical!(round_trip_physical_plan_17c, 62);
+ test_round_trip_physical!(round_trip_physical_plan_17d, 63);
+ test_round_trip_physical!(round_trip_physical_plan_17e, 64);
+ test_round_trip_physical!(round_trip_physical_plan_17f, 65);
+ test_round_trip_physical!(round_trip_physical_plan_18a, 66);
+ test_round_trip_physical!(round_trip_physical_plan_18b, 67);
+ test_round_trip_physical!(round_trip_physical_plan_18c, 68);
+ test_round_trip_physical!(round_trip_physical_plan_19a, 69);
+ test_round_trip_physical!(round_trip_physical_plan_19b, 70);
+ test_round_trip_physical!(round_trip_physical_plan_19c, 71);
+ test_round_trip_physical!(round_trip_physical_plan_19d, 72);
+ test_round_trip_physical!(round_trip_physical_plan_20a, 73);
+ test_round_trip_physical!(round_trip_physical_plan_20b, 74);
+ test_round_trip_physical!(round_trip_physical_plan_20c, 75);
+ test_round_trip_physical!(round_trip_physical_plan_21a, 76);
+ test_round_trip_physical!(round_trip_physical_plan_21b, 77);
+ test_round_trip_physical!(round_trip_physical_plan_21c, 78);
+ test_round_trip_physical!(round_trip_physical_plan_22a, 79);
+ test_round_trip_physical!(round_trip_physical_plan_22b, 80);
+ test_round_trip_physical!(round_trip_physical_plan_22c, 81);
+ test_round_trip_physical!(round_trip_physical_plan_22d, 82);
+ test_round_trip_physical!(round_trip_physical_plan_23a, 83);
+ test_round_trip_physical!(round_trip_physical_plan_23b, 84);
+ test_round_trip_physical!(round_trip_physical_plan_23c, 85);
+ test_round_trip_physical!(round_trip_physical_plan_24a, 86);
+ test_round_trip_physical!(round_trip_physical_plan_24b, 87);
+ test_round_trip_physical!(round_trip_physical_plan_25a, 88);
+ test_round_trip_physical!(round_trip_physical_plan_25b, 89);
+ test_round_trip_physical!(round_trip_physical_plan_25c, 90);
+ test_round_trip_physical!(round_trip_physical_plan_26a, 91);
+ test_round_trip_physical!(round_trip_physical_plan_26b, 92);
+ test_round_trip_physical!(round_trip_physical_plan_26c, 93);
+ test_round_trip_physical!(round_trip_physical_plan_27a, 94);
+ test_round_trip_physical!(round_trip_physical_plan_27b, 95);
+ test_round_trip_physical!(round_trip_physical_plan_27c, 96);
+ test_round_trip_physical!(round_trip_physical_plan_28a, 97);
+ test_round_trip_physical!(round_trip_physical_plan_28b, 98);
+ test_round_trip_physical!(round_trip_physical_plan_28c, 99);
+ test_round_trip_physical!(round_trip_physical_plan_29a, 100);
+ test_round_trip_physical!(round_trip_physical_plan_29b, 101);
+ test_round_trip_physical!(round_trip_physical_plan_29c, 102);
+ test_round_trip_physical!(round_trip_physical_plan_30a, 103);
+ test_round_trip_physical!(round_trip_physical_plan_30b, 104);
+ test_round_trip_physical!(round_trip_physical_plan_30c, 105);
+ test_round_trip_physical!(round_trip_physical_plan_31a, 106);
+ test_round_trip_physical!(round_trip_physical_plan_31b, 107);
+ test_round_trip_physical!(round_trip_physical_plan_31c, 108);
+ test_round_trip_physical!(round_trip_physical_plan_32a, 109);
+ test_round_trip_physical!(round_trip_physical_plan_32b, 110);
+ test_round_trip_physical!(round_trip_physical_plan_33a, 111);
+ test_round_trip_physical!(round_trip_physical_plan_33b, 112);
+ test_round_trip_physical!(round_trip_physical_plan_33c, 113);
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org