You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by "Wenhai Li (Code Review)" <do...@asterixdb.incubator.apache.org> on 2017/03/10 19:10:33 UTC
Change in asterixdb[master]: RangeGenerator aggfunc for the numeric/asciiString.
Hello Michael Blow, Preston Carman, Jenkins,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/806
to look at the new patch set (#62).
Change subject: RangeGenerator aggfunc for the numeric/asciiString.
......................................................................
RangeGenerator aggfunc for the numeric/asciiString.
The following commits from your working branch will be included:
Extract and review for the streaming histogram based parallel sort.
1. Add the streaming histogram for hyracks datatype.
2. Include the ASCII histogram.
3. Inline the rangegenerator inside the AggFuncs.
4. Add the testcases including the random, zipfan, gaussian int/short/long/double/string.
5. Support the null omittion.
Change-Id: I450d0962fbeacfb2b6ab9fae0750f025ef17ba01
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAggFuncIntoStandaloneAggregateRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/FunctionCollection.java
A asterixdb/asterix-app/data/skew/zipfian.tbl
A asterixdb/asterix-app/data/skew/zipfian1.tbl
A asterixdb/asterix-app/data/skew/zipfian2.tbl
A asterixdb/asterix-app/data/skew/zipfianmissing_1.tbl
A asterixdb/asterix-app/data/skew/zipfianmissing_2.tbl
A asterixdb/asterix-app/data/skew/zipfiannull_1.tbl
A asterixdb/asterix-app/data/skew/zipfiannull_2.tbl
A asterixdb/asterix-app/data/tpch0.001/lineitem-part1.tbl
A asterixdb/asterix-app/data/tpch0.001/lineitem-part2.tbl
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_float/avg_float.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int16/avg_int16.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int32/avg_int32.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int64/avg_int64.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int8/avg_int8.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/count_01/count_01.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/global-avg_01/global-avg_01.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/global-rg/global-rg.1.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_double/local-avg_double.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_float/local-avg_float.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int16/local-avg_int16.1.ddl.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int16/local-avg_int16.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int32/local-avg_int32.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int64/local-avg_int64.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int8/local-avg_int8.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-rg/local-rg.1.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg-double-null/rg-double-null.1.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg-double/rg-double.1.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double_missing/rg_double_missing.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double_missing/rg_double_missing.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double_missing/rg_double_missing.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16/rg_int16.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16/rg_int16.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16/rg_int16.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_missing/rg_int16_missing.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_missing/rg_int16_missing.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_missing/rg_int16_missing.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_null/rg_int16_null.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_null/rg_int16_null.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_null/rg_int16_null.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_seminull/rg_int16_seminull.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_seminull/rg_int16_seminull.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_seminull/rg_int16_seminull.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32/rg_int32.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32/rg_int32.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32/rg_int32.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32_missing/rg_int32_missing.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32_missing/rg_int32_missing.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32_missing/rg_int32_missing.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64/rg_int64.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64/rg_int64.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64/rg_int64.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64_missing/rg_int64_missing.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64_missing/rg_int64_missing.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64_missing/rg_int64_missing.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_shipdate/rg_shipdate.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_shipdate/rg_shipdate.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_shipdate/rg_shipdate.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string/rg_string.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string/rg_string.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string/rg_string.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string_missing/rg_string_missing.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string_missing/rg_string_missing.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string_missing/rg_string_missing.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_double/sum_double.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_float/sum_float.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_int16/sum_int16.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_int64/sum_int64.3.query.aql
M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_int8/sum_int8.3.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/load/rg_load/rg_load.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/load/rg_load/rg_load.2.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/load/rg_load/rg_load.3.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/range-hints/order-by-exception_03/order-by-exception_03.1.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/range-hints/order-by-exception_03/order-by-exception_03.2.ddl.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/range-hints/order-by-exception_03/order-by-exception_03.3.update.aql
A asterixdb/asterix-app/src/test/resources/runtimets/queries/range-hints/order-by-exception_03/order-by-exception_03.4.query.aql
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/global-rg/global-avg.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/local-rg/local-rg.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg-double-null/rg-double-null.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg-double/rg-double.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_double/rg_double.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_double_missing/rg_double_missing.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int16/rg_int16.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int16_missing/rg_int16_missing.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int16_null/rg_int16_null.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int16_seminull/rg_int16_seminull.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int32/rg_int32.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int32_missing/rg_int32_missing.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int64/rg_int64.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int64_missing/rg_int64_missing.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_shipdate/rg_shipdate.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_string/rg_string.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_string_missing/rg_string_missing.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/range-hints/order-by-exception_03/order-by-exception_03.4.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/GlobalRangeGeneratorTypeComputer.java
A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/LocalRangeGeneratorTypeComputer.java
A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RangeGeneratorTypeComputer.java
M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/ScalarVersionOfAggregateResultType.java
M asterixdb/asterix-om/src/test/java/org/apache/asterix/om/typecomputer/ExceptionTest.java
M asterixdb/asterix-om/src/test/java/org/apache/asterix/om/typecomputer/TypeComputerTest.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/histogram/AsterixStreamingHistogram.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarRangeGuiderAggregateDescriptor.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractRangeGuiderAggregateFunction.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalRangeGuiderAggregateDescriptor.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalRangeGuiderAggregateFunction.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/LocalRangeGuiderAggregateDescriptor.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/LocalRangeGuiderAggregateFunction.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/RangeGuiderAggregateDescriptor.java
A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/RangeGuiderAggregateFunction.java
M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceCombinerRule.java
M hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/range/FieldRangePartitionComputerFactory.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/HistogramType.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/IHeapList.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/IHistogram.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/IQuantileList.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/AbstractQuantileList.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/AbstractStreamingHistogram.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/DominantQuantile.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/GenericQuantile.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/GenericQuantileComparator.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/MinDifferenceQuantileList.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/MinSumCountQuantileList.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/util/HistogramUtils.java
A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/test/java/org/apache/hyracks/dataflow/std/range/TypeTranslateTest.java
A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/data/skew/prefix.tbl
A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/data/skew/zipfian1.tbl
A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/data/skew/zipfian2.tbl
M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/pom.xml
A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/HistogramAlgorithmTests.java
A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/HistogramFieldType.java
A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/HyracksStreamingHistogram.java
137 files changed, 22,970 insertions(+), 30 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/06/806/62
--
To view, visit https://asterix-gerrit.ics.uci.edu/806
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I450d0962fbeacfb2b6ab9fae0750f025ef17ba01
Gerrit-PatchSet: 62
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li <lw...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Jianfeng Jia <ji...@gmail.com>
Gerrit-Reviewer: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Preston Carman <pr...@apache.org>
Gerrit-Reviewer: Wenhai Li <lw...@gmail.com>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: Yingyi Bu <yi...@google.com>