You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by sn...@apache.org on 2018/11/29 06:12:15 UTC
[incubator-pinot] branch bloomfilter updated (4cb5ab8 -> 469756c)
This is an automated email from the ASF dual-hosted git repository.
snlee pushed a change to branch bloomfilter
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
from 4cb5ab8 Fixing failing test cases
add 781f876 [TE] rootcause - support metric selection from chart labels (#3526)
add 7cb8054 [TE] Attempt to organize and cleanup the email template (#3527)
add e51ffdb Ignore generated documentation files while checking for license headers (#3524)
add 0d3ef38 [TE] datasource - aggregate to one minute level for epoch timestamp (#3530)
add 5925efa [TE] rootcause - primary metric selection from legend label (#3529)
add f47b260 [TE] rootcause - support remove metric from chart label (#3531)
add 85221a2 [TE] Some more minor changes to the look of the template (#3534)
add 4c80614 [TE] roocause - fix dimension table metric select/unselect (#3533)
add 86532ca [TE] frontend - harleyjj/rootcause - fix data inconsistency warning t… (#3532)
add a017ade [TE] Anomaly function to new pipeline YAML converter (#3537)
add 1f1f9b3 [TE] rootcause - prevent metric selection change on dimension table tab change (#3536)
add e19324b [TE] detection - add more logs for merger for debugging (#3540)
add 60bd449 [TE] Pipeline migration - fill in metric timezone (#3542)
add 6a3a46a [TE] Display Holidays only from Top Countries (#3544)
add 0e133b8 [TE] rootcause - remove tooltipEntities property (#3546)
add af24df1 [TE] rootcause - redo route hooks (#3547)
add 9e352da [TE] frontend - harleyjj/rootcause - warning value uses aggregate multiplier and can be turned off if aggregates change without slider being adjusted (#3548)
add c422157 First cut documentation. (#3545)
add 4880bf8 Adding execution stats for numSegmentsQueried/Processed/Matched (#3525)
add 9b1abcb [TE] rootcause - default millisecond datasets to 15min granularity (#3550)
add 3a58e16 Optimize all filter predicates by adding isAlwaysTrue() (#3535)
add 6f41307 [TE] Alerts created by YAML to show up in the alert list (#3551)
add 2de0718 Fix the wrong number reported by numSegmentsQueried (#3552)
add 469756c Updated bloom filter implementation 1. Changed the implementation to use Guava BloomFilter library 2. Added unit test for bloom filter 3. Enforcing the maxmimum size of bloom filter to 1MB
No new revisions were added by this update.
Summary of changes:
docs/High-level-stream.png | Bin 0 -> 39800 bytes
docs/Low-level-stream.png | Bin 0 -> 37306 bytes
docs/Pinot - Hybrid flow.png | Bin 0 -> 15756 bytes
docs/Pinot-Offline-only-flow.png | Bin 0 -> 8179 bytes
docs/PlanNode.png | Bin 0 -> 41056 bytes
docs/ServerSegmentCompletion.dot.png | Bin 0 -> 72968 bytes
docs/architecture.rst | 106 +++++++
docs/client_api.rst | 227 ++++++++++++++
docs/commit-happy-path-1.png | Bin 0 -> 22933 bytes
docs/commit-happy-path-2.png | Bin 0 -> 22939 bytes
docs/committer-failed.png | Bin 0 -> 15412 bytes
docs/controller-failed.png | Bin 0 -> 23599 bytes
docs/controller-segment-completion.png | Bin 0 -> 85092 bytes
docs/creating_pinot_segments.rst | 98 ++++++
docs/delayed-server.png | Bin 0 -> 21690 bytes
docs/expressionTree.jpg | Bin 0 -> 12881 bytes
docs/expressions_udf.rst | 119 ++++++++
docs/hlc-arch.png | Bin 0 -> 47924 bytes
docs/image2014-11-12-1-54-56.png | Bin 0 -> 112436 bytes
docs/image2014-11-12-19-54-12.png | Bin 0 -> 54744 bytes
docs/image2014-11-12-20-59-50.png | Bin 0 -> 36925 bytes
docs/image2014-11-13-11-21-7.png | Bin 0 -> 99075 bytes
docs/image2014-11-13-11-22-57.png | Bin 0 -> 27562 bytes
docs/image2015-4-14-194451.png | Bin 0 -> 83171 bytes
docs/image2015-5-12-7-24-58.png | Bin 0 -> 38969 bytes
docs/image2015-5-13 15-2-9.png | Bin 0 -> 41882 bytes
docs/image2015-5-17 13-32-28.png | Bin 0 -> 89700 bytes
docs/image2015-5-17 17-59-10.png | Bin 0 -> 82542 bytes
docs/image2015-5-19 0-29-34.png | Bin 0 -> 39085 bytes
docs/image2015-5-19 0-58-54.png | Bin 0 -> 54859 bytes
docs/image2015-5-19 1-28-35.png | Bin 0 -> 34374 bytes
docs/image2015-5-19 1-47-44.png | Bin 0 -> 34374 bytes
docs/image2015-5-19 1-59-51.png | Bin 0 -> 37098 bytes
docs/index.rst | 27 +-
docs/intro.rst | 53 +++-
docs/llc-arch.png | Bin 0 -> 39899 bytes
docs/llc.rst | 150 +++++++++
docs/management_api.rst | 6 +
docs/multiple-server-failure.png | Bin 0 -> 18036 bytes
docs/multitenancy.rst | 340 +++++++++++++++++++++
docs/parseTree.png | Bin 0 -> 55136 bytes
docs/partition_aware_routing.rst | 141 +++++++++
docs/pinot-architecture.png | Bin 0 -> 58474 bytes
docs/pinot_hadoop.rst | 72 +++++
docs/pluggable_streams.rst | 126 ++++++++
docs/pql_examples.rst | 234 ++++++++++++++
docs/reference.rst | 42 +--
docs/schema_timespec.rst | 109 +++++++
docs/segment-consumer-fsm.png | Bin 0 -> 30109 bytes
docs/segment-creation.png | Bin 0 -> 9830 bytes
docs/segment-helix-fsm.png | Bin 0 -> 7494 bytes
docs/segment_fetcher.rst | 77 +++++
docs/trying_pinot.rst | 14 +-
docs/zk-setup.png | Bin 0 -> 39246 bytes
.../requesthandler/BaseBrokerRequestHandler.java | 3 +-
.../linkedin/pinot/common/metrics/ServerMeter.java | 4 +-
.../pinot/common/response/BrokerResponse.java | 15 +
.../response/broker/BrokerResponseNative.java | 42 ++-
.../pinot/common/segment/SegmentMetadata.java | 1 -
.../com/linkedin/pinot/common/utils/DataTable.java | 3 +
.../indexsegment/mutable/MutableSegmentImpl.java | 3 +-
.../core/operator/CombineGroupByOperator.java | 2 +
.../pinot/core/operator/CombineOperator.java | 2 +
.../pinot/core/operator/ExecutionStatistics.java | 20 +-
.../operator/blocks/IntermediateResultsBlock.java | 21 ++
.../docidsets/BitmapBasedBlockDocIdSet.java | 41 ---
.../core/operator/filter/AndFilterOperator.java | 25 +-
.../core/operator/filter/BaseFilterOperator.java | 13 +-
.../operator/filter/BitmapBasedFilterOperator.java | 15 +-
.../core/operator/filter/EmptyFilterOperator.java | 8 +-
.../core/operator/filter/FilterOperatorUtils.java | 65 +++-
.../operator/filter/MatchAllFilterOperator.java | 8 +-
.../core/operator/filter/OrFilterOperator.java | 25 +-
.../operator/filter/ScanBasedFilterOperator.java | 16 +-
.../SortedInvertedIndexBasedFilterOperator.java | 15 +-
.../BaseDictionaryBasedPredicateEvaluator.java | 12 +
.../BaseRawValueBasedPredicateEvaluator.java | 15 +-
.../predicate/EqualsPredicateEvaluatorFactory.java | 9 +-
.../predicate/InPredicateEvaluatorFactory.java | 14 +-
.../NotEqualsPredicateEvaluatorFactory.java | 9 +-
.../predicate/NotInPredicateEvaluatorFactory.java | 16 +-
.../filter/predicate/PredicateEvaluator.java | 5 +
.../predicate/RangePredicateEvaluatorFactory.java | 33 +-
.../RegexpLikePredicateEvaluatorFactory.java | 5 -
.../linkedin/pinot/core/plan/FilterPlanNode.java | 37 ++-
.../query/executor/ServerQueryExecutorV1Impl.java | 12 +-
.../core/query/pruner/AbstractSegmentPruner.java | 4 +-
.../query/pruner/ColumnValueSegmentPruner.java | 22 +-
.../core/query/pruner/PartitionSegmentPruner.java | 6 +-
.../core/query/reduce/BrokerReduceService.java | 20 ++
.../core/query/request/ServerQueryRequest.java | 8 -
.../pinot/core/query/scheduler/QueryScheduler.java | 23 +-
.../creator/impl/bloom/BloomFilterCreator.java | 44 ++-
.../creator/impl/bloom/BloomFilterUtil.java | 35 +++
.../index/data/source/ColumnDataSource.java | 6 +-
.../loader/bloomfilter/BloomFilterHandler.java | 102 +++----
.../segment/index/readers/BloomFilterReader.java | 18 +-
.../segment/store/SegmentLocalFSDirectory.java | 1 -
.../startree/operator/StarTreeFilterOperator.java | 58 ++--
.../v2/store/StarTreeMetricDataSource.java | 4 +-
.../operator/filter/AndFilterOperatorTest.java | 27 +-
.../operator/filter/FilterOperatorUtilsTest.java | 98 ++++++
.../{ => core}/operator/filter/IntRangesTest.java | 3 +-
.../operator/filter/OrFilterOperatorTest.java | 20 +-
.../core/operator/filter/TestFilterOperator.java | 78 +++++
...ngeOfflineDictionaryPredicateEvaluatorTest.java | 44 ++-
.../index/creator/BloomFilterCreatorTest.java | 158 ++++++----
.../operator/filter/FilterOperatorTestUtils.java | 85 ------
...nerSegmentAggregationMultiValueQueriesTest.java | 12 +-
...InnerSegmentSelectionMultiValueQueriesTest.java | 6 +-
...terSegmentAggregationMultiValueQueriesTest.java | 32 +-
.../tests/BaseClusterIntegrationTestSet.java | 23 +-
.../tests/HybridClusterIntegrationTest.java | 6 +
...onaryAggregationPlanClusterIntegrationTest.java | 31 +-
.../tests/OfflineClusterIntegrationTest.java | 6 +
.../com/linkedin/pinot/perf/RawIndexBenchmark.java | 7 +-
pom.xml | 2 +
.../app/mirage/endpoints/selfserve.js | 14 +
.../pods/components/rootcause-anomaly/component.js | 63 +++-
.../pods/components/rootcause-anomaly/template.hbs | 2 +-
.../pods/components/rootcause-chart/component.js | 49 ++-
.../pods/components/rootcause-chart/template.hbs | 11 +-
.../rootcause-dimensions-table/component.js | 18 +-
.../pods/components/rootcause-legend/component.js | 42 +--
.../pods/components/rootcause-legend/template.hbs | 22 +-
.../app/pods/manage/alerts/index/route.js | 64 +++-
.../app/pods/rootcause/controller.js | 23 +-
.../thirdeye-frontend/app/pods/rootcause/route.js | 84 ++---
.../app/pods/rootcause/template.hbs | 4 +
.../app/styles/components/rootcause-chart.scss | 25 ++
.../app/styles/components/rootcause-legend.scss | 4 +
.../components/rootcause-anomaly/component-test.js | 91 +++++-
.../alert/content/BaseEmailContentFormatter.java | 16 +-
.../EmailContentFormatterConfiguration.java | 11 +
...HierarchicalAnomaliesEmailContentFormatter.java | 7 +-
.../MultipleAnomaliesEmailContentFormatter.java | 9 +-
.../anomaly/ThirdEyeAnomalyConfiguration.java | 10 +
.../thirdeye/anomaly/events/EventFilter.java | 7 +-
.../anomaly/events/HolidayEventProvider.java | 17 +-
.../anomaly/merge/AnomalyTimeBasedSummarizer.java | 10 +-
.../anomaly/merge/TimeBasedAnomalyMerger.java | 5 +-
.../dashboard/ThirdEyeDashboardApplication.java | 2 +-
.../thirdeye/datasource/pinot/PqlUtils.java | 13 +-
.../detection/DetectionMigrationResource.java | 173 ++++++++++-
.../alert/scheme/DetectionEmailAlerter.java | 2 +-
.../detection/algorithm/DimensionWrapper.java | 63 +++-
.../detection/annotation/DetectionRegistry.java | 2 +-
.../thirdeye/detection/spi/components/Tunable.java | 2 +-
.../detection/wrapper/AnomalyDetectorWrapper.java | 31 +-
.../wrapper/BaselineFillingMergeWrapper.java | 9 +-
.../yaml/CompositePipelineConfigTranslator.java | 107 +++++--
.../yaml/YamlDetectionConfigTranslator.java | 2 +-
.../thirdeye/detection/yaml/YamlResource.java | 41 ++-
.../thirdeye/detector/holiday-anomaly-report.ftl | 122 ++++----
.../com/linkedin/thirdeye/detector/lib/utils.ftl | 4 +-
.../CompositePipelineConfigTranslatorTest.java | 3 +-
.../eventprovider/TestHolidayEventProvider.java | 1 +
.../compositePipelineTranslatorTestResult-1.json | 59 ++--
.../compositePipelineTranslatorTestResult-2.json | 15 +-
.../thirdeye/detection/yaml/pipeline-config-1.yaml | 8 +
...multiple-anomalies-email-content-formatter.html | 68 ++---
161 files changed, 3789 insertions(+), 990 deletions(-)
create mode 100644 docs/High-level-stream.png
create mode 100644 docs/Low-level-stream.png
create mode 100644 docs/Pinot - Hybrid flow.png
create mode 100644 docs/Pinot-Offline-only-flow.png
create mode 100644 docs/PlanNode.png
create mode 100644 docs/ServerSegmentCompletion.dot.png
create mode 100644 docs/architecture.rst
create mode 100644 docs/client_api.rst
create mode 100644 docs/commit-happy-path-1.png
create mode 100644 docs/commit-happy-path-2.png
create mode 100644 docs/committer-failed.png
create mode 100644 docs/controller-failed.png
create mode 100644 docs/controller-segment-completion.png
create mode 100644 docs/creating_pinot_segments.rst
create mode 100644 docs/delayed-server.png
create mode 100644 docs/expressionTree.jpg
create mode 100644 docs/expressions_udf.rst
create mode 100644 docs/hlc-arch.png
create mode 100644 docs/image2014-11-12-1-54-56.png
create mode 100644 docs/image2014-11-12-19-54-12.png
create mode 100644 docs/image2014-11-12-20-59-50.png
create mode 100644 docs/image2014-11-13-11-21-7.png
create mode 100644 docs/image2014-11-13-11-22-57.png
create mode 100644 docs/image2015-4-14-194451.png
create mode 100644 docs/image2015-5-12-7-24-58.png
create mode 100644 docs/image2015-5-13 15-2-9.png
create mode 100644 docs/image2015-5-17 13-32-28.png
create mode 100644 docs/image2015-5-17 17-59-10.png
create mode 100644 docs/image2015-5-19 0-29-34.png
create mode 100644 docs/image2015-5-19 0-58-54.png
create mode 100644 docs/image2015-5-19 1-28-35.png
create mode 100644 docs/image2015-5-19 1-47-44.png
create mode 100644 docs/image2015-5-19 1-59-51.png
create mode 100644 docs/llc-arch.png
create mode 100644 docs/llc.rst
create mode 100644 docs/management_api.rst
create mode 100644 docs/multiple-server-failure.png
create mode 100644 docs/multitenancy.rst
create mode 100644 docs/parseTree.png
create mode 100644 docs/partition_aware_routing.rst
create mode 100644 docs/pinot-architecture.png
create mode 100644 docs/pinot_hadoop.rst
create mode 100644 docs/pluggable_streams.rst
create mode 100644 docs/pql_examples.rst
create mode 100644 docs/schema_timespec.rst
create mode 100644 docs/segment-consumer-fsm.png
create mode 100644 docs/segment-creation.png
create mode 100644 docs/segment-helix-fsm.png
create mode 100644 docs/segment_fetcher.rst
create mode 100644 docs/zk-setup.png
delete mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/operator/docidsets/BitmapBasedBlockDocIdSet.java
create mode 100644 pinot-core/src/main/java/com/linkedin/pinot/core/segment/creator/impl/bloom/BloomFilterUtil.java
rename pinot-core/src/test/java/com/linkedin/pinot/{ => core}/operator/filter/AndFilterOperatorTest.java (77%)
create mode 100644 pinot-core/src/test/java/com/linkedin/pinot/core/operator/filter/FilterOperatorUtilsTest.java
rename pinot-core/src/test/java/com/linkedin/pinot/{ => core}/operator/filter/IntRangesTest.java (96%)
rename pinot-core/src/test/java/com/linkedin/pinot/{ => core}/operator/filter/OrFilterOperatorTest.java (82%)
create mode 100644 pinot-core/src/test/java/com/linkedin/pinot/core/operator/filter/TestFilterOperator.java
delete mode 100644 pinot-core/src/test/java/com/linkedin/pinot/operator/filter/FilterOperatorTestUtils.java
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org