You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by ja...@apache.org on 2019/09/27 02:39:11 UTC
[incubator-pinot] branch map_transform updated (416e8d4 -> 392cfce)
This is an automated email from the ASF dual-hosted git repository.
jackie pushed a change to branch map_transform
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard 416e8d4 Implement all transform function APIs for MapValueTransformFunction
add 3c2f0b3 Adding example for kubernetes deployment on GKE (#4581)
add dbcbd2e [TE] Empty Entity report bug fix (#4598)
add 5e525d0 [TE] Exclude template/blacklist emails before sending out alert (#4601)
add 0a32309 [TE] fix create alert error message NPE (#4594)
add e70a781 Add ApacheCon Slide to README file (#4604)
add 67ea9bb [TE] Merger fix to consider anomalies generated earlier (#4565)
add f5017be [TE] Redesign to support multiple Alert Content formatters (#4599)
add add0f07 [TE] Update AI Availability table (#4606)
add fe7fff6 [TE] Add anomaly type for merged anomaly result (#4605)
add 8f9bb5d [TE] Remove the logic of reading stderr from phantomjs to avoid zombie processes (#4611)
add 45205e8 [TE] frontend - harleyjj/alert-details - handle reported anomaly for anomaly table (#4587)
add fe29287 [TE] frontend - harleyjj/auth - handle expired token and 401 errors plus redirect after login (#4580)
add 76fcc86 [TE] frontend - harleyjj/ai - fix small bug in AI Availability (#4600)
add 09207f9 [TE] consider anomaly type during merge (#4613)
add 8d593f3 [TE] Bug fix to initialize the configDAO (#4616)
add c7b647a [TE][Composite-Alert] Entity Anomaly Merger (#4609)
add c87b110 [TE] Add data set display name and show it on UI (#4619)
add 239237e [TE] the last time stamp filter out the anomalies (#4621)
add 44d1ce9 Remove import of sun.misc.Cleaner to make code compatible with jdk11 (#4624)
add 04c4dc2 Revert "Remove import of sun.misc.Cleaner to make code compatible with jdk11 (#4624)" (#4625)
add eda4176 [TE] [Notification] Refactor email filters to a generalized alert filter (#4620)
add 3668c79 Documentation update for realtime Pinot (#4630)
add f8c2cf3 Put order-by expressions in front of selection expreesions in selection DataSchema (#4629)
add a0c84fe Implement DISTINCT (#4535)
add 761ce2f Add APIs to get leader for all tables or a given table (#4545)
add 31594da Replace Hashmap to Object2IntOpenHashMap in OnHeapStringDictionary (#4568)
add 283a495 Revert "Set processingException when all queried segments cannot be acquired (#3942)" (#4632)
add 49a65a3 Support default value for Byte column (#4583)
add e447117 Remove import of sun.misc.Cleaner to make code compatible with jdk11 (#4634)
add ffa66bf Refactor ControllerLeaderLocator (#4553)
add f30d83b Adding documentation for Pinot Schema. (#4637)
add 07c74b1 User doc for Star-Tree index (#4640)
add c563b82 Add license header for star-tree document (#4641)
add bd1af57 Adding section describing defaultNullValue in Schema page. (#4642)
add 4c4800b Support HTTP POST/PUT JSON schema (#4639)
add 09d3735 Remove sun.misc.Cleaner import (#4643)
add a734931 Add metrics for the number of partitions for which controller becomes leader (#4633)
add c44d371 [TE] fix email alerter throw exception (#4647)
add 0b4fc0c Documentation for completionConfig (#4636)
add ca01c86 [TE] frontend - harleyjj/rca - make first column scrollable in dimensions-algorithm table (#4631)
add 577f1cf Do not update download URL and crypter when segment is not refreshed (#4593)
add d30dfb0 Standardize the Dictionary interface, ensure the BYTES support (#4622)
add 4b5c5ac Misc fixes for aggregation functions (#4579)
add b0fe681 Support ORC files as data source in segment creation (#4635)
add bfb518e Add integration test for ControllerLeaderLocator (#4567)
add b24816a Adding sample ADL settings to documentation (#4645)
add dd4e69e Use timed wait around get() calls in PinotInstance (#4649)
add 9cfcd30 Simplify the logic of List handling in DataTypeTransformer (#4651)
add 392cfce Implement all transform function APIs for MapValueTransformFunction
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (416e8d4)
\
N -- N -- N refs/heads/map_transform (392cfce)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
README.md | 7 +-
docs/admin_guide.rst | 1 +
docs/architecture.rst | 87 +-
docs/code_modules.rst | 5 +-
docs/contribution_guidelines.rst | 3 +-
docs/index.rst | 2 +-
docs/pluggable_storage.rst | 42 +-
docs/pluggable_streams.rst | 9 +-
docs/pql_examples.rst | 9 +-
docs/schema.rst | 138 +
docs/segment_fetcher.rst | 2 +
docs/star-tree/example.png | Bin 0 -> 29005 bytes
docs/star-tree/space-time.png | Bin 0 -> 25225 bytes
docs/star-tree/star-tree.rst | 349 +
docs/star-tree/structure.png | Bin 0 -> 25841 bytes
docs/tableconfig_schema.rst | 191 +-
docs/tuning_pinot.rst | 14 +-
docs/tuning_realtime_performance.rst | 84 +
kubernetes/README.md | 27 +
kubernetes/examples/gke/skaffold/README.md | 119 +
kubernetes/examples/gke/skaffold/cleanup.sh | 39 +
.../gke/skaffold/gke-storageclass-kafka-pd.yml | 28 +
.../gke-storageclass-pinot-controller-pd.yml | 28 +
.../skaffold/gke-storageclass-pinot-server-pd.yml | 28 +
.../gke/skaffold/gke-storageclass-zk-pd.yml | 28 +
kubernetes/examples/gke/skaffold/kafka.yml | 408 +
kubernetes/examples/gke/skaffold/pinot-broker.yml | 88 +
.../examples/gke/skaffold/pinot-controller.yml | 106 +
.../examples/gke/skaffold/pinot-example-loader.yml | 43 +
kubernetes/examples/gke/skaffold/pinot-server.yml | 101 +
.../examples/gke/skaffold/query-pinot-data.sh | 28 +
kubernetes/examples/gke/skaffold/setup.sh | 51 +
kubernetes/examples/gke/skaffold/skaffold.yaml | 35 +
kubernetes/examples/gke/skaffold/zookeeper.yml | 80 +
.../requesthandler/BaseBrokerRequestHandler.java | 19 +-
.../broker/broker/HelixBrokerStarterTest.java | 2 +-
.../pinot/common/exception/QueryException.java | 2 -
.../common/function/AggregationFunctionType.java | 58 +-
.../function/FunctionDefinitionRegistry.java | 6 +-
.../pinot/common/metrics/ControllerGauge.java | 3 +
.../apache/pinot/common/query/ReduceService.java | 4 +-
.../apache/pinot/common/request/BrokerRequest.java | 199 +-
.../org/apache/pinot/common/utils/BytesUtils.java | 31 +-
.../org/apache/pinot/common/utils/DataSchema.java | 25 +-
.../org/apache/pinot/common/utils/DataTable.java | 12 +-
.../common/utils/helix/LeadControllerUtils.java | 49 +
.../parsers/PinotQuery2BrokerRequestConverter.java | 84 +-
.../apache/pinot/pql/parsers/Pql2AstListener.java | 2 +-
.../org/apache/pinot/pql/parsers/Pql2Compiler.java | 3 +-
.../pql/parsers/pql2/ast/FunctionCallAstNode.java | 59 +-
.../pinot/pql/parsers/pql2/ast/OrderByAstNode.java | 57 +-
.../parsers/pql2/ast/OutputColumnListAstNode.java | 91 +
.../pinot/pql/parsers/pql2/ast/SelectAstNode.java | 25 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java | 55 +-
.../apache/pinot/common/data/FieldSpecTest.java | 11 +
.../function/AggregationFunctionTypeTest.java | 3 +-
.../apache/pinot/pql/parsers/Pql2CompilerTest.java | 68 +-
.../request/BrokerRequestSerializationTest.java | 9 +-
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 327 +
pinot-common/src/thrift/request.thrift | 1 +
.../pinot/controller/LeadControllerManager.java | 39 +-
.../pinot/controller/api/resources/Constants.java | 1 +
.../PinotLeadControllerRestletResource.java | 202 +
.../api/resources/PinotSchemaRestletResource.java | 51 +-
.../pinot/controller/api/upload/ZKOperator.java | 18 +-
.../helix/core/PinotHelixResourceManager.java | 6 +-
.../controller/LeadControllerManagerTest.java | 17 +-
.../api/PinotInstanceRestletResourceTest.java | 141 +-
.../api/PinotSchemaRestletResourceTest.java | 30 +
.../controller/api/upload/ZKOperatorTest.java | 141 +
.../pinot/controller/helix/ControllerTest.java | 11 +
.../controller/helix/PinotControllerModeTest.java | 23 +-
.../helix/core/retention/RetentionManagerTest.java | 7 +-
.../validation/ValidationManagerTest.java | 5 +-
.../org/apache/pinot/core/common/DataFetcher.java | 22 +-
.../apache/pinot/core/common/ObjectSerDeUtils.java | 54 +-
.../core/common/datatable/DataTableBuilder.java | 3 +-
.../core/common/datatable/DataTableImplV2.java | 21 +-
.../core/data/aggregator/AvgValueAggregator.java | 2 +-
.../DistinctCountHLLValueAggregator.java | 2 +-
.../aggregator/MinMaxRangeValueAggregator.java | 2 +-
.../aggregator/PercentileEstValueAggregator.java | 2 +-
.../PercentileTDigestValueAggregator.java | 2 +-
.../core/data/manager/BaseTableDataManager.java | 48 +-
.../core/data/manager/InstanceDataManager.java | 14 -
.../pinot/core/data/manager/TableDataManager.java | 15 -
.../apache/pinot/core/data/order/OrderByUtils.java | 58 +-
.../apache/pinot/core/data/readers/FileFormat.java | 2 +-
.../recordtransformer/DataTypeTransformer.java | 18 +-
.../core/data/recordtransformer/PinotDataType.java | 221 +-
.../indexsegment/mutable/MutableSegmentImpl.java | 58 +-
.../io/util/DictionaryDelegatingValueReader.java | 79 -
.../core/io/util/FixedByteValueReaderWriter.java | 11 +-
.../org/apache/pinot/core/io/util/ValueReader.java | 15 +-
.../io/util/VarLengthBytesValueReaderWriter.java | 53 +-
.../core/operator/CombineGroupByOperator.java | 2 +-
.../operator/filter/ExpressionFilterOperator.java | 21 +-
.../predicate/InPredicateEvaluatorFactory.java | 2 +-
.../predicate/NotInPredicateEvaluatorFactory.java | 3 +-
.../predicate/RangePredicateEvaluatorFactory.java | 47 +-
.../query/DictionaryBasedAggregationOperator.java | 2 +-
.../operator/query/EmptySelectionOperator.java | 26 +-
.../query/MetadataBasedAggregationOperator.java | 2 +-
.../core/operator/query/SelectionOnlyOperator.java | 126 +
.../core/operator/query/SelectionOperator.java | 282 -
.../operator/query/SelectionOrderByOperator.java | 179 +
.../transform/TransformBlockDataFetcher.java | 68 +-
.../core/operator/transform/TransformOperator.java | 21 +-
.../function/AdditionTransformFunction.java | 41 +-
.../transform/function/BaseTransformFunction.java | 752 +-
.../DateTimeConversionTransformFunction.java | 27 +-
.../function/DivisionTransformFunction.java | 69 +-
.../function/IdentifierTransformFunction.java | 34 +-
.../function/LiteralTransformFunction.java | 34 +-
.../function/MapValueTransformFunction.java | 110 +-
.../function/MultiplicationTransformFunction.java | 41 +-
.../function/SingleParamMathTransformFunction.java | 48 +-
.../function/SubtractionTransformFunction.java | 68 +-
.../function/TimeConversionTransformFunction.java | 9 +-
.../transform/function/TransformFunction.java | 35 +-
.../function/TransformFunctionFactory.java | 11 +-
.../function/ValueInTransformFunction.java | 36 +-
.../core/plan/AggregationGroupByPlanNode.java | 2 +-
.../pinot/core/plan/AggregationPlanNode.java | 2 +-
.../plan/DictionaryBasedAggregationPlanNode.java | 9 +-
.../plan/MetadataBasedAggregationPlanNode.java | 13 +-
.../apache/pinot/core/plan/SelectionPlanNode.java | 33 +-
.../apache/pinot/core/plan/TransformPlanNode.java | 83 +-
.../core/plan/maker/InstancePlanMakerImplV2.java | 6 +-
.../query/aggregation/AggregationExecutor.java | 3 +-
.../aggregation/DefaultAggregationExecutor.java | 61 +-
.../core/query/aggregation/DistinctTable.java | 235 +
.../aggregation/function/AggregationFunction.java | 75 +-
.../function/AggregationFunctionFactory.java | 16 +-
.../function/AggregationFunctionUtils.java | 56 +-
.../function/AggregationFunctionVisitorBase.java | 53 +-
.../function/AvgAggregationFunction.java | 45 +-
.../function/AvgMVAggregationFunction.java | 23 +-
.../function/CountAggregationFunction.java | 39 +-
.../function/CountMVAggregationFunction.java | 21 +-
.../function/DistinctAggregationFunction.java | 160 +
.../function/DistinctCountAggregationFunction.java | 48 +-
.../DistinctCountHLLAggregationFunction.java | 55 +-
.../DistinctCountHLLMVAggregationFunction.java | 21 +-
.../DistinctCountMVAggregationFunction.java | 21 +-
.../DistinctCountRawHLLAggregationFunction.java | 48 +-
.../DistinctCountRawHLLMVAggregationFunction.java | 30 +-
.../function/FastHLLAggregationFunction.java | 43 +-
.../function/MaxAggregationFunction.java | 39 +-
.../function/MaxMVAggregationFunction.java | 21 +-
.../function/MinAggregationFunction.java | 39 +-
.../function/MinMVAggregationFunction.java | 21 +-
.../function/MinMaxRangeAggregationFunction.java | 46 +-
.../function/MinMaxRangeMVAggregationFunction.java | 23 +-
.../function/PercentileAggregationFunction.java | 44 +-
.../function/PercentileEstAggregationFunction.java | 44 +-
.../PercentileEstMVAggregationFunction.java | 21 +-
.../function/PercentileMVAggregationFunction.java | 21 +-
.../PercentileTDigestAggregationFunction.java | 45 +-
.../PercentileTDigestMVAggregationFunction.java | 21 +-
.../function/SumAggregationFunction.java | 39 +-
.../function/SumMVAggregationFunction.java | 21 +-
.../groupby/DefaultGroupByExecutor.java | 2 +-
.../query/executor/ServerQueryExecutorV1Impl.java | 32 +-
.../core/query/pruner/AbstractSegmentPruner.java | 3 +-
.../core/query/reduce/BrokerReduceService.java | 323 +-
.../core/query/request/ServerQueryRequest.java | 2 +-
.../query/selection/SelectionOperatorService.java | 251 +-
.../query/selection/SelectionOperatorUtils.java | 368 +-
.../converter/stats/RealtimeColumnStatistics.java | 6 +-
.../impl/dictionary/BaseMutableDictionary.java | 72 +
.../dictionary/BaseOffHeapMutableDictionary.java | 86 +-
.../dictionary/BaseOnHeapMutableDictionary.java | 35 +-
.../dictionary/BytesOffHeapMutableDictionary.java | 180 +-
.../dictionary/BytesOnHeapMutableDictionary.java | 143 +-
.../dictionary/DoubleOffHeapMutableDictionary.java | 222 +-
.../dictionary/DoubleOnHeapMutableDictionary.java | 160 +-
.../dictionary/FloatOffHeapMutableDictionary.java | 223 +-
.../dictionary/FloatOnHeapMutableDictionary.java | 160 +-
.../dictionary/IntOffHeapMutableDictionary.java | 223 +-
.../dictionary/IntOnHeapMutableDictionary.java | 160 +-
.../dictionary/LongOffHeapMutableDictionary.java | 224 +-
.../dictionary/LongOnHeapMutableDictionary.java | 160 +-
.../impl/dictionary/MutableDictionary.java | 92 -
.../impl/dictionary/MutableDictionaryFactory.java | 2 +-
.../dictionary/StringOffHeapMutableDictionary.java | 176 +-
.../dictionary/StringOnHeapMutableDictionary.java | 119 +-
.../creator/impl/SegmentColumnarIndexCreator.java | 13 +-
.../creator/impl/SegmentDictionaryCreator.java | 3 +-
.../core/segment/creator/impl/V1Constants.java | 39 +-
.../stats/AbstractColumnStatisticsCollector.java | 23 -
.../stats/StringColumnPreIndexStatsCollector.java | 5 +-
.../index/column/PhysicalColumnIndexContainer.java | 8 +-
.../index/data/source/ColumnDataSource.java | 4 +-
.../loader/bloomfilter/BloomFilterHandler.java | 8 +-
.../defaultcolumn/BaseDefaultColumnHandler.java | 5 +
.../core/segment/index/readers/BaseDictionary.java | 75 +-
...aryReader.java => BaseImmutableDictionary.java} | 68 +-
.../segment/index/readers/BytesDictionary.java | 42 +-
.../core/segment/index/readers/Dictionary.java | 50 +-
.../segment/index/readers/DoubleDictionary.java | 18 +-
.../segment/index/readers/FloatDictionary.java | 18 +-
.../core/segment/index/readers/IntDictionary.java | 23 +-
.../core/segment/index/readers/LongDictionary.java | 18 +-
.../segment/index/readers/OnHeapDictionary.java | 5 +-
.../index/readers/OnHeapDoubleDictionary.java | 43 +-
.../index/readers/OnHeapFloatDictionary.java | 38 +-
.../segment/index/readers/OnHeapIntDictionary.java | 28 +-
.../index/readers/OnHeapLongDictionary.java | 43 +-
.../index/readers/OnHeapStringDictionary.java | 73 +-
.../segment/index/readers/StringDictionary.java | 84 +-
.../pinot/core/segment/memory/PinotByteBuffer.java | 6 +-
.../virtualcolumn/DocIdVirtualColumnProvider.java | 56 +-
.../SingleStringVirtualColumnProvider.java | 76 +-
.../executor/StarTreeAggregationExecutor.java | 7 +-
.../startree/executor/StarTreeGroupByExecutor.java | 2 +-
.../startree/plan/StarTreeTransformPlanNode.java | 3 +-
.../startree/v2/builder/BaseSingleTreeBuilder.java | 2 +-
.../startree/v2/store/StarTreeLoaderUtils.java | 2 +-
.../org/apache/pinot/core/util/ArrayCopyUtils.java | 15 +
.../pinot/core/util/FixedIntArrayOffHeapIdMap.java | 96 +-
.../server/realtime/ControllerLeaderLocator.java | 226 +-
.../data/manager/BaseTableDataManagerTest.java | 5 -
.../data/recordtransformer/PinotDataTypeTest.java | 150 +-
.../util/VarLengthBytesValueReaderWriterTest.java | 8 +-
.../function/ValueInTransformFunctionTest.java | 3 +-
...ngeOfflineDictionaryPredicateEvaluatorTest.java | 26 +-
.../function/AggregationFunctionFactoryTest.java | 107 +-
.../impl/dictionary/DictionaryPerfRunner.java | 140 -
.../impl/dictionary/MultiValueDictionaryTest.java | 71 +-
.../impl/dictionary/MutableDictionaryTest.java | 94 +-
.../SegmentGenerationWithBytesTypeTest.java | 24 +-
.../index/loader/SegmentPreProcessorTest.java | 12 +
...eaderTest.java => ImmutableDictionaryTest.java} | 199 +-
.../ImmutableDictionaryTypeConversionTest.java | 385 +
.../pinot/core/startree/v2/BaseStarTreeV2Test.java | 2 +-
.../apache/pinot/queries/DistinctQueriesTest.java | 719 +
...erSegmentAggregationSingleValueQueriesTest.java | 73 +
...InnerSegmentSelectionMultiValueQueriesTest.java | 48 +-
...nnerSegmentSelectionSingleValueQueriesTest.java | 60 +-
...terSegmentAggregationMultiValueQueriesTest.java | 32 +-
...erSegmentAggregationSingleValueQueriesTest.java | 68 +-
.../DefaultAggregationExecutorTest.java | 2 +-
.../AggregationGroupByTrimmingServiceTest.java | 7 +-
.../pinot/query/executor/QueryExecutorTest.java | 50 +-
.../selection/SelectionOperatorServiceTest.java | 209 +-
.../v1/creator/BitmapInvertedIndexTest.java | 9 +-
.../segments/v1/creator/DictionariesTest.java | 99 +-
.../realtime/ControllerLeaderLocatorTest.java | 174 +-
.../src/test/resources/data/newColumnsSchema3.json | 10 +
pinot-integration-tests/pom.xml | 10 +
.../tests/ClusterIntegrationTestUtils.java | 28 +-
.../tests/MapTypeClusterIntegrationTest.java | 20 +-
.../tests/OfflineClusterIntegrationTest.java | 29 +
.../ControllerLeaderLocatorIntegrationTest.java | 168 +
.../pinot/orc/data/readers/ORCRecordReader.java | 3 +
.../org/apache/pinot/perf/BenchmarkDictionary.java | 117 +-
.../perf/BenchmarkOffHeapDictionaryMemory.java | 10 +-
.../pinot/perf/BenchmarkOfflineIndexReader.java | 10 +-
.../pinot/perf/BenchmarkStringDictionary.java | 10 +-
.../pinot/perf/StringDictionaryPerfTest.java | 83 +-
.../starter/helix/HelixInstanceDataManager.java | 16 -
.../SegmentOnlineOfflineStateModelFactory.java | 5 -
pinot-tools/pom.xml | 5 +
.../tools/admin/command/CreateSegmentCommand.java | 28 +-
.../resources/sample_data/airlineStats_data.json | 19492 +++++++++++++++++++
.../resources/sample_data/airlineStats_data.orc | Bin 0 -> 584802 bytes
.../app/mirage/factories/alert.js | 6 +-
.../app/pods/aiavailability/route.js | 30 +-
.../app/pods/application/route.js | 1 +
.../app/pods/components/alert-details/component.js | 2 +-
.../thirdeye-frontend/app/pods/home/index/route.js | 13 +
thirdeye/thirdeye-frontend/app/pods/login/route.js | 25 +-
.../app/pods/manage/alerts/index/route.js | 22 +-
.../app/pods/manage/alerts/route.js | 13 +-
.../app/pods/manage/explore/route.js | 39 +-
.../thirdeye-frontend/app/pods/manage/route.js | 13 +-
.../app/pods/manage/yaml/route.js | 26 +-
.../thirdeye-frontend/app/pods/rootcause/route.js | 13 +-
.../app/pods/services/query-cache/service.js | 8 +-
.../app/styles/pods/custom/dimensions-table.scss | 13 +-
thirdeye/thirdeye-frontend/app/utils/anomaly.js | 8 +
.../thirdeye-frontend/app/utils/api/anomaly.js | 2 +-
.../acceptance/self-serve-alert-tuning-test.js | 2 -
.../thirdeye/alert/commons/AnomalyFeedConfig.java | 6 +-
.../alert/content/EmailContentFormatter.java | 62 -
.../EmailContentFormatterConfiguration.java | 128 -
.../thirdeye/alert/fetcher/BaseAnomalyFetcher.java | 4 +-
.../apache/pinot/thirdeye/anomaly/AnomalyType.java | 18 +-
.../thirdeye/anomaly/alert/AlertJobContext.java | 1 +
.../thirdeye/anomaly/alert/AlertTaskInfo.java | 1 +
...nshotHelper.java => AlertScreenshotHelper.java} | 56 +-
.../anomaly/alert/util/AnomalyReportGenerator.java | 2 +-
.../thirdeye/anomaly/alert/util/EmailHelper.java | 29 +-
.../anomaly/alert/v2/AlertTaskRunnerV2.java | 40 +-
.../tasks/FunctionCreationOnboardingTask.java | 7 +-
.../onboard/tasks/NotificationOnboardingTask.java | 34 +-
.../AbstractModularizedAnomalyFunction.java | 2 +-
.../dashboard/resources/v2/DataResource.java | 13 +-
.../v2/rootcause/MetricEntityFormatter.java | 4 +-
.../thirdeye/datalayer/dto/AnomalyFunctionDTO.java | 7 +-
.../thirdeye/datalayer/dto/DatasetConfigDTO.java | 9 +
.../datalayer/entity/DatasetConfigIndex.java | 10 +
.../thirdeye/datalayer/pojo/DatasetConfigBean.java | 57 +-
.../datalayer/pojo/MergedAnomalyResultBean.java | 14 +
.../{StringUtils.java => ThirdEyeStringUtils.java} | 2 +-
.../pinot/thirdeye/detection/ConfigUtils.java | 29 +
.../pinot/thirdeye/detection/DataProvider.java | 5 +
.../thirdeye/detection/DefaultDataProvider.java | 39 +-
.../alert/DetectionAlertFilterNotification.java | 62 +
.../alert/DetectionAlertFilterResult.java | 16 +-
.../alert/StatefulDetectionAlertFilter.java | 40 +
.../filter/DimensionDetectionAlertFilter.java | 25 +-
.../alert/filter/PerUserDimensionAlertFilter.java | 19 +-
.../ToAllRecipientsDetectionAlertFilter.java | 36 +-
.../alert/scheme/DetectionEmailAlerter.java | 123 +-
.../thirdeye/detection/algorithm/MergeWrapper.java | 126 +-
.../thirdeye/detection/spi/model/AnomalySlice.java | 52 +-
.../validators/DetectionConfigValidator.java | 20 +-
.../detection/wrapper/AnomalyDetectorWrapper.java | 4 +-
.../wrapper/BaselineFillingMergeWrapper.java | 19 +-
.../wrapper/ChildKeepingMergeWrapper.java | 5 +-
.../wrapper/EntityAnomalyMergeWrapper.java | 77 +
.../thirdeye/detection/wrapper/GrouperWrapper.java | 2 +-
.../thirdeye/detection/yaml/YamlResource.java | 30 +-
.../yaml/translator/DetectionConfigTranslator.java | 23 +-
.../translator/DetectionMetricAttributeHolder.java | 13 +-
.../formatter/DetectionConfigFormatter.java | 109 +-
.../commons/EmailEntity.java | 2 +-
.../content/BaseNotificationContent.java} | 327 +-
.../notification/content/NotificationContent.java | 55 +
.../content/templates/EntityGroupKeyContent.java} | 41 +-
.../templates/HierarchicalAnomaliesContent.java} | 90 +-
.../content/templates/MetricAnomaliesContent.java} | 48 +-
.../templates/OnboardingNotificationContent.java} | 42 +-
.../formatter/ADContentFormatterContext.java} | 8 +-
.../formatter/AlertContentFormatterFactory.java} | 14 +-
.../formatter/channels/EmailContentFormatter.java | 125 +
.../src/main/resources/schema/create-schema.sql | 2 +
.../commons/TestEmailContentFormatterFactory.java | 39 -
.../alert/fetcher/TestContinuumAnomalyFetcher.java | 7 +-
.../transform/testAnomalyRemovalFunction.java | 2 +-
...UtilsTest.java => ThirdEyeStringUtilsTest.java} | 4 +-
.../pinot/thirdeye/detection/DataProviderTest.java | 2 +-
.../detection/alert/filter/AlertFilterUtils.java | 65 +
.../filter/DimensionDetectionAlertFilterTest.java | 33 +-
.../filter/PerUserDimensionAlertFilterTest.java | 38 +-
.../ToAllRecipientsDetectionAlertFilterTest.java | 43 +-
.../DetectionTimeWindowSuppressorTest.java | 14 +-
.../detection/algorithm/MergeWrapperTest.java | 58 +-
.../wrapper/EntityAnomalyMergeWrapperTest.java | 136 +
.../formatter/DetectionConfigFormatterTest.java | 14 +-
.../ContentFormatterUtils.java | 8 +-
.../templates/TestEntityGroupKeyContent.java} | 31 +-
.../TestHierarchicalAnomaliesContent.java} | 20 +-
.../templates/TestMetricAnomaliesContent.java} | 19 +-
.../TestOnboardingNotificationContent.java} | 19 +-
.../TestAlertContentFormatterFactory.java | 41 +
.../compositePipelineTranslatorTestResult-1.json | 93 +-
359 files changed, 32374 insertions(+), 6672 deletions(-)
create mode 100644 docs/schema.rst
create mode 100755 docs/star-tree/example.png
create mode 100755 docs/star-tree/space-time.png
create mode 100644 docs/star-tree/star-tree.rst
create mode 100755 docs/star-tree/structure.png
create mode 100644 docs/tuning_realtime_performance.rst
create mode 100644 kubernetes/README.md
create mode 100644 kubernetes/examples/gke/skaffold/README.md
create mode 100755 kubernetes/examples/gke/skaffold/cleanup.sh
create mode 100644 kubernetes/examples/gke/skaffold/gke-storageclass-kafka-pd.yml
create mode 100644 kubernetes/examples/gke/skaffold/gke-storageclass-pinot-controller-pd.yml
create mode 100644 kubernetes/examples/gke/skaffold/gke-storageclass-pinot-server-pd.yml
create mode 100644 kubernetes/examples/gke/skaffold/gke-storageclass-zk-pd.yml
create mode 100644 kubernetes/examples/gke/skaffold/kafka.yml
create mode 100644 kubernetes/examples/gke/skaffold/pinot-broker.yml
create mode 100644 kubernetes/examples/gke/skaffold/pinot-controller.yml
create mode 100644 kubernetes/examples/gke/skaffold/pinot-example-loader.yml
create mode 100644 kubernetes/examples/gke/skaffold/pinot-server.yml
create mode 100755 kubernetes/examples/gke/skaffold/query-pinot-data.sh
create mode 100755 kubernetes/examples/gke/skaffold/setup.sh
create mode 100644 kubernetes/examples/gke/skaffold/skaffold.yaml
create mode 100644 kubernetes/examples/gke/skaffold/zookeeper.yml
rename {pinot-core/src/test/java/org/apache/pinot/core/query/aggregation => pinot-common/src/test/java/org/apache/pinot/common}/function/AggregationFunctionTypeTest.java (97%)
create mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotLeadControllerRestletResource.java
create mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/api/upload/ZKOperatorTest.java
delete mode 100644 pinot-core/src/main/java/org/apache/pinot/core/io/util/DictionaryDelegatingValueReader.java
create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/operator/query/SelectionOnlyOperator.java
delete mode 100644 pinot-core/src/main/java/org/apache/pinot/core/operator/query/SelectionOperator.java
create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/operator/query/SelectionOrderByOperator.java
create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/DistinctTable.java
create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/DistinctAggregationFunction.java
create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/realtime/impl/dictionary/BaseMutableDictionary.java
delete mode 100644 pinot-core/src/main/java/org/apache/pinot/core/realtime/impl/dictionary/MutableDictionary.java
rename pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/{ImmutableDictionaryReader.java => BaseImmutableDictionary.java} (78%)
delete mode 100644 pinot-core/src/test/java/org/apache/pinot/core/realtime/impl/dictionary/DictionaryPerfRunner.java
rename pinot-core/src/test/java/org/apache/pinot/core/segment/index/readers/{ImmutableDictionaryReaderTest.java => ImmutableDictionaryTest.java} (59%)
create mode 100644 pinot-core/src/test/java/org/apache/pinot/core/segment/index/readers/ImmutableDictionaryTypeConversionTest.java
create mode 100644 pinot-core/src/test/java/org/apache/pinot/queries/DistinctQueriesTest.java
create mode 100644 pinot-integration-tests/src/test/java/org/apache/pinot/server/realtime/ControllerLeaderLocatorIntegrationTest.java
create mode 100644 pinot-tools/src/main/resources/sample_data/airlineStats_data.json
create mode 100644 pinot-tools/src/main/resources/sample_data/airlineStats_data.orc
delete mode 100644 thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/alert/content/EmailContentFormatter.java
delete mode 100644 thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/alert/content/EmailContentFormatterConfiguration.java
copy pinot-core/src/main/java/org/apache/pinot/core/data/readers/FileFormat.java => thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/anomaly/AnomalyType.java (71%)
rename thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/anomaly/alert/util/{EmailScreenshotHelper.java => AlertScreenshotHelper.java} (56%)
rename thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/datalayer/util/{StringUtils.java => ThirdEyeStringUtils.java} (98%)
create mode 100644 thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/alert/DetectionAlertFilterNotification.java
create mode 100644 thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/wrapper/EntityAnomalyMergeWrapper.java
rename thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/{alert => notification}/commons/EmailEntity.java (97%)
rename thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/{alert/content/BaseEmailContentFormatter.java => notification/content/BaseNotificationContent.java} (70%)
create mode 100644 thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/notification/content/NotificationContent.java
rename thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/{alert/content/EntityGroupKeyContentFormatter.java => notification/content/templates/EntityGroupKeyContent.java} (85%)
rename thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/{alert/content/HierarchicalAnomaliesEmailContentFormatter.java => notification/content/templates/HierarchicalAnomaliesContent.java} (67%)
rename thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/{alert/content/MetricAnomaliesEmailContentFormatter.java => notification/content/templates/MetricAnomaliesContent.java} (81%)
rename thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/{alert/content/OnboardingNotificationEmailContentFormatter.java => notification/content/templates/OnboardingNotificationContent.java} (75%)
rename thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/{alert/content/EmailContentFormatterContext.java => notification/formatter/ADContentFormatterContext.java} (88%)
rename thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/{alert/commons/EmailContentFormatterFactory.java => notification/formatter/AlertContentFormatterFactory.java} (65%)
create mode 100644 thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/notification/formatter/channels/EmailContentFormatter.java
delete mode 100644 thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/alert/commons/TestEmailContentFormatterFactory.java
rename thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/datalayer/util/{StringUtilsTest.java => ThirdEyeStringUtilsTest.java} (75%)
create mode 100644 thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/detection/alert/filter/AlertFilterUtils.java
create mode 100644 thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/detection/wrapper/EntityAnomalyMergeWrapperTest.java
rename thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/{alert/content => notification}/ContentFormatterUtils.java (83%)
rename thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/{alert/content/TestEntityGroupKeyEmailContentFormatter.java => notification/content/templates/TestEntityGroupKeyContent.java} (91%)
rename thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/{alert/content/TestHierarchicalAnomaliesEmailContentFormatter.java => notification/content/templates/TestHierarchicalAnomaliesContent.java} (89%)
rename thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/{alert/content/TestMetricAnomaliesEmailContentFormatter.java => notification/content/templates/TestMetricAnomaliesContent.java} (88%)
rename thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/{alert/content/TestOnboardingNotificationContentFormatter.java => notification/content/templates/TestOnboardingNotificationContent.java} (89%)
create mode 100644 thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/notification/formatter/TestAlertContentFormatterFactory.java
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org