You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by jl...@apache.org on 2019/07/16 17:40:52 UTC
[incubator-pinot] branch add-logic-for-lead-controller-resource
updated (f7132cd -> 2858350)
This is an automated email from the ASF dual-hosted git repository.
jlli pushed a change to branch add-logic-for-lead-controller-resource
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard f7132cd Address PR comments
discard 43c7e73 Add logic for lead controller resource on controller side
discard b9185d6 Add logic for leveraging lead controller resource
add 3be84f2 Include timestamp in logs (#4379)
add 5e2a810 upgrade fastutil to 8.2.3 (#4325)
add 33756ef [TE] Support Anomaly Hierarchy for Composite alerts (#4371)
add 29bad86 [TE] Inject entity name into grouper to enrich the grouped anomalies (#4378)
add dc63643 Adding support for udf/expression in filter predicates (#4365)
add e28eeb3 [TE] Log the registry components (#4384)
add 33f583b Offheap String Dictionary Benchmark (#4381)
add 732a7b9 Add more key/value pairs into LOG2M_TO_SIZE_IN_BYTES in HllSizeUtils (#4398)
add 49d8fa7 #4317 Feature/variable length bytes offline dictionary for indexing bytes and string dicts. (#4321)
add 02f6181 CompletionConfig for realtime tables (#4367)
add ae1ecef [TE] add a thread pool to run preview tasks (#4405)
add de62d8e [TE] Make email template and subject pluggable under email alert scheme (#4409)
add 7df1a43 [TE] Add Merger after metric grouper; other minor clean up (#4410)
add e71ac1b [TE] Do not authorize when auth is disabled (#4414)
add 0b8af63 Fix codecov link from old url to new (#4412)
add 07e2c72 Move tests for API resources to a different package (#4415)
add 52f69d1 [TE] Added support for Vertica as a data source (#4404)
add a7c419a Adding Calcite SQL Parser and make another entry point to query Pinot (#4387)
add e041ec5 [TE] frontend - harleyjj/detection-health - UI for model performance (#4413)
add 6bec451 [TE] Inject the port from config into Commons mail SSL Smtp port (#4420)
add dac9ae2 prompt on fail schema (#4395)
add b6fe4e6 [TE] detection health ui adjustments (#4421)
add bca6756 make kafka version number controlled by config number (#4396)
add 0993974 [TE] Fix exception handling - Propagate and display the error message/exception on frontind (#4419)
add aadcd36 In ClusterChangeMediator, stop enqueue/process changes if already stopped (#4422)
add 52f9b08 Add pinot community inviter (#4424)
add 2f691fc [TE] detection health - coverage fix (#4428)
add 079b86d Config for overriding initial rows threshold in segment size auto tuning (#4376)
add 50aaaf9 [TE] added ThirdEye configuration documents (#4429)
add f671072 Add URIUtils class to handle URI/URL encoding/decoding (#4426)
add 626c43a Misc fix for controller tests (#4431)
add 12e77bd #4401 Reuse a ThreadLocal byte[] when reading String elements from the variable length value reader. (#4432)
add e720448 Remove the massive console print and log from CalciteSqlCompilerTest (#4434)
add 67afc71 Cleanup un-necessary setups in HelixSetupUtils (#4430)
add b1d28fd Fix potential resource leak in the way we close a collection of closeables (#4418)
add c0cc01b Add logic for leveraging lead controller resource
add 178170b Add logic for lead controller resource on controller side
add 2858350 Address PR comments
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (f7132cd)
\
N -- N -- N refs/heads/add-logic-for-lead-controller-resource (2858350)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
README.md | 4 +-
.../java/org/apache/pinot/client/Connection.java | 44 ++-
.../client/JsonAsyncHttpPinotClientTransport.java | 30 +-
.../apache/pinot/client/PinotClientTransport.java | 6 +
.../{PinotClientTransport.java => Request.java} | 34 +-
.../apache/pinot/client/PreparedStatementTest.java | 40 +-
.../apache/pinot/client/ResultSetGroupTest.java | 38 +-
pinot-api/src/test/resources/log4j2.xml | 2 +-
.../broker/broker/helix/ClusterChangeMediator.java | 12 +-
.../requesthandler/BaseBrokerRequestHandler.java | 28 +-
...enNestedPredicatesFilterQueryTreeOptimizer.java | 3 +-
.../requesthandler/PinotQueryParserFactory.java | 49 +++
.../broker/requesthandler/PinotQueryRequest.java | 31 +-
.../request/PqlAndCalciteSqlCompatibilityTest.java | 123 ++++++
pinot-broker/src/test/resources/log4j2.xml | 2 +-
pinot-common/pom.xml | 4 +
.../pinot/common/config/CompletionConfig.java | 63 +++
.../apache/pinot/common/config/IndexingConfig.java | 20 +-
.../SegmentsValidationAndRetentionConfig.java | 30 +-
.../apache/pinot/common/utils/CommonConstants.java | 12 +-
.../common/utils/FileUploadDownloadClient.java | 16 +-
.../org/apache/pinot/common/utils/FileUtils.java | 59 ++-
.../org/apache/pinot/common/utils/StringUtil.java | 4 +-
.../org/apache/pinot/common/utils/URIUtils.java | 88 +++++
.../common/utils/request/FilterQueryTree.java | 11 +
.../pinot/common/utils/request/RequestUtils.java | 59 ++-
.../org/apache/pinot/filesystem/LocalPinotFS.java | 91 ++---
.../pinot/{pql => }/parsers/AbstractCompiler.java | 7 +-
.../utils/BrokerRequestComparisonUtils.java | 235 ++++++++++++
.../parsers/PinotQuery2BrokerRequestConverter.java | 5 +-
.../org/apache/pinot/pql/parsers/Pql2Compiler.java | 63 +--
.../parsers/pql2/ast/BetweenPredicateAstNode.java | 2 +-
.../pql2/ast/ComparisonPredicateAstNode.java | 22 +-
.../pql/parsers/pql2/ast/InPredicateAstNode.java | 2 +-
.../pinot/pql/parsers/pql2/ast/OrderByAstNode.java | 2 +-
.../pql/parsers/pql2/ast/OutputColumnAstNode.java | 2 +-
.../pql/parsers/pql2/ast/PredicateListAstNode.java | 2 +-
.../pql2/ast/RegexpLikePredicateAstNode.java | 2 +-
.../parsers/CalciteSqlCompiler.java} | 18 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java | 273 +++++++++++++
.../parsers/SqlCompilationException.java} | 19 +-
.../apache/pinot/startree/hll/HllSizeUtils.java | 14 +-
.../pinot/common/config/IndexingConfigTest.java | 3 +
.../pinot/common/config/TableConfigTest.java | 28 ++
.../apache/pinot/common/utils/URIUtilsTest.java | 68 ++++
.../apache/pinot/pql/parsers/Pql2CompilerTest.java | 2 +-
.../request/BrokerRequestSerializationTest.java | 10 +-
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 425 +++++++++++++++++++++
pinot-common/src/test/resources/log4j2.xml | 2 +-
pinot-common/src/test/resources/pql_queries.list | 5 +
.../{pql_queries.list => sql_queries.list} | 247 ++++++------
pinot-connectors/pinot-connector-kafka-0.9/pom.xml | 6 +-
.../apache/pinot/controller/ControllerConf.java | 46 +--
.../apache/pinot/controller/ControllerStarter.java | 16 +-
.../api/resources/FileUploadPathProvider.java | 5 +-
.../resources/LLCSegmentCompletionHandlers.java | 25 +-
.../api/resources/PinotControllerHealthCheck.java | 4 +-
.../api/resources/PinotSegmentRestletResource.java | 20 +-
.../PinotSegmentUploadRestletResource.java | 60 ++-
.../resources/PinotTableConfigRestletResource.java | 2 +
.../api/resources/PinotTableRestletResource.java | 9 +-
.../pinot/controller/api/resources/TableViews.java | 4 +-
.../helix/ControllerRequestURLBuilder.java | 21 +-
.../helix/core/PinotHelixResourceManager.java | 36 +-
.../helix/core/PinotTableIdealStateBuilder.java | 24 --
.../helix/core/SegmentDeletionManager.java | 34 +-
.../realtime/PinotLLCRealtimeSegmentManager.java | 34 +-
.../segment/FlushThresholdUpdateManager.java | 11 +-
.../SegmentSizeBasedFlushThresholdUpdater.java | 14 +-
.../helix/core/util/HelixSetupUtils.java | 240 +++++-------
.../controller/validation/StorageQuotaChecker.java | 32 +-
.../src/main/resources/static/js/init.js | 2 +
.../api/{resources => }/AccessControlTest.java | 10 +-
.../FileUploadPathProviderTest.java | 3 +-
.../api/{resources => }/PinotFileUploadTest.java | 2 +-
.../PinotInstanceRestletResourceTest.java | 2 +-
.../PinotSchemaRestletResourceTest.java | 2 +-
.../PinotSegmentRestletResourceTest.java | 2 +-
.../PinotTableRestletResourceTest.java | 2 +-
.../PinotTenantRestletResourceTest.java | 2 +-
.../SegmentCompletionProtocolDeserTest.java | 2 +-
.../SegmentCompletionUtilsTest.java | 2 +-
.../{resources => }/ServerTableSizeReaderTest.java | 3 +-
.../api/{resources => }/TableSizeReaderTest.java | 2 +-
.../api/{resources => }/TableViewsTest.java | 3 +-
.../helix/ControllerPeriodicTaskStarterTest.java | 38 +-
.../pinot/controller/helix/ControllerTest.java | 54 +--
.../controller/helix/PinotControllerModeTest.java | 288 +++++++-------
.../helix/core/PinotHelixResourceManagerTest.java | 106 ++---
.../helix/core/realtime/SegmentCompletionTest.java | 6 +-
.../segment/FlushThresholdUpdaterTest.java | 28 +-
pinot-controller/src/test/resources/log4j2.xml | 2 +-
.../realtime/HLRealtimeSegmentDataManager.java | 5 +
.../realtime/LLRealtimeSegmentDataManager.java | 91 +++--
.../core/data/readers/sort/PinotSegmentSorter.java | 75 ++--
.../generator/SegmentGeneratorConfig.java | 13 +
.../indexsegment/mutable/MutableSegmentImpl.java | 13 +-
.../impl/FixedByteSingleValueMultiColReader.java | 16 +-
.../core/io/util/FixedBitIntReaderWriter.java | 16 +-
.../core/io/util/FixedByteValueReaderWriter.java | 16 +-
.../org/apache/pinot/core/io/util/ValueReader.java | 3 +-
.../io/util/VarLengthBytesValueReaderWriter.java | 278 ++++++++++++++
.../impl/FixedByteSingleValueMultiColWriter.java | 23 +-
.../pinot/core/operator/blocks/TransformBlock.java | 4 +
.../RangelessBitmapDocIdIterator.java | 2 +-
.../dociditerators/SVScanDocIdIterator.java | 8 +-
.../operator/filter/ExpressionFilterOperator.java | 381 ++++++++++++++++++
.../core/operator/filter/FilterOperatorUtils.java | 3 +
.../predicate/PredicateEvaluatorProvider.java | 7 +-
.../org/apache/pinot/core/plan/FilterPlanNode.java | 24 +-
.../aggregation/groupby/utils/DoubleToIdMap.java | 2 +-
.../aggregation/groupby/utils/FloatToIdMap.java | 2 +-
.../aggregation/groupby/utils/IntToIdMap.java | 2 +-
.../aggregation/groupby/utils/LongToIdMap.java | 2 +-
.../query/pruner/ColumnValueSegmentPruner.java | 5 +
.../core/query/pruner/PartitionSegmentPruner.java | 5 +
.../converter/RealtimeSegmentConverter.java | 11 +-
.../core/realtime/impl/RealtimeSegmentConfig.java | 20 +-
.../pinot/core/realtime/stream/StreamConfig.java | 55 ++-
.../realtime/stream/StreamConfigProperties.java | 8 +-
.../segment/creator/ColumnIndexCreationInfo.java | 10 +-
.../creator/impl/SegmentColumnarIndexCreator.java | 16 +-
.../creator/impl/SegmentDictionaryCreator.java | 72 ++--
.../impl/SegmentIndexCreationDriverImpl.java | 6 +-
.../fwd/SingleValueSortedForwardIndexCreator.java | 10 +-
.../inv/OffHeapBitmapInvertedIndexCreator.java | 26 +-
.../segment/index/loader/IndexLoadingConfig.java | 11 +
.../defaultcolumn/BaseDefaultColumnHandler.java | 8 +-
.../index/readers/ImmutableDictionaryReader.java | 18 +-
.../segment/store/SegmentLocalFSDirectory.java | 2 +-
.../pinot/core/startree/StarTreeDataTable.java | 35 +-
.../startree/operator/StarTreeFilterOperator.java | 5 +-
.../v2/builder/OffHeapSingleTreeBuilder.java | 53 +--
.../MutableSegmentImplAggregateMetricsTest.java | 1 +
.../mutable/MutableSegmentImplTest.java | 3 +-
.../mutable/MutableSegmentImplTestUtils.java | 4 +-
.../util/VarLengthBytesValueReaderWriterTest.java | 170 +++++++++
.../pinot/core/startree/hll/HllFieldSizeTest.java | 2 +-
.../apache/pinot/queries/FastHllQueriesTest.java | 4 +-
.../util/IntDoubleIndexedPriorityQueueTest.java | 2 +-
.../test/java/org/apache/pinot/util/TestUtils.java | 32 +-
pinot-core/src/test/resources/log4j2.xml | 2 +-
pinot-integration-tests/pom.xml | 3 +-
.../tests/BaseClusterIntegrationTest.java | 18 +-
.../tests/BaseClusterIntegrationTestSet.java | 85 +++++
.../tests/ClusterIntegrationTestUtils.java | 41 +-
.../pinot/integration/tests/ClusterTest.java | 11 +-
.../DeleteAPIHybridClusterIntegrationTest.java | 24 +-
...mentBuildPushOfflineClusterIntegrationTest.java | 7 +
.../tests/HybridClusterIntegrationTest.java | 7 +
...ridClusterIntegrationTestCommandLineRunner.java | 4 +-
.../MultiNodesOfflineClusterIntegrationTest.java | 7 +
.../tests/OfflineClusterIntegrationTest.java | 70 +++-
.../tests/PinotURIUploadIntegrationTest.java | 8 +-
.../tests/RealtimeClusterIntegrationTest.java | 13 +
.../tests/SegmentCompletionIntegrationTests.java | 7 +
.../src/test/resources/log4j2.xml | 2 +-
pinot-minion/src/test/resources/log4j2.xml | 2 +-
pinot-perf/pom.xml | 7 +-
.../pinot/perf/BenchmarkDictionaryCreation.java | 12 +
.../pinot/perf/BenchmarkStringDictionary.java | 118 +++---
.../pinot/perf/StringDictionaryPerfTest.java | 86 ++++-
.../api/{resources => }/BaseResourceTest.java | 2 +-
.../api/{resources => }/TableSizeResourceTest.java | 2 +-
.../api/{resources => }/TablesResourceTest.java | 2 +-
pinot-tools/pom.xml | 3 +-
.../tools/query/comparison/ClusterStarter.java | 4 +-
.../realtime/provisioning/MemoryEstimator.java | 18 +-
pom.xml | 93 ++++-
thirdeye/README.md | 3 +
thirdeye/docs/configuration.rst | 210 ++++++++++
thirdeye/docs/index.rst | 2 +-
thirdeye/docs/production.rst | 27 --
thirdeye/docs/quick_start.rst | 8 +-
.../app/pods/components/alert-details/component.js | 2 +
.../app/pods/components/alert-details/template.hbs | 80 +++-
.../pods/components/detection-health/component.js | 197 ++++++++++
.../pods/components/detection-health/template.hbs | 84 ++++
.../app/pods/components/stats-cards/component.js | 42 +-
.../app/pods/components/stats-cards/template.hbs | 61 ++-
.../app/pods/components/yaml-editor/template.hbs | 2 +-
.../app/pods/manage/explore/route.js | 35 +-
.../app/pods/manage/explore/template.hbs | 1 +
.../app/pods/manage/yaml/route.js | 44 ++-
.../app/pods/manage/yaml/template.hbs | 3 +-
.../app/pods/self-serve/create-alert/template.hbs | 2 +-
.../app/styles/shared/_styles.scss | 118 ++++++
thirdeye/thirdeye-pinot/config/h2db.mv.db | Bin 2461696 -> 4947968 bytes
.../anomaly/merge/AnomalyTimeBasedSummarizer.java | 2 +
.../dashboard/DetectionPreviewConfiguration.java | 50 +++
.../dashboard/ThirdEyeDashboardApplication.java | 2 +-
.../dashboard/ThirdEyeDashboardConfiguration.java | 9 +
.../bao/jdbc/MergedAnomalyResultManagerImpl.java | 104 +++--
.../datasource/sql/SqlResponseCacheLoader.java | 37 ++
.../pinot/thirdeye/datasource/sql/SqlUtils.java | 3 +
.../thirdeye/detection/DetectionPipeline.java | 35 +-
.../detection/DetectionPipelineLoader.java | 10 +-
.../detection/alert/DetectionAlertTaskFactory.java | 13 +-
.../alert/scheme/DetectionEmailAlerter.java | 62 ++-
.../annotation/registry/DetectionRegistry.java | 5 +
.../components/TriggerConditionGrouper.java | 6 +-
.../thirdeye/detection/health/DetectionHealth.java | 4 +-
.../detection/health/DetectionTaskStatus.java | 14 +
.../validators/DetectionConfigValidator.java | 23 +-
.../thirdeye/detection/wrapper/GrouperWrapper.java | 6 +
.../thirdeye/detection/yaml/YamlResource.java | 63 ++-
.../yaml/translator/DetectionConfigTranslator.java | 32 +-
.../translator/SubscriptionConfigTranslator.java | 9 +-
.../bao/TestGroupedAnomalyResultsManager.java | 6 +-
.../bao/TestMergedAnomalyResultManager.java | 46 ++-
.../pinot/thirdeye/detection/DataProviderTest.java | 2 +
.../thirdeye/detection/alert/SendAlertTest.java | 2 +-
.../components/TriggerConditionGrouperTest.java | 2 +-
.../thirdeye/detection/yaml/YamlResourceTest.java | 6 +-
.../detection/yaml/alertconfig/alert-config-5.yaml | 3 +
.../compositePipelineTranslatorTestResult-1.json | 91 +++--
.../compositePipelineTranslatorTestResult-4.json | 28 +-
.../compositePipelineTranslatorTestResult-5.json | 110 +++---
218 files changed, 5651 insertions(+), 1858 deletions(-)
copy pinot-api/src/main/java/org/apache/pinot/client/{PinotClientTransport.java => Request.java} (61%)
create mode 100644 pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/PinotQueryParserFactory.java
copy pinot-common/src/main/java/org/apache/pinot/pql/parsers/AbstractCompiler.java => pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/PinotQueryRequest.java (60%)
create mode 100644 pinot-broker/src/test/java/org/apache/pinot/broker/request/PqlAndCalciteSqlCompatibilityTest.java
create mode 100644 pinot-common/src/main/java/org/apache/pinot/common/config/CompletionConfig.java
create mode 100644 pinot-common/src/main/java/org/apache/pinot/common/utils/URIUtils.java
copy pinot-common/src/main/java/org/apache/pinot/{pql => }/parsers/AbstractCompiler.java (81%)
create mode 100644 pinot-common/src/main/java/org/apache/pinot/parsers/utils/BrokerRequestComparisonUtils.java
copy pinot-common/src/main/java/org/apache/pinot/{pql/parsers/AbstractCompiler.java => sql/parsers/CalciteSqlCompiler.java} (61%)
create mode 100644 pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
rename pinot-common/src/main/java/org/apache/pinot/{pql/parsers/AbstractCompiler.java => sql/parsers/SqlCompilationException.java} (69%)
create mode 100644 pinot-common/src/test/java/org/apache/pinot/common/utils/URIUtilsTest.java
create mode 100644 pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
copy pinot-common/src/test/resources/{pql_queries.list => sql_queries.list} (87%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/AccessControlTest.java (87%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/FileUploadPathProviderTest.java (95%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/PinotFileUploadTest.java (98%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/PinotInstanceRestletResourceTest.java (99%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/PinotSchemaRestletResourceTest.java (98%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/PinotSegmentRestletResourceTest.java (98%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/PinotTableRestletResourceTest.java (99%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/PinotTenantRestletResourceTest.java (98%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/SegmentCompletionProtocolDeserTest.java (99%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/SegmentCompletionUtilsTest.java (96%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/ServerTableSizeReaderTest.java (98%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/TableSizeReaderTest.java (99%)
rename pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources => }/TableViewsTest.java (98%)
create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/io/util/VarLengthBytesValueReaderWriter.java
create mode 100644 pinot-core/src/main/java/org/apache/pinot/core/operator/filter/ExpressionFilterOperator.java
create mode 100644 pinot-core/src/test/java/org/apache/pinot/core/io/util/VarLengthBytesValueReaderWriterTest.java
rename pinot-server/src/test/java/org/apache/pinot/server/api/{resources => }/BaseResourceTest.java (99%)
rename pinot-server/src/test/java/org/apache/pinot/server/api/{resources => }/TableSizeResourceTest.java (98%)
rename pinot-server/src/test/java/org/apache/pinot/server/api/{resources => }/TablesResourceTest.java (99%)
create mode 100644 thirdeye/docs/configuration.rst
delete mode 100644 thirdeye/docs/production.rst
create mode 100644 thirdeye/thirdeye-frontend/app/pods/components/detection-health/component.js
create mode 100644 thirdeye/thirdeye-frontend/app/pods/components/detection-health/template.hbs
create mode 100644 thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/DetectionPreviewConfiguration.java
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org