You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by ki...@apache.org on 2019/01/21 09:28:06 UTC
[incubator-pinot] branch nested-object-indexing created (now
eacb83c)
This is an automated email from the ASF dual-hosted git repository.
kishoreg pushed a change to branch nested-object-indexing
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
at eacb83c Adding support to index nested fields within a column for e.g. map, json, Text etc. WIP
This branch includes the following new commits:
new 7c309bc Initial commit
new 936bf02 port over core
new 729e5d7 ported over couple more tests
new cb85d55 some cleanup; server structure
new 0a7bca7 added server resources
new 056776d added dimensions
new 21725e6 added config GET route
new f1572fe Refactor and added /collections
new e1b2169 wait for finish in load
new 865f7c6 fix number of threads for executor
new d3a931d do not use manager executor service in request path
new 6f1ffd8 folded application / config classes together
new 385a85c moved max record store entries / root uri to server config
new 5cdd409 added some stupid banner thing that is kind of cool
new 8e60491 Revert "moved max record store entries / root uri to server config"
new 416bd9b instantiate record stores via config
new 98c5ea2 removed sample server config (irrelevant now)
new 48733f5 thirdeye-core uses slf4j
new 64e0809 default value of 10000 for maxRecordStoreEntries
new 7a6ca6a Update README.md
new 089659a Update README.md
new 2188f29 Update README.md
new d261194 moved constants to own class
new 059e50c Merge branch 'master' of github.com:linkedin/thirdeye
new d6be10f Update README.md
new f9730ff Update README.md
new b1c81dc Update README.md
new 7744860 changed maxRecordStoreEntries default to 100k
new 4c69da9 Update README.md
new 85f7001 ignore local tmp dir
new 2346b0f Merge branch 'master' of github.com:linkedin/thirdeye
new 5a6b3ae added Avro file reader
new 4b9e511 can bootstrap using avro file
new ecf46ba Avro record stream has FileReader constructor arg
new c9584a3 added partial fixed buffer impl
new b27098a implemented dimension methods in fixed buffer
new 9907cb7 time buckets in fixed buffer
new 5df69c3 time bucket stored as an integer
new faefffd added test for roll over and fixed bugs
new 6984e90 FixedBuffer -> FixedCircularBuffer
new 5ce06ce circular buffer record store factory
new 970405c thirdeye-bootstrap project
new 818345f bootstrap tool constructs threshold function
new 4d358dd added step to compute all combos (this will not work)
new 40df03a changed bootstrap to just build tree
new 817d07e added TSV support
new f783b2e suffixes are .buf and .idx
new 217efcb (think) initial impl works
new bed2772 removed 2^(dim - fixed) combo thing
new 9d58306 always create catch-all other bucket
new 7df7951 fixed bug where avro stream did not set time
new 2da8309 restore method
new 15b6434 bootstrap -> load resource
new 2c61853 server /restore resource
new 2d15960 fixed bugs
new 63af9c4 TODO comment
new 8069ad2 scan if we do not find exact match for all records
new 352c3a7 added synchronization
new ae5ab40 TODO comment
new 16f9ed8 seekBucket function
new 268d011 find closest "other" bucket if no match
new 401dfb0 bootstrap fat jar
new 0b1517f use one re-sizing buffer instead of list of fixed buffers
new d5a841a fixed couple places not calling getBuffer()
new 7d725bf fixed couple bugs, added buffer dump tool
new 8ed6e23 think bootstrap / restore works now
new 5daa814 create a bucket for each time in range
new 91f3bd0 cleaned up index stuff in log buffer
new bdac272 if value not in index, use other
new 8220705 more informative log
new 554adaa partial impl of cleaned up circular buffer
new 72f7fbb more partial impl
new a7e0502 impl of dimension methods
new 0d7d7ae think impl complete, now test / bugfix
new 4054df5 added getEntrySize
new 3dbf481 fill buffer method
new e744456 testIterator
new 23a64f2 added new circular buffer factory
new aa6298d use factory to create record store
new 2b5eeb5 new circular buffer seems to work (at least can restore it)
new 7fa6643 tree can be restored, but think getMetricSums is way off
new f774b88 better buffer output format
new e1235c8 fix buffer position bug, but numbers are still a little off...
new fa8df67 better buffer dump format
new 4458829 fixed bug not scanning all time buckets
new 79d3229 added circular buffer clear (resets metric values to 0)
new a4af468 more tests
new a218ae8 bootstrap tool can clear or keep metric values
new c6b9456 validate time buckets in checkBuffer
new b9ffc7c refactored bootstrap packages
new 301239d buffer dump tool uses values from config
new 20712dc sorted out jackson deps
new abd3212 extension of circular buffer read/write from hdfs
new 68125e0 impl of hdfs factory, needs test
new 231ab1a class for hadoop job
new 54230ba added some logging in star tree manager
new b27ea4f better banner
new b98c3e9 StarTreeRecordStore#encode
new 5c7bebe progress on MR job but still wrong
new 60ebe09 StarTree#search -> gets node, StarTree#query -> gets metrics
new 8a9664d revised bootstrap job
new 1201459 bootstrap job generates stuff, but leaving out some leaves
new 0a25822 small change to sample job config
new ea5ee5f having trouble w/ memory limits
new 5043035 more memory efficient reduce
new a0e197c fix bug in find all
new bab3e58 can restore, but metrics missing
new 8f38df3 generate all combos in tree
new 7f34604 traverse specific/star correctly
new 13e5418 fixed bug (star / other value were non default)
new e21a1a8 StarTreeConfig has collection
new 86f0868 simplified bootstrap tool cli
new f24f5d8 threshold function gets whole sample
new 645a012 changed server restore to task
new d44713b fixed abs impl
new 4dad02b /load task
new d9e94d7 fixed config registration bug
new d1c0c0f log split values
new c556dd5 fixed rollup inspection method
new 6f9b537 added merge tool
new 156ec45 do not try to load mmaped buffer immediately
new 5a64945 use stream instead of aux buffer in load job
new cfae07a re-write rootDir w/ server config value
new 1eef297 used buffered output stream
new 9f482fc partial avro bootstrap job
new 923c8b1 use hadoop 2.2
new eb53519 fixed avro issue
new fe7c851 fixed bug w/ AvroKey vs. AvroValue
new 5a50352 print time information in merge job
new 595f0da set size in bytebufferimpl
new 57fab67 renamed tool
new 30bc82d renamed hadoop jobs
new b76a129 add other record in reducer
new bb3c6d7 added log for number of records in each leaf
new e79ea92 reverted other record generation to what it was
new 683a07b in circular buffer getDimensionValues, filter out special
new abb9c20 removed some unused classes
new 20655d5 changed metric from Long to Integer
new 54db7ec cleaned up /metrics api
new 6c5d18d keep only avro bootstrap job
new c9a3e95 removed unused method
new fe0c2c5 removed old readme
new 74e2c1c added new readme
new 11e6fb0 updated README
new 121b63d updated README
new 99fa99f Update README.md
new 84ee00a Update README.md
new f1a49bf Update README.md
new 6137614 renamed variable
new 4c903e4 Update README.md
new 1489878 Update README.md
new 437496f Merge branch 'master' of github.com:linkedin/thirdeye
new 29b3b95 Update README.md
new 16b3e59 Merge branch 'master' of github.com:linkedin/thirdeye
new 31d7b4a Update README.md
new ed4e412 Merge branch 'master' of github.com:linkedin/thirdeye
new 1936e95 POST to /metrics/{collection}
new da2de85 fixed issue with time buckets in buffer generation
new eb6ae64 added /configs resource
new 531f862 get metrics respects rollup for ! query
new b74b9f8 added getStats
new d401dea added /collections/{collections} route to readme
new 8181da5 refactored server stuff
new 4c36d17 add min/maxTime to stats
new d13685f updated readme
new a242623 dump stats in bootstrap tool
new 8df8b59 changed to LogBufferImpl
new f93a260 change factory name to LogBufferImpl as well
new 58d92d7 Update README.md
new 163a058 better log messages
new 2bf1aaf Merge branch 'master' of github.com:linkedin/thirdeye
new 010723d increased buffer size when building
new 972ccd3 forgot to flush printWriter
new 8bfdc8d log number of time buckets in reducer
new ed53117 more logging, flush printWriter in buffer dumper
new 50f50a1 log info in circular buffer
new 6a4e3b8 alias records to ? in build phase
new d200f8b cleaned up comments
new 91d7b58 dimension/metric names in /collections/{c} resource
new 14b0b8f change log level
new c2eb6ee default 50k record store entries
new bd85ae2 node path name includes value
new 4f598b7 updated README
new b762ce9 Update README.md
new 370353c fixed bug where dimension equality computed for every bucket
new ba02062 time bucket optimization
new 4f70b61 always roll up in /metrics queries
new 821a619 StarTree#query -> StarTree#getAggregate
new 82d2a20 added getTimeSeries
new a3cb0d1 added /timeSeries resource
new 8ed699f added time series thing
new 8b5bc2d /timeSeries per-metric basis
new 38ec3ea throw exceptions jersey can use
new 1b47658 cleaned up getDimensionValues code
new 9f71f86 honor ALL dimension value in shouldTraverse
new f510263 alias rolled-up dimensions into ? in response
new 15659d1 added a record analysis tool
new 271f18d moved thirdeye dashboard stuff over to this repo
new 2c721dc added lookBack to heat map
new 725f22f added lookBack to UI
new ace7ce3 shade lookBack regions in time series
new 7929355 changed shade color for ranges
new 10f9ea9 keep time series fixed at top too
new 5c56bef time series does not move (too buggy)
new 97c44e8 return list of time series
new 72a960a shadeColor is yellow
new fd3075d added link to go back to top of page
new fdaad1a generate time series before heat maps
new 92e0924 removed unused method
new 4fd3fc5 added ability to get cartesian product of time series
new cde83e7 metrics can do multimap cartesian products too
new f6df0cf shade color lighter shade of line color
new 7c5f601 support multivalue queries in Data
new 7146ff9 only one js file
new e8693aa removed commented out code (that code showed N/A cells)
new 5b22a22 added little squiggle arrow link
new e508612 render modal time series
new d5cfc95 order according to heat map order in multi-time-series
new 45d5686 mouseover for squiggle time series arrow
new 706208e added a picture for empty screen
new e080bd3 changed timeseries arrow
new 6d16d4b ratios are (current - baseline) / sum(baseline)
new 09e3938 show percent change w.r.t. whole instead of self
new 82d0e55 fixed modal rendering issues
new 02e2454 removed Options > Quick-start
new 0aac891 added cobertura
new 6ec7c2b Merge branch 'master' of github.com:linkedin/thirdeye
new 17e5907 moved avro-related bootstrap functions to separate class
new e67ce33 moved job property constants to another class
new bba5650 changed name of bootstrap mapper/reducer
new 765b65c Hadoop job to partition incremental batch into Avro files
new 58ebb67 better output for record analysis tool
new b64862c moved avro helper methods to core
new 78757ca log4j sl4fj impl in core
new 3aee998 Revert "log4j sl4fj impl in core"
new 230347b StarTreeBulkLoaderImpl to batch update record stores
new cff2b4d exposed bulk load via /tasks/bulkLoad
new 5d0137f added cleanup task
new 32f0c84 Remove duplicates from shaded JARs
new 53fb9d6 added help text
new 44436a2 added help images / help complete
new 9bccfb7 help is explicit button
new d1fe68e /tasks/bulkLoad?action=download can pull from external source
new 6c3157a look-back -> time window
new 53ed518 /tasks/bootstrap downloads star tree structure / buffers and installs in rootDir
new e1131e7 changed wording
new cd96652 added abookImport job config
new e2dc0ed changed job config to headless account
new 5c9c880 inherit version from parent pom.xml
new 618e646 Adding aggregation job
new d7ac549 Adding aggregation phase job
new d3275b6 Adding additional dependency
new 7a30f2d fixed compile bug
new 59851d5 Map reduce aggregation job to aggregate metrics over X days
new 33f6371 Fixing merge conflicts
new 6181828 Refactoring and added skeleton for Rollup job
new e217661 Adding phase 2 and phase 3 jobs for Rollup
new ce50d18 fixed comment copy/paste error
new 41a52fb generate html/xml cobertura reports
new 2af1b4b some tests for api package
new 5ee363f fixed some test dependency stuff
new d4374e5 added tests for StarTreeUtils#merge, #filterQueries, #expandQueries
new 538b0df test for avro conversion
new aa55c95 test for StarTreeBulkLoaderAvroImpl
new e4a63ac added test for restoring tree
new 818bea3 unit tests for query
new da14631 unit tests for record
new 9769c1b more unit tests for star tree config
new ce3aed5 added test for min other value
new a50fc44 ability to ignore time in bootstrap tool
new 329a246 StarTreeConfig should accept splitOrder parameter
new 86f1831 Minor changes to model objects
new fa3a12b merging conflicts
new 69f5fb5 Create StarTreeRecordStoreBlackHoleImpl
new 0bf2118 added build script
new 78ad70a added more memory efficient path for avro conversion
new 25e42ca Create thirdeye-realtime module with Kafka consumer
new 33d5ed1 run tests in build
new 8368f16 removed time series chart, does not belong in server
new fdd2de2 added distributed params to config
new 009bb9a connect to helix if distributed in lifecycle
new 776c2a9 register state machine engine
new 4b84d01 log state transitions
new 71b6fdd task to partition leaves
new ea15f22 StarTreeManager#open explicit
new 13b73fe StarTreeManager#create is idempotent
new 4c90e3e black hole record store factory
new 6374919 ability to stub record stores with black hole impl
new eea1db8 stub is NOP if tree exists
new 354cf64 state transition code opens subsets of record stores
new 49935eb external data source (http impl)
new 72af378 instantiate a HttpClient in application
new 10fd0c4 construct and pass external data source to transition handler
new 90eeed0 logic to pull external archive
new c29750f synchronize on external data source wholly (naive but seems to work)
new d129d0a fix locking
new 48e51f8 added reset() method
new 740e2e7 4 phases of rollup
new 369106a 4 phases of roll up
new 71389ec moved partition task to stand-alone tool
new d637444 removed diff in root dir
new e98497c Separate CLI tools from thirdeye-bootstrap into thirdeye-tools
new f873d2f add log4j.xml to thirdeye-tools for console output
new b9b48a6 added .reviewboardrc
new 92916e2 THIRDEYE-15 star tree generation using a single task. Perf improvements in aggregation and rollup jobs
new a86ae21 StarTreeBootstrapJob outputs SequenceFile of serialized indexes/buffers
new 62bd5bb Merge branch 'master' of gitli.corp.linkedin.com:thirdeye/thirdeye-mirror
new cf731c0 Create /starTree/{collection} resource to serve the raw star tree binary for a collection
new 830479d THIRDYE-74 Create Helix-enabled client that uses star-tree binary to route requests
new 967371f Renamed HelixImpl -> ClusteredClientImpl
new ac3526c THIRDEYE-82 StarTreeDumperTool should output some metadata about tree
new 6bace80 committing acls and .reviewboardrc file
new a49c433 committing README
new 4161e22 trimming the acls whitelist
new ec9206c adding .gitignore
new 9e6789a THIRDEYE-87 Support for all numeric data types
new 391f307 Segment Creation
new 4a11958 THIRDEYE-86 If "?" explicitly specified in dimension value, go to other node
new dd43a6b pinot 2.0 Query Planning, Processing and Execution Implementation
new e59c441 removed scratch test code that was failing
new eecf56c Segment Datasource and block and filtering implementation
new 4be4808 Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
new 22c2308 controller API Implementation
new bfd6e45 THIRDEYE-88 Fix issues in star tree generation phase
new fe30381 THIRDEYE-88 Fix issues in star tree generation phase
new 3a3c295 THIRDEYE-79 Consolidate API classes into thirdeye-api module
new 31dc4dd PINOT-1181 Interface for Block, DataSource, Operator for single and Multivalue
new 278705c pinot 2.0 Controller backend
new 8ba5aed utils, common, api, hadoop, integration tests, pom files and missing tests from core and controller
new 3ce4374 Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
new 917e722 Helix Broker
new 45280d6 Helix server starter
new 3830d2a adding groups in the .reviewboardrc
new b1db004 PINOT-1011 Core data readers and writers, all high level datasources and operators will be composed using these readers/writers
new 532a17c Scatter/Gather transport layer along with Helix based routing
new 88ed9fe missing files between github repo added and to this repo, also everything has been copied over beyond this point and this repo should be open for business
new cb31bdf Revert "missing files between github repo added and to this repo, also everything has been copied over beyond this point and this repo should be open for business"
new 0204ae1 modified .gitignore to exlcude test*.xml files and test-output, also changed .reviewboardrc to fix git review create bug, also added all the missing files
new bca53e7 now acls only has tester as a whitelist user
new a543286 dummy commit
new 82f94a3 adding acls
new b831f1b THIRDEYE-89 Support URI-based archive source
new 88710ec A function to compute least-other combination for bootstrap
new 4f8a028 removing a duplicate copy of DataSource interface and removing all references
new 0d8c7c3 adding branch in .reviewboard rc file
new 7f2be02 adding some rb realted helper in the README file
new d8d6706 THIRDEYE-92 star tree generation and bootstrap job on hadoop
new 7a97554 THIRDEYE-92 star tree generation and bootstrap job on hadoop
new 0a58d32 Adding broker-level metrics
new f1c2259 THIRDEYE-92 Startree generation and bootstrap phase, review part 1
new 7b37cb0 Updated metrics code to inject the BrokerMetrics dependency
new f688464 1. Adding segmentMetadata start/end time into segmentMetadata creation, segmentMetadata loader. 2. Adding RetentionManager to scan segments and delete them based on configs.
new 652d3c3 Adding SegmentDeletionManager to handle deletes segment from helix property store and local disk. Adding more test into TestRetentionManager
new 0013567 Refactor SegmentTimeUnit to TimeUnit.
new 7d966fe THIRDEYE-92 Bootstrap phase to create leaf buffer index files
new c2274eb THIRDEYE-92 Removing debug statements
new 434a097 THIRDEYE-92 Bug fixes to bootstrap and star tree generation phases
new 95e0679 THIRDEYE-92 Bug fixes to bootstrap and star tree generation phases
new 405eb20 THIRDEYE-97, THIRDEYE-96 Adding stats for aggregation job. First phase of supporting various datatypes in star tree
new 19c4311 THIRDEYE-97, THIRDEYE-96 Adding support for various datatypes in star tree
new 51d1dcc THIRDEYE-97, THIRDEYE-96 Adding support for various datatypes in star tree. Review comments
new 836989e THIRDEYE-97, THIRDEYE-96 Adding support for various datatypes in star tree. Adding NumberUtils RB=412290
new 0c57a11 THIRDEYE-96 Adding support for different data types in star tree
new 8a52c33 THIRDEYE-96 Adding support for different data types in star tree
new 6be048b THIRDEYE-96 Adding support for different data types in star tree
new 8277b64 THIRDEYE-98, THIRDEYE-58 Dashboard should display time series in granularity other than hours
new 6a8080a THIRDEYE-102,THIRDEYE-101 Add tool-tip that shows plot point on time series chart
new f8d0d84 THIRDEYE-98, THIRDEYE-58 Dashboard should display time series in granularity other than hours
new 2f0e632 THIRDEYE-105 Time series point selection does not change input form values
new 5088bcd THIRDEYE-103 Display multiple normalized metrics on time series
new 7a4dce0 Added hooks to listen for new MetricsRegistries being used, so that users can add custom actions to monitor those metrics in their production systems
new ea5c6ec THIRDEYE-107 Dimension selection should be cleared when collection changes
new b9e6a76 THIRDEYE-108 Remove threshold function from star tree
new 0c2061b THIRDEYE-109 Move thirdeye-bootstrap dimension/metric API classes to thirdeye-core
new cef3b70 Added test and correction for the silly Commons Configuration API
new 2df0798 Adding static exceptions and error code to Pinot 2.0
new 0a98c41 Removed version from thirdeye-bootstrap/pom.xml (inherits from parent)
new 5235230 THIRDEYE-110 StarTreeConfig should not instantiate StarTreeRecordStoreFactory
new fffb6c2 THIRDEYE-100 Clean up and unify server / bootstrap star tree configs
new b5e251f THIRDEYE-116 Should clear metric options when collection changes
new 1fb258b THIRDEYE-117 User should be able to aggregate time series at different granularities
new ee51a0b THIRDEYE-117 User should be able to aggregate time series at different granularities
new 755992e Several UI related changes...
new c45cc00 Adding segment validation
new 88f9d65 Removing useless local variable
new 224aa70 Adding custom segment name setting for segment creation.
new 260d240 Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
new ab0f819 Store the segment push time and refresh time in Helix properties
new 54993d2 Refactoring config classes
new ac18d67 Fixing the bug of missing time column during segment metadata persistance.
new 8bbb9ef Refactored repetitive code
new 57658eb Bugfix for issue introduced during refactoring
new c2b8a3e Visualization bugs
new 9af5868 Change addProperty to setProperty
new 899d000 Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
new 4a01385 Removed references to validation at file upload time
new 0b486f7 Swallow and log exceptions that happen during validation, as not to prevent future executions of the validation from happening.
new 0d1028f Ignore segments that have no time interval, avoid emitting metrics when there is no push time metadata.
new d04704e PINOT-1208 phase 1 of refactoring/code cleanup - removing dead files
new d159861 THIRDEYE-90 Create StarTreeRecordStore implementation which separates dimension/metric on-disk storage
new 90ea271 THIRDEYE-126 StarTreeRecordStoreFactoryFixedImpl should listen to file-system changes
new d66f432 Skip the findbugs report for now, as it times out on while running on Hudson
new cb9a407 Made avro version configurable, can be overridden through the commandline using -Davro.version=1.4.0
new dfc1783 removing unused tests and adding jfim to the owners list in the acls
new 5552d2b Several JIRAs...
new 6fbd5c3 Review feedback based on RB-408710 1. Change System.out.println to log.debug 2. Delete unused method 3. Move constants and default configs to CommonConstants
new b52c46d Feedback for RB-408636. 1. Remove unused methods. 2. Use AtomicReference for state
new 0530688 Changes made based on previous RB Comments, across segment creator, DataSource etc
new 4b5b153 Removing exception that was being thrown when we set a predicate on the DataSource
new ad22bc1 THIRDEYE-11 Output time-series should be able to display a function of metrics
new 00fcc3f Added meters for exceptions that occur during broker processing along with the appropriate logging
new e180c53 THIRDEYE-133 Create analyze hadoop workflow step
new 43975ac Updated readme
new 83101aa snapshot heat map
new c4fc04c (Multiple)
new 9444acd THIRDEYE-151 Move heatmap code from thirdeye-core to new thirdeye-visualization module
new e327e9d Bugfix for unit tests that were not updated to create a table for their segments, causing the PinotResourceManager to reject segments for tables that did not exist
new d493819 Bugfix for the ControllerSentinelTest, caused by serialization and deserialization of DataResource not being isomorphic (when metadata is null, it is deserialized as a NullNode rather than an ObjectNode, causing an exception when the constructor is invoked by Jackson, as NullNode isn't an ObjectNode
new 6a0f57f Made constants final
new ef0d2ec THIRDEYE-149 Polish thirdeye-dashboard
new 8e3b1b3 THIRDEYE-149 Polish thirdeye-dashboard
new 13ba7da THIRDEYE-149 Polish thirdeye-dashboard
new 966f1ba Minor code cleanups: Removed some System.out.println calls, fixed a variable name, removed a hardcoded class name, logged an error properly and rethrew the exception instead of NPE'ing on the next line
new 2ef037c Refactored checks for brokerResource
new a0661cc Fixed PINOT-1233, PINOT-1235
new cc47786 For some reason, the commons-compress dependency is broken in our artifactory because ligradle tries to pull commons-compress-1.4.1-sources.jar which doesn't exist there. The 1.8.1 version seems to contain this.
new 3c423b2 fixed commented tests in pinot-core
new 8cd4a70 Downgrade the commons-compress version back to 1.4.1, as the other one is also broken in Artifactory.
new 33a03c3 Review feedback for RB-407626.
new 5adda6c Move HelixInstanceDataManager to server module Move timeout to ServerQueryExecutorV1Imple Fix some tests. Adding Metrics constants to CommonConstants.
new a127009 THIRDEYE-149 Polish thirdeye-dashboard
new b1fecda THIRDEYE-149 Polish thirdeye-dashboard
new 5419267 THIRDEYE-149 Polish thirdeye-dashboard
new 5bf83e9 Fixed the pinot-integration-tests maven project so that it builds and is part of the main build
new f685883 Review feedback for RB-408183.
new 32e95a3 1. Make Server and Broker StateModel constants. 2. Remove dropServer and dropBroker functions. 3. drop instance before checking live instance.
new 7397c7b Bug Fix for PINOT-1223: Got exception when trying to render groupby results for empty broker response.
new 27994c2 Visualization-related JIRAs
new 2124804 Adding small simple data source to fix failed unit test. Remove IndexSegmentUtils.
new 0e82e71 THIRDEYE-158 Enable bootstrap flow to push to thirdeye-server
new 337a7ee first draft of realtime interfaces
new 08520ec second draft of realtime interfaces
new 8758962 third draft of realtime interfaces
new 7f18a56 removing seal method from the RealtimeIndexSegment
new fa4692d getRealtimeSegment in RealtimeSegmentDataManager will send back a MutableIndexSegment
new ac7488a Add hasNullValue into column metadata
new 05668a9 Fix failed tests.
new 754079e Fixed unit test with incorrect maven dependencies
new c971d9b THIRDEYE-168 Add ContributionDifferenceHeatMap implementation
new 83bec56 Fixing issue with empty/null time interval/duration causing failed to load segment.
new b904ec6 Adding destroy method to IndexSegment interface to handle destory a segment in heap and mmap mode when we delete a segment from resourceDataManager
new 7580bea fixing typo and empty space
new c292a5a Fixing refresh logic to destroy segment correctly.
new f6a457e minor changes to the UI of the snapshot dashboard
new 2d342cb Merge: f6a457e c971d9b
new 93be867 Adding version map to routing table builder to prevent unnecessary routing table build process.
new 9533eb9 Fixing segment refresh won't get notified to server.
new 68f5d78 Tasks / healthcheck
new 212b869 Fixing broken JSON unit tests, set up Maven projects to run properly with Sonar
new 134e31a Configure surefire to only use one fork, without reusing processes
new de4e72d Replaced some hardcoded string constants by actual constants
new de1c66e THIRDEYE-177 Split push logic out from startree_bootstrap_phase2
new 8cce8cb THIRDEYE-177 Split push logic out from startree_bootstrap_phase2
new acbdce5 THIRDEYE-178 Aggregation Unit and Moving Average UI options should be mutually exclusive
new 74fe700 THIRDEYE-180 Add ability to normalize all metrics to one metric (as opposed to self)
new a4dbc31 PINOT:1271 Support negative numbers in FixedBitEncoding
new 478d42e PINOT:1271 Support negative numbers in FixedBitEncoding
new 8ae48b0 PINOT:1271 Support negative numbers in FixedBitEncoding
new 21f722c Fixed the 4 of the 5 server integration tests, stopped exception swallowing in unit tests, added utility to extract files from JARs
new 9c5e2ab Fixed unit tests that were using local file resources to unpack their resource from the JAR if needed
new 5782dd2 THIRDEYE-180 Add ability to normalize all metrics to one metric (as opposed to self)
new 6f5cf67 Adding returnSegmentReaders method, so we only deleted segment when no segment read happens.
new 07a7299 Fixing segment refresh logic to make sure idealStates changes relected on externalView.
new ecdc734 Added missing hashCode method for DataResource, removed a field that was compared for equality twice
new 6036567 Fixed unit tests with columns that contained null values with the recent FixedBitWidth API changes
new f4eeb19 Moving classes that should not be in the main source tree to tests, removed two unused classes.
new 5db33da won't return segments for now.
new c2a9cd5 Adding log for HelixResourceManager refresh logic.
new d5e2081 Using modification time to replace data version when externalview updated.
new b75f254 Using modification time to replace data version when externalview updated.
new b343dc7 Make HelixInstanceDataManager to implement DataManager. Adding more log to SegmentOnlineOffline.
new 1b925ff Fixing InstanceDataManager with QueryEngine
new bb01ffb THIRDEYE-183 MetricStore and DimensionStore should use ByteBuffer#duplicate
new 07649e1 Renamed method that looks like a constructor to init
new 1846ce8 Simplified IndexType code, added unit test
new dfd1a60 Refactor some code of segment loading and query layer. Adding more logs.
new 9f0a658 THIRDEYE-18 Design framework to call anomaly detection function on meaningful combinations
new 8e4f6c4 Replaced condition that is always true by the intended one
new 9a3d38b Replaced equality check with NaN (which is always false) with Double.isNaN
new 6cc6f5f Added server metrics, refactored common code between broker and server metrics into AbstractMetrics
new 7292b75 refactor segment upload and refresh logic and adding more logs
new 8dcdac4 Fixing dynamically created ResourceDataManager doesn't start.
new 8df88f7 Remove local directory bootstrap process. Return segment reference after query processing
new 1293245 Kafka-related commits (and misc)
new 6a43682 THIRDEYE-188 Create high-level Kafka consumer implementation
new 74cf3ba adding an implementation of RealtimeSegment and changes that go along with it
new f5d9937 changes based on RB feedback
new 77b7665 Adding gbrandt,ajaspal to acl
new 9bc8265 adding a SchemaBuilde to make schema construction easy
new cfc60e6 Merge: 77b7665 6a43682
new 7289d33 THIRDEYE-190 Move logic in ThirdEyeJob#main to ThirdEyeJob#run
new 8a45748 - THIRDEYE-163 : New MultipleAggregateBasedRollupFunction that handles multiple metrics
new b1029fb - THIRDEYE-167 : Extend AnalysisPhaseJob to compute dimension value distribution
new 63150a4 THIRDEYE-194 Add back time series by dimension value view
new e618be5 Adding hasDictionary() method into SegmentMetadata. Default is always true.
new 0afcdd6 Adding aggregation functions for no dictionary columns
new 09897d3 renaming Builder to SchemaBuilder
new 9561fd6 Merge: 09897d3 0afcdd6
new 3a2f7b4 Fixed Javadoc, added comments, removed some duplicate code, used static imports to shorten some code
new 068a81c fixing all the tests
new 20772f3 fixing tests in pinot-server
new 9649979 Merge: 20772f3 3a2f7b4
new 1fcb7a5 Fixing selection with no dictionary.
new 31dcc17 Remove unused code. Fixing TestHelixBrokerStarter Remove resource version map in HelixExternalViewBasedRouting
new 5f39282 Fixing other tests: pinot-core TestSelectionQueries.testSelectionIteration TestSelectionQueriesForMultiValueColumn.testSelectionIteration
new 27ee213 Refactored ControllerSentinelTest to extract code that could be reused to ControllerTest
new 5a0b717 Made the file upload restlet return HTTP errors when an exception occurs, added unit test
new bfe3329 THIRDEYE-196 Refactor UI components, render server-side
new 571de78 THIRDEYE-196 Refactor UI components, render server-side
new afbef8a THIRDEYE-203 Publish shaded jar for thirdeye-bootstrap and accept thirdeye.phase config
new ffdae88 Adding TimeBoundaryService interface and its simple externalViewBased implementation.
new b748aac Adding test for TimeBoundaryService
new 6e27023 Adding another workflow to process federated request. After fully tested, we can merge the logic of single request to it.
new 0e8bf4f Fixing contains check for HelixExternalViewBasedTimeBoundaryService
new 39b8398 changes based on review feedback on RB 432628 .. realtime segment iml
new f3a4da8 adding realtime dimensions and metrics SerDe
new 5be3bfd using realtime dimensions and metrics SerDe in the RealtimeIndexSegmentImpl
new be59d43 started writing tests for comparing realtime and offline segment on the same data
new 25fa17e PINOT-1277 phase 1 of realtime kafka comsumption.Includes the skeleton flow
new 3e9bd4c Adding ResourceType Offline/Realtime suffix
new 8de47fa THIRDEYE-196 Refactor UI components, render server-side
new 1b0a7ed added a high level kafka consumer wrapped around the StreamProvider interface, also added a kafka decoder interface to decode incoming messages and added a schema based decoder implementatin
new cf4772b merge conflicts
new 8e79eb7 review feedback and also making KafkaAvroDecoder extract md5 schema String from incoming events and use that to fetch schema from schema registry
new 39acde7 Adding unit test for SearchableByteBufferUtil, added assertion in SearchableByteBufferUtil for Float.NaN and Double.NaN
new df08325 Added metric for the offline delay in hours instead of milliseconds
new 3a6fcf4 Move this to pinot-common, so that the code actually builds
new c4e91fe adding npawar to acls
new 387988a Merge: c4e91fe 3a6fcf4
new e5bd6ca changes to pom files to remove slf4j errors
new 794efda - Making sure that the incremental updates are run against the most recent tree.bin THIRDEYE-200
new b91d0dc removing duplicated implimentations of BlockDocIdSet across several block implementations into a Single BlockUtils class, also removed bitmap.toArray and using bitmap.iterator instead
new f256099 - Throwing a relevant error message when no dimension combination can be rolled up, currently we are getting NPE
new 8a4358f Made the kafka version overridable through the command line
new 998c33f Merge: e5bd6ca 8a4358f
new fcd7b6d Added Apache 2 license in the POM, removed checkstyle, cobertura and findbugs plugins (since we use sonar anyway) and added the maven license plugin
new e8384d3 Made validation metrics return Double.MIN_VALUE instead of (Double)Long.MIN_VALUE when the value is invalid, fixed inverted unit conversion
new 100a911 Fixed a bug where columns with a cardinality of zero (all nulls) cause the number of bits needed to encode values to be Integer.MIN_VALUE, as log(0) = -Infty
new 26944e8 Fixed bug where the matched resources would be a realtime resource where there is only an offline resource.
new 6ed8823 Code cleanup
new 02a097e Adding metadata utils skeleton
new 313091e Fixing dependency change for BrokerRequestUtil changes.
new aec97ae removing conflicts and fixing helix property store apis
new 6a610f2 Adding license headers
new 83e4ce7 - Introducing getRollupAggregationGranularity functionality to account of time dimension based aggregation in the RollupThresholdFunction API.
new d3e77db THIRDEYE-209 Metric selection resets when heat map changes
new 1d096e9 Adding ZKMetadata into pinot-common as an interaction with helix. Replace many places currently using ZNRecord by *ZKMetadata.
new a45d533 Move KafkaProperties to common
new 5eda2e4 pinot helix resource manager conflicts
new 6363aca Improve upload error logging by adding a message, so that the full stack trace goes into the logs
new 0ea8c41 THIRDEYE-213 Aggregation on dashboard double counts end point
new b13a923 Fix nullpo exception when timeunit is null, replace type coercion through string concatenation by the appropriate call to Long.toString
new 7a8dda3 THIRDEYE-212 Add funnel view to dashboard
new 9abdf0e Whitespace fixes: removed spaces/tabs on empty lines, removed spaces/tabs at the end of lines, replaced all tabs by two spaces
new ef1ce5c Adding missing license headers
new aa37613 Added logging, made most restlets return HTTP 500 on errors, improved messages sent to clients on errors.
new 41901fa pinot helix resource manager conflicts
new 1756a93 Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
new 555326b adding a realtime state mode, realtime resource data manager, realtime segment data manager, realtime segments listener, nothing is tied up as of now checking this in since xiang and I need to work on same parts together at the same time and this is the foundation
new fc65f01 adding licenses
new 54e49f2 THIRDEYE-223 Drill-down with "" dimension value breaks UI
new b420448 THIRDEYE-205 Baseline / current time series points have "undefined" as metricName in label
new 98e3161 THIRDEYE-226 Time mismatch on the UI
new 6920c5b Adding KafkaHighLevelCOnsumerBasedRoutingTableBuidler and surrondings.
new 7a18bd5 Move ZKMetadata related utls from HelixHelper to ZKMetadataProvider in pinot-common.
new abc719b Adding addSegment(SegmentZKMetadata); Interface to InstanceDataManager and ResourceDataManager.
new 9e294cc Make ZKMetadataUtls only depends on PropertyStore.
new df339d1 THIRDEYE-238 Funnel view does not reflect moving average / aggregate
new ecb3399 Removed useless object creation
new 43d60d2 Added a better toString for the ZK metadata
new 19cefa3 THIRDEYE-12 A user should view heat maps in order specified in StarTree by default
new 7738c12 Adding missing license headers
new eaec333 Kafka high level consumer integration
new 3f2246e Fixed names that had uppercase characters in the routing table builder factory class names map
new ee65cb7 Refactored realtime and offline name checks into the same method, fixed setOfflineResourceZkMetadata to use the offline name instead of the undecorated name
new a1ca23b - THIRDEYE-106 MRUnit testcases for pre processing stages of bootstrap workflow.
new 27a498d Fixed NPE on server due to path changes, causing all segments to error out instead of coming online
new ebce4c8 Extracted common code to get the property store
new 465b14f Adding missing license header
new bf75ff8 Use helix cluster name, not the resource name, when getting the property store
new a73ab5f THIRDEYE-218 Call StarTreeManager#remove instead of #close in CollectionsResource#deleteCollection
new 806a7cb fixing test realtime segment data manager
new 76985af THIRDEYE-239 Heat map cells do not reflect aggregate smoothing option
new 003dbf2 - THIRDEYE-243 Fixing MetricTimeSeries test
new 67bf81e THIRDEYE-233 Tool-tip for volume heat map ratios should be percentage, not float
new 8f4a2e5 THIRDEYE-233 Tool-tip for volume heat map ratios should be percentage, not float
new f8c7fc1 THIRDEYE-239 Heat map cells do not reflect aggregate smoothing option
new 803aa65 THIRDEYE-242 Using aggregate smoothing option and funnel view breaks UI
new ef3c53a THIRDEYE-249 Add /aggregate resource to compute aggregate for metrics in time range
new 6280c19 THIRDEYE-225 .funnel-quotient color should render visible on projectors
new 65e9e85 throwing unsupported access exception, in case of getRaw access in blockDockIds that are not backed by bitmaps
new 1b1d57d - THIRDEYE-241 Adding a cleanup method to delete the files created due to MRUnit tests.
new 8b0e274 Realtime/Offline Resource Bug fixing
new 65ebe31 THIRDEYE-250 StarTreeRecordStoreFactoryFixedImpl should be more robust to file-system changes
new c2f4573a Fixing TableNamePruner and SentinalQueryTest.
new 13f08b5 Fixing mismatched querysource issue. Migrate TimeBoundaryService to use ZKMetadata.
new 24fb654 - THIRDEYE-236 Adding checks while reading time column in the analysis phase to make sure it is a valid timestamp
new befa30f Added integration test that creates several segments from Avro data, uploads them to the controller and queries them through the broker endpoint
new 0f5d7ca Fixed integration test with latest changes
new 378a129 Updated integration test with RB feedback, fixed min, max and sum aggregation functions on filtered dictionary columns with null values
new 3b67709 Adding data file containing 100k rows to be reused between integration tests
new e4e0dde Adding missing license headers
new 4c78ee3 Changed test dependencies to be properly scoped to test and to depend as necessary on the other test jars
new ab970cb Removed an unnecessary offline suffix in the helix instance data manager after a rebase
new f4cb4fc Reverting rebase merge commit to be at xiafu's version
new 0129668 Extracted code to start the broker, controller and server into utility classes to allow tests to use them without inheriting from ControllerTest or ClusterTest. Added code to start Zookeeper during a unit test, as not to require a local Zk instance already started up.
new 81c2cee Adding missing license headers
new 15e2c49 Adding Kafka starter
new 690a857 Renamed integration test to OfflineClusterIntegrationTest
new 37c6972 PINOT-1362: Fixed table names appearing as "null" in the validation metrics
new 8bb1710 THIRDEYE-265 Drill down on "" value followed by any other drill down breaks UI
new 5c98ff6 THIRDEYE-252 Funnel breaks page when metric values are zero
new 2d977e8 Replaced hardcoded class names in strings with references to the actual classes, so that future refactorings don't break the tests
new 9c15691 Fixed integration test which broke after a move refactoring
new 3c8e272 Fixed helix broker starter test
new c1c49f1 Moved temporary files used in tests to /tmp
new 0f2f76e THIRDEYE-216 Server returns 500 when delete collection that does not exist
new 6f5a939 Start Zk as part of unit tests instead of relying on an instance running on localhost port 2181
new 2fbec41 Fixed broken unit test
new 6b35e58 Fixing getEnum exception for TimeUnit field
new 16cfad8 Make pinot-controller /dataresources/ endpoint return application/json header mimetype
new 72618b3 Realtime E2E test bug fixing.
new 605e23e THIRDEYE-258 Make ThirdEyeApplication deployable in-process
new b98fcd1 Fixed assertion occurring during unit test
new 2cbbdd1 THIRDEYE-268 Remove composite record store implementation
new 9ffedc1 Bug fixing for Realtime Indexing persistence.
new 39a6f27 Bug fixing: infiniti adding realtime segment to Helix.
new b802522 Fixed missing metadata when querying the dataresources REST endpoint with a resource name
new 9170cea THIRDEYE-256 StarTreeManagerImpl#remove should remove collection from openTrees
new 6b5f30b THIRDEYE-214 POST /collections/* should fail if resource already exists
new 1835614 THIRDEYE-214 POST /collections/* should fail if resource already exists
new e041d42 THIRDEYE-214 POST /collections/* should fail if resource already exists
new a436631 THIRDEYE-214 POST /collections/* should fail if resource already exists
new 4a2684b Fixed property store accesses failing due to the property store having an incorrect property store ZK path
new ed70da1 Added exception logging for the segment REST endpoint
new b6ef718 Realtime query processing. 1. add getTotalDocs into IndexSegment Interface. 2. move ResourceDataManager to get SegmentDataManager not OfflineSegmentDataManager 3. make different block implementations for selection query getRawEvent() method.
new a380bb5 Fixing concurrency issue of adding new segment from bootstrap.
new 0aa7c83 Make FieldType all upper case.
new 74bdefa making sure that we updated the maxmultivalues map
new 5cf30d4 Fixing broken query for realtime selection order by time/metric columns. Fixing select * query.
new 179d229 not explicitly converting time column type to long anymore
new d83ea4c returning specific data type based RealtimeBlockValueSets for time and metric columns to aid agggreation function computations
new cbb0963 Fixed PINOT-1376 (mismatched queries between Pinot 1 and 2) due to an exception occurring in Roaring Bitmap
new 9afb776 Fixed unit test after the FieldType case changes
new 82f146e THIRDEYE-246 Add test for non-beta HeatMap implementations
new e7df447 THIRDEYE-244 Dimension-by-value time series should display dimension value
new f03de55 - THIRDEYE-254 Exposing number of reducers to be configured in startree bootstrap phase2
new 87c3560 Fixing time value object inconsistency bug. Fixing groupBy query to take no dictionary column as group key. Fixing BlockValueSetIterator return type issue
new 0d463d7 - THIRDEYE-296 Taking aggregation bucket size into consideration while doing aggregation.
new 974cdd2 - THIRDEYE-219 : Adding MRUnit tests for rollup phases and startree generation.
new 9fd9079 - THIRDEYE-298 : Using input time unit size while computing source time window
new 1696da4 Fixed NPE
new 8e62a5c Fixed issues when the segment temp directory is shared over NFS between multiple controllers
new 5ce17ca Add instance ID and current timestamp to the temporary untar directory
new faa827a Added logging whenever we are not refreshing a segment due to it already existing, return success whenever we are not refreshing the segment due to it being identical
new 702f991 1. Adding realtime retention. 2. For RealtimeSegmentMetadata, still put hybrid resource name in SegmentZKMetadata and converted offline SegmentMetadata. 3. Fixing issue when we drop a RealtimeSegment, we may delete it twice.
new 5d8aacc THIRDEYE-290 : Server should return 400 Bad Request if provided invalid dimension
new 42f92e2 Revert "THIRDEYE-290 : Server should return 400 Bad Request if provided invalid dimension"
new 3f89411 THIRDEYE-290 : Server should return 400 Bad Request if provided invalid dimension
new 1c9cd28 THIRDEYE-290 : Server should return 400 Bad Request if provided invalid dimension
new 5460b28 THIRDETE-290 : Server should return 400 Bad Request if provided invalid dimension
new 7ef7947 THIRDEYE-290 : Server should return 400 Bad Request if provided invalid dimension
new 09eb333 THIRDEYE-229 Initial implementation for high-level Kafka consumer
new 831d961 Fixing realtime indexing issue with dimBuff hash computation. Changing data type from IntBuffer to ByteBuffer for dimBuff.
new 11d62f1 Adding RealtimeQuerySentinelTest
new 2a3cfbb [PINOT-1347] Fixing bugs for realtime indexing. RealtimeSentinelTest for aggregation and aggregationGroupBy
new e299338 Bug fixing for federated broker test. 1. fix logic for RealtimeColumnDataSource set range predicate as for realtime dictionary, dictId is from 1 to dictionary size. 2. in MutableDictionaryReader, getString is also used for other data type dictionary, so change it to toString(). 3. fixing offline ColumnDataSourceImpl would set filteredBitMap to null in Range query, which actually should be an empty bitMap. This is the root cause of DummyBlock get called. 4. fixing NPE in [...]
new c641c14 THIRDEYE-296 Complete implementation of snapshot heatmaps
new 428afc6 - THIRDEYE-221 : Adding MRUnit test cases for startree bootstrap phases 1 and 2
new 247badc THIRDEYE-304 : Make representation of divide-by-zero metric values on the dashboard uniform
new 9e87bb3 THIRDEYE-307 Add startTimeMillis to kafka.yml configuration
new 5da1714 Adding dynamic groupId creation to avoid duplication. If config: stream.kafka.hlc.group.id config is setting, we will pick up the string from it and as groupId. Otherwise, we will create it dynamically by given: [resourceName + timestampInMills].
new ff02a23 [THIRDEYE-293] Adding Generic Join Job
new f7aaaf8 Workaround for HELIX-590, where multiple concurrent writes to the same znode would cause some of the writes to fail silently, causing some partitions to not be brought online
new b9b2f73 adding Joe as one of the committers
new 65c5c74 Removed wait for partitions to be online in the external view, made the online to offline timeout configurable
new 1d9a1b7 In controller, make eval(..) function synchnized to make sure one time only one thread is creating realtime segments.
new 63c2f26 - {THIRDEYE-302} : Dropping record with timestamp field set to 0
new c5d838d [THIRDEYE-293] Fixing broken test case
new b841281 THIRDEYE-308 High-loss of dimensional data (most records go to other) from Kafka consumer
new f136bfc THIRDEYE-291 Add day as unit for baseline on input
new 2720fdb [THIRDEYE-293] Fixing broken test case
new 8192064 - {THIRDEYE-314} : Catching exceptions in StarTreeRecordStoreFactoryDefaultImpl.MetricStoreRefreshWatcher to make sure that the file system watcher thread never dies.
new 20569c5 Fixing issue with multi-value column equals null.
new eb77b49 converting enum and boolean fields in incoming realtime record toString
new 8715887 THIRDEYE-316 DropWizardApplicationRunner should expose MetricRegistry in return value
new 8d6cc33 Bug fixing for handling null value in multi-value column.
new 690c46c THIRDEYE-299 : Add TimeRangeHealthCheck to server
new 3a81f50 THIRDEYE - 296 : Complete implementation of snapshots heatmap
new 3a8e34d THIRDEYE-299 : Add TimeRangeHealthCheck to server
new 531373d THIRDEYE-320 Publish shaded jar for thirdeye-server
new 312ecda THIRDEYE-316 DropWizardApplicationRunner should expose MetricRegistry in return value
new 0d5f238 Adding enum to string converter for Object[].
new 7d2461f Close all file handles after creating a dictionary, added a utility to unload direct byte buffers from memory
new f4d73da Adding missing license header
new a31a85b Compilation fixes
new 651188b THIRDEYE-321 Multi-time series accept: */* rendering error
new 3f98617 Removing references to AutoCloseable, since it's a JDK 7 feature
new fe86a79 THIRDEYE-316 DropWizardApplicationRunner should expose MetricRegistry in return value
new d294cdd Fixing for getting data type inside avro from union data type.
new fdfe62f THIRDEYE-320 Publish shaded jar for thirdeye-server
new 04b3406 THIRDEYE-300 : Add StarTreeHealthCheck to server
new 6e8178c THIRDEYE-322 Downgrade thirdeye-server avro to 1.4.0
new 31fbdfe Remove unused config in segment creation; Adding more logs.
new aa0217e fixing for set/get time column name in SegmentGeneratorConfig
new d0e6416 Adding Kafka metrics
new eb98e5d Fixing the issue of fetch kafka schema everytime when indexing event.
new 75063b9 Fixing NPE from uploading large segment.
new 3dcfe6f Fixing issue when delete resource, controller won't clean up PropertyStore clearly.
new 388e507 THIRDEYE-319 : Add health check to identify possible holes in the data
new 8079d8c THIRDEYE-319 : Add health check to identify possible holes in the data
new 4efd9f7 THIRDEYE-319 : Add health check to identify possible holes in the data
new 1ccfcee THIRDEYE-323 GET "/" should redirect to "/dashboard"
new 81f0aaf THIRDEYE-323 : GET "/" should redirect to "/dashboard"
new a642851 Move realtime segment persistence dir to "$resourceDataDir/_tmp" to prevent potenial filling up /tmp during large segment creation.
new d73701f Adding dynamic config to initial RealtimeSegmentDataManager
new 9c97c8d Fixed unit test that broke after the time column fixes
new a4db2b0 Changed hardcoded path in unit test to one that's looked up at runtime
new 3abad54 Fixed unit tests (DefaultReduceService, QueryExecutor) that broke after the time column fixes
new 63c2519 THIRDEYE-318 : Output counters for the total sum of each metric in startree_bootstrap_phase1
new f72569c THIRDEYE-318 : Output counters for the total sum of each metric in startree_bootstrap_phase1
new abbcf7c Adding if-check for skipTo(docId) method. For realtime segment query, we may skip to a docId larger than the searchable index.
new 23612be PINOT-1395 Make Predicate to be abstract and make EQ, NEQ, IN, NOT_IN, RANGE predicates.
new f693746 Lazily load bitmap inverted indices
new 6353105 Properly handle Exceptions for timeBoundaryService.
new d557cfb Rethrow load exceptions so that Helix can properly mark the partition as ERROR instead of ONLINE
new 599d182 Add segment load retry logic with exponential backoff
new 240a2aa Clean up temporary files when an exception occurs during download or unpacking of a segment file
new 8f37e20 Fixed TestDimensionsAndMetricsSerDe, which was expecting null, not the string "null"
new 676ee02 Fixed unit test that depended on zk running on localhost:2181
new 73ba084 Adding Mayank to the ACL
new f695ec7 THIRDEYE-324 HDFS data directory should use wall clock time
new eed0e54 Adding code for python-based pinot dashboard
new 699024c Incorporating Dhaval's suggestions
new 9309762 Incorporating more of Dhaval's suggestions
new e0e3a11 [PINOT-1410]: Quick Start Initial Version on local host
new 6ce29e3 Converting pinot-dashboard readme to markdown
new 5454e00 Converting pinot-dashboard to be deployable egg
new e8e6aa8 PINOT-1434: Adding performance benchmark driver, set up local cluster, uploads segments and runs perf queries.
new c1cf2e1 Adding comment for usage of .unTar versus .unTarOneFile
new f3b75c1 Run queries against H2 instead of using hardcoded result values
new 66bd443 Run generated queries instead of fixed queries
new 84cb446 Generate different queries for H2 and PQL dialects
new a687578 Adding missing hashCode method (for parity with equals)
new 1ada48b Code cleanup: Added logging instead of printStackTrace statements, used IOUtils.closeQuietly instead of try/catch blocks
new dd81c74 Removed hashCode method that was unused, as none of the FilterQueryTreeConstructors are on the key side of a hashmap
new 9f16989 More code cleanups: removed unused imports, merged ifs, renamed variables and removed unused fields
new bd2ac20 More code cleanups: Fixed equals methods that did not take Object, added missing @Override annotations
new 67eaf62 PINOT-1456 Setting default value for limit and offset
new eb5e157 Refactored integration test, made functions in the query generator lowercase
new 6cd65b3 THIRDEYE-104 Add SQL support for star tree queries
new 18e7dfc THIRDEYE-266 A user should be able to delete individual time segments
new 484934c PINOT-1456 Setting default value for selection limit and offset in request.thrift file and regenerated the source files
new f44037d Exception handling cleanup and added exception logging for pinot-core
new a877799 Added projection queries to the query generator
new 22c99c8 More code cleanups: Added logging, added missing @Overrides annotations, etc.
new 9d81e31 THIRDEYE-325 : CollectionsResource should record lastPostDataMillis on a per-collection basis
new 3c05523 Fixing dictionary interfaces and implementation
new 4d5ea8d THIRDEYE-334: Default date on dashboard should be max date loaded into collection
new d8da0e8 PINOT-1434 Consolidate Helix access into one class
new 7a761ca 1. Bug fixing for string dictionary collect handling null value for multi-value column. 2. Bug fixing for LongDictionary Reader.
new 933b201 PINOT-1465: Fixing complex filter tree constructor use duplicated hashcode as id.
new 30a09d7 Added realtime integration test
new 4f13ac7 Added missing license header
new 8c6fd1c Fixing NPE during filtering unexisted value in realtime segment. Fixing no docs matched issue for aggregation functions.
new 058d5ba making read faster in realtime segment
new 1a51b1a incrementing the success count as well in the realtime segment impl
new 136f7e6 Added logging where missing, replaced throw new RuntimeException(e) calls by Utils.rethrowException
new 67ab98b Fixing RealtimeDictionary issue. Now RealtimeQuerySentinelTest passed.
new 3e9bb89 Fixing issue of block iterator that last doc get EOF.
new e593344 Replaced System.exit in a test by a volatile boolean check
new ca07c81 PINOT-1448: First set of Performance improvements targetted towards selection queries
new 8e5687f Refactored interegration tests to share the common tests, added hardcoded query set
new 3a4e5f1 Rebuilding the Thrift-generated code now requires -DbuildThrift=true, as it breaks subsequent builds later due to there being no license headers on the autogenerated code
new 468bcfb Remove _segmentId and _docId from selection. Fixing time value iterator. Fixing multi-value block reader of null value selection. Remove unused log from TermsFilterQueryTreeConstructor. Unit test fixing for: TestPlanMaker,TestSelectionQueries,TestSelectionQueriesForMultiValueColumn
new 08a8266 Adding default null value to FieldSpec. Adding default null value to realtime dictionary. Fixing the issue that time field only persisted dictId to disk. Simpliying rowComparator logic in SelectionOperatorService. Removing dummy timeFieldBlockIterator.
new da1417f Adding simple support for filtering metric column. This is a temp workable solution. Need to improve this later.
new ba6e81a THIRDEYE-337 Collection restore with no data dirs requires server bounce
new 75ac4e4 THIRDEYE-338 Refactor dashboard into separate thirdeye-dashboard module
new 7387ba0 Added hybrid integration test
new 84e1ed0 Renamed variable that referred to offline servers, removed code that is not needed/fixed compilation since RB466683 is in master (test refactorings)
new 5a3e038 PINOT-1448: RB=467880 Optimizing the bitmap intersection logic by ordering the evaluation according to the posting list length
new cead35b finxing inconsistensies between realtime segment and the converted offline segment, now data matches across the realtime and the converted offline segments
new 899ab7b making size consistent across types of realtime blocks
new 6e1f095 remove auto generated comment
new 9d02e92 fixing realtime block iterators hasNext logic
new ffc1bfd Generate a random number in the getRandomFileName instead of just using System.currentTimeMillis, as it causes a race condition if multiple threads are converting segments and start conversion at the same millisecond.
new 72b1ed0 Adding inverted index for realtime segment metric column.
new 222fc9d PINOT-1448: Removing unneccesary conversion of bitmap index from immutable to mutable
new 149ea7e Fixing pinot-server unit test
new c923543 Adding configurable broker timeout
new e96b601 Change method name for Jackson
new d2be513 Making aggregation functions to take care of null value. Changing query set for integration tests.
new 9145c2c added a static queries file and also now segment creator assigns default null values for all data types, no more null checks
new 5056065 converted unnessecary log.info to log.debug and also generated the static query set again
new a8c5c49 made changes to make realtime test schema creation correctly and also handling default values for null in relatime ingestion as well
new 3e9996b adding sorted column inverted index creator as an inverted index creator
new 555ac3b adding sorted column inverted index reader of type inverted index reader
new 488c6a8 using InvertedIndexReader instead of Bitmap*Reader
new 671e398 split the actual data source predicate filtering and bitmap or'in .. moving the dictionaryId filtering into a separate util class so that it can be used elsewhere too
new 273c30c refactoring And and Or operation performed on bitmaps inside BAnd and BOr operators in to a util class BitmapUtils
new 7d4303f now using BitmapUtils for or'ing at the predicate level as well
new 48c488f more refactoring to DataSource, Blocks and Predicate evals
new d16e44c added a sorted inverted index block, currently does nothing different
new 13d94e9 Added a DataSourceMetadata inteface and relavant impls, some more refactoring, also added a backup filter2 method
new 5a31474 logging query , expected and actual values on Assertion failures
new c0cc92f Adding Scan based AND/OR block/iterator, which compute the filtering on the fly.
new 6cb692d adding license headers
new 4500698 Adding logic to filter plan node to take care of filtering with/without inverted index.
new 2658c30 single and multi value blocks without inverted index
new 9179509 single and multi value blocks without inverted index changes
new bf180e1 single and multi value blocks without inverted index changes
new 7deb925 THIRDEYE-340 Add funnel view to thirdeye-dashboard
new 9a9833e Added missing integration test data file update, removed table creation in hybrid and realtime modes
new 7405617 (ThirdEye dashboard tickets)
new 0d769d9 fixing some edge case bugs in data block iterators, also added a blocks test to compare BlockDocIds with filtering across fwd index and inverted index filtering
new 2f59b4f changed the integration test query set since the data set has been changed
new 9137561 Enable segment pruners for time interval and data schema. Adding hardcoded query set of OfflineClusterIntegrationTest
new d8082fd license headers
new 44f2c15 Fixing RetentionTest by change the delay time to start.
new 9f28b45 Fixed data not being written properly in the realtime integration test, added kafka directory creation/cleanup, fixed single valued fields being treated as multivalued, added logging for kafka production
new 3e45f31 Added thirty minute per test time limit, to avoid stalled tests
new f248729 Fixing NPE in ControllerSentinelTest. Fixing TestPinotResourceMananger by using empty server instances.
new c895923 Added logging for exceptions occurring in the state model transitions, rethrow exceptions instead of swallowing them, so that Helix can get them
new b2279f3 Added property to override the segment creation time, so that we can keep it on segment conversion
new e43ed83 Added two minute time limit for realtime consumption to complete on integration tests
new ece347e Fixed inverted constructor, removed unnecessary boxing, removed unused variable assignment
new 01f188b Refactor HelixHelper from pinot-controller to pinot-common. Change the module dependency for pinot-controller to test scope in pinot-broker. Fix failed pinot-broker unit test.
new 3f72bd5 THIRDEYE-344 Add ratio support to dashboard
new 03e2f2a THIRDEYE-344 Add ratio support to dashboard (fix bug)
new 9cc533d PINOT-1478: Adding base reader writer interface for realtime
new 2a09a05 license headers and constructor for *ReaderWriter
new f956578 modified constructor for *ReaderWriter based on feedback
new f70c106 Fixing NPE from segment pruner by adding schema to realtime segment metadata. Fixing NPE from not in predicate. Adding hard coded query set to RealtimeClusterIntegrationTest.
new 4d13592 Fixing {Offline/Realtime}ClusterIntegrationTest. Change retention to be 3000 days to avoid segments being deleted during test.
new e476d9b Added missing hashCode methods to classes having equals methods
new a570939 Added unit tests for ZK metadata equals and hashCode methods, fixed issues with equals and hashCode
new 6219b34 Adding missing file, cleaned up some equals and hashCode methods
new 4dec6a9 Adding missing license header
new d9734ab Added equals/hashCode to KafkaStreamMetadata and Schema, removed star imports
new b1a96b1 Fixed failing unit test
new 59102d4 Clean up OfflineClusterIntegrationTest/RealtimeClusterIntegrationTest. Move the common test cases to BaseClusterIntegrationTest
new df1dcd6 THIRDEYE-345: Separating Hadoop workflow into DIMENSION_INDEX and METRIC_INDEX
new d0d4a8b Initialize the metadata of the data table with some defaults, fixes NPEs in DefaultReduceService and BrokerRequestHandler
new f7784d5 Fixed Kafka consumer path for hybrid
new 1942330 [PINOT-1410]: Quick Start Pinot Admin Console contd. Cleanup after RB feedback.
new f6f40f5 THIRDEYE-344 Add ratio support to dashboard (fix bug)
new 37d0630 avoiding recomuting filteredIds twice
new abeb830 Fixing HybridClusterIntegrationTest by changing some member variables from static to non-static in HelixServerStarter and SegmentOnlineOfflineStateModelFactory. Fixing the override issue from error msgs to response Fixing EqualityUtils for map fields.
new 05ba487 THIRDEYE-346 : Throw exception from hadoop jobs for Azkaban to identify failed jobs
new 1b8be22 THIRDEYE-335 : DELETE of collection resulted in NPE
new 57d0c78 Remove test annotation for TestPerfBenchMarkDriver
new 5b02f6e PINOT-1478: Adding implementation for ReaderWriter for SingleColumnSingleValue and SingleColumnMultiValue
new 7df2e50 minor cleanup
new 4208e2a Realtime Segment now using *ReaderWriter for indexing and scanning
new 5db84fa feedback changes on Realtime Segment Changes
new f180c13 THIRDEYE-348 : LeaseExpirationException on hdfs as multiple mappers writing to same file in StarTreeGeneration
new f7248a8 Adding IndexLoadingConfigMetadata and use it to switch on loading inverted index.
new eecd898 Change jvm setting for mvn test
new c3458c6 Removed some redundant column type code, added time type for time columns
new e13ac0e Removed some unused variables, extracted some hardcoded constants
new 98c1127 Removed Praveen from the product owners, to cut down on email spam
new 1decce8 THIRDEYE-336 : RollupPhaseThreeJob during bootstrap should not fail if it can't find dimension combination
new f533ef7 Refactorings changes for Pinot1 vs Pinot2 comparisons
new c24181c Refactorings changes for Pinot1 vs Pinot2 comparisons
new 8d8b299 Separate SelectionOperator to SelectionOnlyOperator and SelectionOrderByOperator. Abstract static functions to SelectionOperationUtils.
new b3794de Copying DropWizardApplicationRunner to thirdeye-dashboard
new 524c0b9 Fixed unit test coverage not appearing in sonar
new 6c4ec80 Upgrade thirdeye-server dropwizard dependency to 0.8.1
new 7843a55 Reverting (Upgrade thirdeye-server dropwizard dependency to 0.8.1)
new 41c21cc Reorder the data schema segment pruner checking sequence. Swallow zk shutdown exceptions for OfflineClusterIntegrationTest.
new cecbd57 Use jackson 2.4.3 in thirdeye-server
new 4de59cd Chunk POST request in server_push
new 2d4f601 Made segment creation multi-threaded.
new f26f8c3 PINOT-1448: Rewriting the Filter Operator Plan and execution phase to incorporate efficiency
new 8ef3198 Bug fix: File size must be in bytes not bits.
new b35e582 THIRDEYE-350 : Output metadata file with min and max time after StartreeBootstrapPhase2
new 42e01ec THIRDEYE-350 : Output metadata file with min and max time after StartreeBootstrapPhase2
new 427e4f5 THIRDEYE-350 : Output metadata file with min and max time after StartreeBootstrapPhase2
new c979071 THIRDEYE-350 : Output metadata file with min and max time after StartreeBootstrapPhase2
new a7c2de8 PINOT-1449: Refactoring RangePredicate Evaluation logic into one class
new 8a5a9d3 Use available number of threads.
new 1e38d18 Adding another port for brokerTest
new e45ae07 THIRDEYE-351 : Bootstrap should continue gracefully if no records below threshold
new e14f81f THIRDEYE-351 : Bootstrap should continue gracefully if no records below threshold
new ede2404 Adding mutable dimension store
new d447905 PINOT-1457 Converted code to use slf4j instead of log4j or JUL
new ed55988 THIRDEYE-262 (Several dashboard bug fixes)
new 5856cbc Minor UI fix
new 6ec9582 Added api to set port for broker.
new 12fed8d Logger cleanup part 2: All loggers renamed to LOGGER, all loggers made static and final (except where we can't)
new 9318505 Fixed loggers in if statements for thirdeye
new 0c17e1e Downgrading thirdeye Kafka dependency to 0.8.1 for container compatibility
new f6fc9b8 Added another exclude for thirdeye-realtime
new 6a4f5fe packages refactoring for index creation
new 082d9ab refactored forward index creation during segment creation
new 84689bb added license headers
new 97392f5 more index creator refactoring
new 62c058d foundation for Segment Loading refactoring
new 3a37ba7 finished implementation of the *ColumnIndexContainers, also checking in a new IndexSegmentImpl and DataSourceImpl, the plan is to get the new impls to replace the new impls eventually
new 3c1eea2 IndexSegmentImpl now uses the new *ColumnIndexContainer, also cleaned up segment loading
new 94876d3 changing perf drive to use the new apis
new 26527d2 merged with kishore's changes, QueriesSentinelTest passing
new f076305 merged with kishore's changes, also made modifications to Range Delimiter and MutableDictionary(min,max) values
new 8730e26 adding dictionary check into the bitmap operator for realtime vs offline range computation
new 1cb42d5 fixing realtime dictionary get checks, removing default checks for == nulls, all tests pass
new 66242ba Adding more logs for dealing with segment deletion.
new 4884ccc adding license headers
new 7b31014 Revert "Adding mutable dimension store"
new f2c9706 Invalidate the segment metadata if refresh fails, log if for some reason the ZK write to the ideal state fails
new 9ef95f3 Code cleanups: Added exception logging, adding missing hashCode method, made fields final, changed ControllerRequestURLBuilder to be immutable
new a23aa97 Added multiple node offline cluster integration test
new 1e798cd Added missing license header
new 34d59a6 Display stats in dashboard tooltip
new 56a0907 Minor fix, dashboard metadata in table
new 02aa2a4 Minor fix, do not select default metric if some selected
new 7de0378 Minor fix, offset bug in dashboard overlay timeseries
new 10db28a Use separate path to static content to avoid conflicting with mppy tools wrapper
new 38884de now sorting the data on the sort column while converting segment from realtime to offline
new ad451a1 adding dictionary for metrics in realtime segment
new 8b287cb removing the hasDictionary check from the filter plan node
new 4d7f7ce added predicate evaluators
new 9bcac65 using predicate evaluators in all the filter operators
new 2eabbc4 adding default values in the query generator for nulls, also using bitmap operator when inverted index is present in the filter plan node
new a0742a6 adding license headers
new a9b7139 removing fastutil sets from segment creation
new 78674fe renaming dicIds to dictionaryIds in the evaluators
new 26ef88c Adding provisions to hook into the wrapper's auth mechanisms
new 69a7ad0 Fixing broken version pin
new b02cc23 Using BlockingQueue for server merging phase. Fixing NPE in related combine functions. Remove Exception handlers in MSelectionOnlyOperator. Handling metadata only DataTable Ser/DeSer.
new 69b8a9d PINOT-1507: Adding skipList based storage for Multi Valued columns
new 1812532 Adding sorted column list config into DataResourceZKMetadata and link it with RealtimeSegmentDataManager.
new d5ca514 Added log4j.properties for pinot-admin
new e0a488f using FixedBitSkipList reader/writer for MultiValue columns
new 4085079 changes to the *Stats collector
new fab6dbf Fixed broken unit test, added unit test for findNthBitSetAfter, corrected implementation of findNthBitSetAfter
new a7f55ea Removed Sonar warning about passing an int to longBitsToDouble
new 0941a44 Added comments and renamed a variable to make OrBlockDocIdSet more readable
new 75a0380 Fixed file handle leaks
new 8ae99b4 Re-added generated queries to the integration tests, added logging of mismatched queries with debugging information, added log replay, fixed number of generated aggregation functions to 1 for easier debugging, changed query comparison to check that Pinot results are contained within 10k rows of H2 results (skipping comparison if more than 10k rows are returned by H2), added dataset with no null values, changed query generation to not quote numbers sent to H2
new c4409c1 Skip check of selection queries for now
new f123088 Extracting sorted columns list from controller command, key is metadata.sorted.columns
new 7654a19 Fixed thirdeye-realtime to use new data directory structure
new 40320da Fixing range function in StringMutableDictionary Fixing PriorityQueue constructor, as raw.length == 0 is not valid. Adding handle ParsePQLException in Broker.
new 3abaf44 Minor fix - more logging in thirdeye kafka
new 2ef1674 Minor fix - use IOUtils instead because of classpath issue
new 7dd6a93 Fixing the issue of selection will reduce number of rows more than size from query.
new 183b962 Fixing scan based matching docIds find issue by change the binary search to hashset. Adding a test case in QueriesSentinelTest to cover this case.
new c379ac2 Escape HTML from dimension values
new be00ebf THIRDEYE-365 Add support for nested fields in ThirdEyeAvroUtils
new a18c751 Fixing MProjectionOperator got called many times issue.
new f437d61 removed unused blocks
new 61486ef fixing testReatlimeSegment
new a4204e5 Adding interface for FilterBlockDocIdSet and abstract class for BaseFilterBlock to force some implementation for filtering. Fixing selection unit tests by not hard coded results set. Fixing AndOperator/OrOperator logics to handle all the BaseFilterBlock.
new 50109ce PINOT-1518: Removing JUnit dependency
new 1eb3287 remove adding of default value to realtime dictionary on initialization among other things
new 0a06c5d PINOT-1518: Removing JUnit dependency
new e6055ab 1. Fixing String comparison in string dictionary by reversing the way to store a String: first put String then use * to fill the gap. 2. Rewrite the sorted column docIdIterator, the logic before takes multiple pairs but return all the docs between the left value from first pair and right value from last pair. 3. Fixing BitmapDocIdSet advance(int) and next() method. next() should first trim data in priorityQueue, then add new pairs, after all added, we shouldn't remove th [...]
new ad89fa7 postData streams upload instead of keeping all in memory
new d7d83fe Adding file loggers for controller/broker/server in quick-start
new b6aca04 Fixing filtering query bugs with *BlockDocIdSet changes.
new f942cd6 PINOT-1518: Clean remaining JUnit instances
new c8db539 Ignore text and log files from the license header check
new b8a9ab2 Put HelixSetupUtils and ZKMetadataUtils into package: com.linkedin.pinot.controller.helix.core.util
new cabd6b6 Added support for multi-value columns to the query generator and H2 insertion
new 4fd37bd Updated log4j.properties
new 13bcd24 Remove table concept from codebase
new 3e8e1d9 Several tickets...
new 878899e PINOT-1231: Rename all tests with Test prefix (eg. TestFoo) to suffixed form (eg. FooTest)
new 3a3fc45 Adding tenant into controller
new 6dec3c4 adding schema apis
new 14f7f4f adding new table APIs ... currently they are not wired
new 1d247cb finishing up all table APIs update and get
new 103234f renaming ResourceType to TableType
new 7595719 adding license headers
new b6c198a PINOT-1529: CSV record reader
new 23439d6 PINOT-1529: CSV record reader cleanup
new d85d2b8 PINOT-1529: JSON record reader.
new 374675d removing constants with enum in Tenant name builder
new adeb1f5 removing constants with enum in Tenant name builder
new 503e6ea Adding ControllerSentinalTest for v2 apis with addTable works.
new d419663 Fixing RetentionController with v2 APIs
new bbbadb6 Fixing ValidationManager with controller v2 apis.
new 9261db1 Fixing broker,controller,transport v2 apis Remove useless test Add deprecated notion to v1 apis
new fe208d6 adding get calls on table for get brokers, servers and get configs
new 9652e3b fixing compile issue
new f451992 adding sorted columns in the indexing config
new 2aa7046 renaming Resource to table in pinot-server
new f997507 making server and core using the new table Config
new 156b791 adding a Property Store helper
new 6c52c8c bug fix in the property store helper
new d4c1bbb making realtime, offline and hybrid integration test pass
new 2d37836 making realtime, offline and hybrid integration test pass
new e97750f Fixing pinot-broker test. Delete deprecated tests.
new f84e19a Removed dead code that accesses Helix, added HelixHelper.updateIdealState which has retry logic to update the Helix state in ZK in the case of concurrent updates
new ca57f5d PINOT-1529: Bug fixes in JSON/CSV Record Readers.
new eb4698f PINOT-1529: Bug fixes in JSON/CSV Record Readers.
new 9582d39 [THIRDEYE-371] Select the trees to query such that the number of trees to query is minimized
new 3847f13 THIRDEYE-372 : Include wall clock start and end time and schedule in metadata file
new 7289e57 refactoring tenant
new e5d235c add tenant tools command added
new b7685f0 added table update apis
new 66c00fe adding missing segment get APIs
new 9abf0a3 renaming addTenant to createTenant
new 955cf22 added add table, add schema and add tenant commands, also added sample_schema, sample_tenant, sample_offline_table and sample_realtime_table request.jsons
new f97936e PINOT-1529: Enhance CSV record reader to take a config file.
new 1ad763e Clean up controller v1 apis.
new 42a905c Move all the resource related methods, variables, configs to table related stuffs.
new 5084cd5 Fixing NullPointerException during getSegmentInfo. Return Empty response instead of null if routing table contains nothing, this is corresponding to querying a table without any data.
new 1e381ca [THIRDEYE-371] Select the trees to query such that the number of trees to query is minimized. Bug fixes and adding test case
new c13aea2 [THIRDEYE-371] Select the trees to query such that the number of trees to query is minimized. Bug fixes and adding test case
new 09a1d13 Removing author tags for open source preparation.
new cc8f478 Adding instance enable/disable Apis.
new 01a65c1 Added setters, renamed class, and other cleanups.
new 5bead1a THIRDEYE: Multiple changes related to unit of time, metadata and selecting trees to query
new b8983dd THIRDEYE-375 Clicking tabular view off-by-one error
new f95348d Adding copyright headers to pinot dashboard python files
new d6591a4 THIRDEYE-372 : Include wall clock times, aggregation unit, bucket size, sxhedule and data time in millis in metadata file
new 4223ed6 Pinot client
new 3bc076c Added logging for the pinot client, update the integration test to use the pinot client ZK path, added connection setup through properties
new 31fc05b Added UI for AddTenant and AddTable commands
new 9e2243a Adding balancedRandomRoutingTableBuilder
new ccbf00b Adding single tenant cluster as a default option.
new 666f4de [PINOT-1597] Improving the performance of Planning Phase
new e3029ef Removed classmexer due to uncertain license terms
new b60c3d3 Wire balancedRandomRoutingTable to broker.
new 6ed14d6 Fixed javadoc generation and various javadoc compilation errors
new 989c577 Removed author tags in the client
new 32e4b24 THIRDEYE-379 : Segment descriptors should receive time in millis
new 73ee15e Fix NPE in integration tests when starting the broker
new a699a29 THIRDEYE-376 Metric tabular view is off-by-one
new 7c70b57 Fixing NPE in getting routing table builder
new 2bfef9a Made /tables on the controller list all tables instead of just saying that no table name was specified
new 46d7d0f THIRDEYE-377 : Cleanup job to remove folders generated by hadoop workflow
new 69e8608 THIRDEYE-377 : Cleanup job to remove folders generated by hadoop workflow
new a30f52d Accommodating changes to controller API endpoints
new 1681504 Adding more metrics for server phasing
new c64d844 PINOT-1599 Javadoc fixes post-refactoring
new 1a67a1a PINOT-1538 Added license headers for thrift and antlr files
new 3e49def Fixing the valueOf a dateType for double type.
new e24727e PINOT-1607 Add sleep between requests to fix flaky unit test
new 9bd2f4e Fix NPE in metrics if the BrokerRequest has a null query source
new c292bc1 Fixed NPE on broker startup
new ae9a965 Fixed segment exception logging
new 3217640 More Pinot 2.0 Dashboard work
new 143c4aa Fixing gitignore for pinot-dashboard specific vim swap files
new 6ba84ae Adds timezone dropdown, Volume/Contribution Modal
new b551ed8 Migrates from search to hash for timezone
new 92af593 Adds else statement to auto set timezone if timezone is set as empty
new 3b52e90 Run 10k queries instead of 1k, correct comparison in other branch, use cast to double for average queries
new 9716316 Fixing the string write for dictionary with special characters, as for special chars, the string.lenth() != string.getBytes().length.
new 979e4c3 THIRDEYE-388 : Add "thirdeye.skip.missing" config property to skip input.paths that don't exist
new ef516c9 THIRDEYE-388 : Add "thirdeye.skip.missing" config property to skip input.paths that don't exist
new 5f41978 PINOT-1308: Replacing internal data in test.
new 4d7029b [PINOT-1597]:Adding custom serde's instead of using java's object serialization
new dd2e6c5 fixing shema NPE
new 1d35219 PINOT-1308: Replacing internal data in test.
new 620e5bf Given charset to getBytes() method to avoid platform default encoding mismatch.
new 0b4a508 PINOT-1308: Yet another round of data scrubbing
new 893718c Accommodating changes to controller API endpoints
new 41477e7 More Pinot 2.0 Dashboard work
new 006490e Fixing gitignore for pinot-dashboard specific vim swap files
new 468814e Adding tenant creation functionality
new af047c1 Merge branch 'dashboard' of gitli.corp.linkedin.com:pinot/pinot2_0 into dashboard
new 4ee94d4 fixing segment view dates
new 36e96eb PINOT-1308: First round of cleaning regression failures after data scrubbing
new c7f7fce Tenant and Table creation tweaks
new d12d914 Merge: 36e96eb c7f7fce
new 157a644 Added test where we freeze ZK for a couple of seconds to test error recovery
new 03edade Adding dictionary caching during segment creation.
new b74be5d Adding trace logging into NettyServer for pinot-server request latency.
new cb7b5ff THIRDEYE-380 Add support to query in-memory tree
new e7047c2 PINOT-1308: Test fixing after scrubbing internal data, contd.
new 73eccf6 Set the version of the libraries for which various versions existed through transitive dependencies
new 51a1dbc Minor improvements to the client
new 9bc5d1b THIRDEYE-377 : Cleanup job to remove folders generated by hadoop workflow
new 3a71e17 THIRDEYE-377 : Cleanup job to remove folders generated by hadoop workflow
new a9b5a21 Clean up netty dependencies
new 23e03f1 THIRDEYE-380 Add support to query in-memory tree
new d8e4ca5 THIRDEYE-388 : Add "thirdeye.skip.missing" config property to skip input.paths that don't exist
new f3c7385 Fixed unit test post-anonymization
new ac44f14 Fixed another unit test
new a3779d4 Fixed another unit test
new 49fa1b8 Fixed another unit test
new 3844e84 PINOT-1308: Fixing tests failures after scrubbing internal data.
new 24eb76a Added logging
new bb4479c Remove getLock when querying the segments and use ConcurrentHahMap.
new 46cad3c PINOT-1608: Quickstart cleanup
new b269443 Fixing NPE for getting next row, if the column from avro is not in pinot schema. Now we will just simply skip it.
new 3c5c361 Remove all the callables from metrics and addPhasingTime after the call.
new 22f1581 Fixing multi-value group by test with new data set.
new 3f70b31 fixing ChunkIndexCreationDriverImplTest tests
new 4e6fc7f fixed RealtimeFileBasedReaderTest
new e6e35d8 To fix all the unit test.
new f95faaa THIRDEYE-393 : Purge all segments of lower granularity once one of more coarse granularity is availble on HDFS
new a6819a0 THIRDEYE-393 : Purge all segments of lower granularity once one of more coarse granularity is availble on HDFS
new a56195e THIRDEYE-389 : Use proper umask when creating HDFS directories
new 39841ad THIRDEYE-389 : Use proper umask when creating HDFS directories
new 8a4f6e4 THIRDEYE-389 : Use proper umask when creating HDFS directories
new 7d7d361 NPE fix
new bff9432 PINOT-1308: Replacing all data files with internal references with new ones.
new 5d696d6 PINOT-1308: Replacing all data files with internal references with new ones.
new 5996f77 PINOT-1610 Make integration test length configurable
new c48e114 Fixed unit test
new 5dfb59e Quickstart bug fix
new 6579497 refactored schema and made all the necessary changes
new 2f5c88c [PINOT-1597] Loading all segment files using DirectMemory to avoid GC issues
new 99d8d6e adding a live instances listener, not wired yet
new 2079039 Fixing the file descriptor leak for mmap mode and heap mode. Adding test for heap/mmap mode reader and make sure no file opens after close() method is called.
new 525dd87 Fixing lsof issue.
new eff5603 Adds info tooltip to Aggregate and Moving Average
new ef7c803 Changes tooltip text
new aad717e THIRDEYE-400 thirdeye-dashboard breaks with data resulting from XSS attacks
new 08a2919 NPE fix
new a668ebd THIRDEYE-390 : Add Hadoop phase to poll for existence of input.paths
new 7f68be5 THIRDEYE-390 : Add Hadoop phase to poll for existence of input.paths
new 6755a6e THIRDEYE-390 : Add Hadoop phase to poll for existence of input.paths
new c4bb0dd removed schema version
new e59fbca Updated readme to make it the same as the GitHub wiki's landing page
new 7812a6c Adding filter query schema check in DataSchemaSegmentPruner.
new a7e1614 THIRDEYE-356 Remove dashboard related code from thirdeye-server
new ba8aafe THIRDEYE-330 : Create DataGeneratorTool which can generate Avro data sets to load into ThirdEye
new 32f2c27 THIRDEYE-330 : Create DataGeneratorTool which can generate Avro data sets to load into ThirdEye
new 375159a THIRDEYE-330 : Create DataGeneratorTool which can generate Avro data sets to load into ThirdEye
new 031d3b1 Fixing NPE for flitering query. Fixing exception in combinePlanNode.run() will cause query to be halted.
new 6aac094 [PINOT-1597] Removing lock while updating metrics, limit the parallelism per query based on number of segments
new c51c075 Removing pinot-util module
new c17cbce quickstart and the new console
new 389b8c3 deleting un-necessary css files
new 050b52b review changes
new 7c57155 attaching a liveinstance change listener
new f906178 adding some more logging to live instance change listener
new 03307be Pinot-Admin cleanup.
new bbf6538 Adding pinot-dist module to generate the pinot distribution for users to download
new dbced7c Pinot hadoop job
new 4750b2e Build fix
new 12d42c7 Modifying chaosMonkey test with new quick-start api, the test still fails though
new 86aa4e6 Modifying chaosMonkey test with new quick-start api, the test still fails though
new 75150f7 PINOT-1589 Remove dependency on pinot-common from pinot-api
new ceab4bc Adding detailed pql parsing error into query response.
new c5a5eca Adding double value comparator.
new c2e1582 Remove the assertion for getOpenFiles calls.
new d927e5e Fixing the issue that OfflineTableDataManager won't take the index load config from property store.
new 35bbb23 THIRDEYE-401 Flesh out info button docs on dashboard
new d330f0a THIRDEYE-402 Removing references to specific data sets
new cd38b84 Adding license header
new 564122f Fixed chaos monkey integration test
new 63b4763 Quickstart changes
new 22dad82 Simple quickstart using pinot-admin
new 50b10b1 making sure that we skip retention strategy if time column is null
new f359610 renaming KafkaTestUtils to KafkaStarterUtils and moving it to src/main from src/test
new 3137ea2 adding a pre-shutdown hook for graceful shutown of server and pro active routing table updates =
new 003fe53 fixing tests
new 2401770 adding instance config change listener on helixbrokerstater
new 96dde84 Adds warning alert when only one metric is selected and funnel metric is up
new ba34619 Cleans up funnel code. Moves checker into funnel render function. Makes funnels position relative
new 59ba73a Only render funnel if multiple metrics exist
new 36e3307 Changes warning text for funnel when only one metric is selected
new f662b9a Fixing the wrong method call for get offline table config
new 5f98d3e Fixed arguments in AddSchemaCommand
new d798729 Hides split button on overlay
new 1a46b31 THIRDEYE-283 A user should be able to specify relative times in URI
new a23ac74 THIRDEYE-398 Add optional "format" attribute to time configuration column
new 9572730 THIRDEYE-406 : startree_bootstrap_phase2 fails when trying to create metadata for empty files
new a266baf Made switches optional for controller/broker/server starting
new ab6259b Made switches optional for controller/broker/server starting
new dec0ed4 quickstart working with query examples after startup and pretty printing
new 1d4a3b1 Add support for multivalues in KafkaJSONMessageDecoder
new 94fa466 fixing realtime segment generation logic, new realtime segments were not being added in the property store by PinotRealtimeSegmentsManager
new 80dbdce Always make time granularity to DAYS if it's smaller than HOURS to avoid huge memory consumptions.
new 51c6b35 Adding Kafka-related commands
new 8e56be5 Clean up for quick start
new 8ec4126 making sure that we convert null to default values values in case key is present but value is null, also MV empty arrays needs to become array of size 1 with default null value
new 831a271 fixing typos and kafka topic name
new 8845715 removing acls folder
new 0ad5311 Updating README
new 7a245ea adding LICENSE
new 8786511 removing empty test file
new b565c3d Adding Apache 2.0 License and NOTICE
new 93e0316 Comment typo in last SQL example.
new 8a2c725 Merge pull request #2 from kdemarest/master
new 9c1368e THIRDEYE-294 A user should be able to define custom dashboards
new ea8289f Fixed some exception logging
new 425cf00 Merge: ea8289f 8a2c725
new b760dc0 fixing IllegalStateException in quickstart offline and realtime
new 44a119f Merge pull request #8 from dhaval2025/master
new 997ea5c adding link to discussion group.
new 22d30be Added Gitter badge
new 29a9480 Merge pull request #9 from gitter-badger/gitter-badge
new 9b1b0cc THIRDEYE-362 : Expire segments of finer granularity once coarser granularity segments available
new 0de9128 THIRDEYE-362 : Expire segments of finer granularity once coarser granularity segments available
new e77dc5a THIRDEYE-362 : Expire segments of finer granularity once coarser granularity segments available
new c62d3e9 THIRDEYE-362 : Expire segments of finer granularity once coarser granularity segments available
new 5410bb1 THIRDEYE-362 : Expire segments of finer granularity once coarser granularity segments available
new faddb73 Merge: 5410bb1 29a9480
new 1446923 Bug fixing: StatsCollector will throw exception if incoming data type is not the same.
new 14d1eed Remove dependency for old hadoop-core and clean up hadoop dependency to hadoop-client
new a6d4d5f Fixing log4j appender, and exception due to missing time column
new 5185f7a creating inverted index when trying to load a column where inverted index is specified in loading configs but the index file does not exist
new 92f4551 Fix for issue where quick-start was required to be run from a particular directory.
new 01d6af9 Fix for issue where quick-start was required to be run from a particular directory.
new 1477aa2 THIRDEYE-408 : Wait phase should wait until input file is completely copied
new 548de82 THIRDEYE-409 : Metadata should use aggregation granularity, not input granularity
new d547298 PINOT-1675: Making AddTenant/Table/Schema commands just do a dry run by default, and actually execute when -exec option specified
new 2434545 PINOT-1675 contd: Printing -exec option when running
new 59cd26a Adding retry policies
new 0122c1c THIRDEYE-410 : Remove dependency thirdeye-visualization from thirdeye-server
new 25aba5a Made some constants private static final, added some missing logging
new 3494074 Minor client logging improvements, removed extraneous line in class javadoc comment, fixed some issues, minor code cleanup
new 77ec5a2 Added controllerHost option in CRUD commands
new 794578c THIRDEYE-409 : Metadata should use aggregation granularity, not input granularity
new 72a3b62 Fixing realtime retention NPE
new c875cc0 Fixing localhost issue in pinot-admin
new 319e1aa Merge pull request #17 from fx19880617/master
new 1166c84 Merge: c875cc0 319e1aa
new e63051c [PINOT-1693] Adding additional log statements to help debug latency spike in broker
new 5057423 [PINOT-1696]: Fix for PostQueryCommand in quickstart-offline test
new d7cec1a adding some log statements during inverted index creation
new ff393b7 THIRDEYE-182 : Use dimension distribution for rollup function
new c26f5ef THRIDEYE-182 : Use dimension distribution for rollup function
new f8954ac [PINOT-1701]: Configuring separate appender for pinot-admin.
new 26b4c61 [PINOT-1708]: Adding command to delete Pinot cluster
new 0559a8a Centers primary header text. Fixes issue with column coloring on new metrics
new f5f495b Formats timestamp in Intraday Metrics, adds table striping
new f0be3a5 Creates function to get local timezone or timezone set in hash, updates code to use said function
new 232618e PINOT-1715: Fixing pinot broker initialization parameters - Connection Thread pool configuration
new bd881b9 THIRDEYE-417 : Print segment name when StarTreeManager loads segment
new 224f48c THRIDEYE-417 : Print segment name when StarTreeManager loads segment
new b9ad13f THIRDEYE-97 : Generate stats for each job in the hadoop pipeline
new a27fc12 THIRDEYE-424 : StarTreeGeneartion phase leaves open file handles for each leaf
new 50fe04a THIRDEYE-424 : StarTreeGeneartion phase leaves open file handles for each leaf
new 3d9b393 THIRDEYE-403 Drill-down on RATIO is broken
new b19ba11 THIRDEYE-419 : Handle overflow in StarTreeManager file system watch
new 9938e7d THIRDEYE-419 : Handle overflow in StarTreeManager file system watch
new a0c2eaa THIRDEYE-419 : Handle overflow in StarTreeManager file system watch
new 911afea THIRDEYE-419 : Handle overflow in StarTreeManager file system watch
new 90075ce THIRDEYE-432 Combine heat map implementations
new 86c65ba THIRDEYE-431 Remove "split"; all metrics on own charts
new 8540a0a THIRDEYE-429 Time series hover / click is broken
new f54e3f1 THIRDEYE-428 Heat maps should show numbers to two decimal points
new ec6a937 THIRDEYE-426 Remove rollUp from ThirdEyeKafkaConsumer
new 8165caf THIRDEYE-425 Integrate custom dashboards with interactive UI
new 35251d5 THIRDEYE-399 Alias "?" to "OTHER" on dashboard
new 937be42 Merge branch 'te432'
new c614b09 THIRDEYE-419 : Handle overflow in StarTreeManager file system watch
new 1608829 THIRDEYE-434 : TestStarTreeManagerImpl failure
new 17ec935 THIRDEYE-435 Funnels do not appear on same horizontal line
new 78aa23e THIREYE-436 : Archive contains unclosed entries in StartreeGeneration
new 88532e1 THIRDEYE-441 : Split server push to package / push jobs
new c17fa2a THIRDEYE-441 : Split server push to package / push jobs
new fe812b8 THIRDEYE-427 A user should be able to define an alias for a metric / dimension
new 1200e5a THIRDEYE-444 Dashboard resets input to hours if days selected
new 32a674c THIRDEYE-427 A user should be able to define an alias for a metric / dimension
new 6796369 Put equalsverifier into the test scope
new 01a8ebf Fixing the offline/realtimeTableConfig used in AbstractTableConfig.
new ab02225 Added Swagger documentation of the REST API, fixed some Restlets having typos in their class names
new d574c6e THIRDEYE-448 Cannot create custom dashboard if collection dir does not exist
new a0cae5d Merge pull request #28 from fx19880617/master
new d4f52ae Merge: d574c6e a0cae5d
new 75798e4 Fixing compile error on eclipse
new 35a4e6f THIRDEYE-472 Enable nested Avro parsing in thirdeye-bootstrap
new 36d07e1 THIRDEYE-463 : thirdeye-server does not release file handles, so disk space runs out
new 2fd4dca Adding postfix config segment.name.postfix to hadoop segment creation job configs.
new e66c953 THIRDEYE-434 : TestStarTreeManagerImpl failure
new 727202b THRIDEYE-434 : TestStarTreeManagerImpl failure
new a1f0073 THIRDEYE-434 : TestStarTreeManagerImpl failure
new c42c633 THIRDEYE-471 Change date in intra-day view from YYYY-dd-MM to YYYY-MM-dd
new 8ddb291 THIRDEYE-450 Add annotated heat map cell to heat map help modal
new ac467f8 THIRDEYE-469 Add total and current and baseline ratios w.r.t. total to heat map tool tip
new a26c9ad [PINOT-1802]: Standalone perf benchmark executable
new 0b33742 Avoid throwing NPE
new 0d5e219 PINOT:1797 Handling null values for string column. Setting them to default value 'null'
new 2b9a8b5 THIRDEYE-454 Side-nav input should set to last time bucket with data, not first without data
new 9b218d9 THIRDEYE-454 Side-nav input should set to last time bucket with data, not first without data
new dfd78e8 DIsabling snapshot due to slow response time on abook
new 8333ab9 THIRDEYE-469 Add total and current and baseline ratios w.r.t. total to heat map tool tip
new a230d1f THIRDEYE-491 : Create and expose dataTimeLag metric in thirdeye-server
new e159ef8 THIRDEYE-491 : Create and expose dataTimeLag metric in thirdeye-server
new c56cc15 THRIDEYE-491 : Create and expose dataTimeLag metric in thirdeye-server
new d173e7f THIRDEYE-491 : Create and expose dataTimeLag metric in thirdeye-server
new 6b5e838 [PINOT-1815]: Added fix for typo in Readme.md to correct the query console URL.
new c76bad5 THIRDEYE-414 A user should be able to submit feedback from the dashboard
new d1f76b2 THIRDEYE-414 A user should be able to submit feedback from the dashboard
new e223373 PINOT-1821 Fixed IAE in FixedBitSkipListSCMVWriter.<init> caused by integer overflow
new 5b36e76 Added details to IllegalArgumentExceptions thrown
new 34ca1b8 PINOT-1827: Correct schema related NullPointerExceptions
new bab1b9f [PINOT-1799]: Take config file as input for starting controller/broker/server in pinot-admin
new aa1db3e Fixed exception message that caused a NPE when logging an exception
new 6982e89 Fixed enum constant lookup due to the metadata containing the lowercase version of the enum constants
new 59c129b Added preconditions for custom bit set construction
new f2f7ad0 Squashed commit of the following:
new ea37ac3 Removing references in report-configs
new c901443 Revert "Removing references in report-configs"
new fa3393b Revert "Squashed commit of the following:"
new af7527f THIRDEYE-228 Adds support for disjunctive queries
new 2ceb417 Merging thirdeye-anomaly and thirdeye-reporting into master
new 71a4a6d THIRDEYE-414 A user should be able to submit feedback from the dashboard
new 1b7cec8 THIRDEYE-492 Make min / max time on dashboard respond to time zone selection
new 85e453b THIRDEYE-492 thirdeye-dashboard/src/main/resources/assets/js/thirdeye.sidenav.js
new b3399b6 THIRDEYE-492 Make min / max time on dashboard respond to time zone selection
new e95d155 Fixed broken unit tests on some non-US locales
new c1804ea Changed CustomBitSet to have long offsets for bits instead of integer offsets
new 5338761 Use long bit offsets when reading/writing fixed bit width files
new 674d80c Fixed unit test using incorrect REST API endpoints
new b3bfed3 Clarified error messages when obtaining a default null value, fixed metric fields falling through to the dimension types
new b4b15e2 Fixed TestSchema unit test
new f5c9953 Adding SqlUtils#getSql(..., Map<String, String> dimValues) back
new 0e4d8ae THRIDEYE-97 : Generate stats for each job in the hadoop pipeline
new bb404a6 THIRDEYE-97 : Generate stats for each job in the hadoop pipeline
new b68f1a0 THIRDEYE-522 : Fix anomaly ranking after schema change
new 48ad970 PINOT-1838 Specify dependency version in parent pom instead of child pom, to fix generated ivies used in internal MP build
new 4a131e5 [PINOT-1670] Adding controller api's for enable/disable/drop of segment/instance/table/tenant
new 41f1c46 Generating SNAPSHOT jar for thirdeye-reporting
new a659cdb THIRDEYE-532 : Add function table name as a column in anomaly result table. Added additional comments and some cleanup for exception handling.
new 6cf7efb THIRDEYE-484 : Create MP wrapper for thirdeye-reporting
new 7fb0ec4 [PINOT-1419]: Handle concurrent ideal state by multiple controllers
new fb9be7a THIRDEYE-414 A user should be able to submit feedback from the dashboard
new 713ad06 Merge branch 'THIRDEYE-414-feedback-btn'
new a8bcd0b [PINOT-1829]: Changing JAVA version to 1.8 in the pom file.
new 0bc52d6 Fixed broken ControllerInstanceTest
new 2769e22 Fixed broken Javadoc
new f77ea69 PINOT-1846 Refactored TimeUnit.valueOf
new e95cf11 Multiple changes to thirdeye-reporting: Adding config for email templates path Adding config for anomaly tables database Adding logging Correcting test triggers
new fcdbfe4 Multiple changes to thirdeye-reporting: Adding config for email templates path Adding config for anomaly tables database Adding logging Correcting test triggers
new d5518f3 Multiple changes to thirdeye-reporting: Adding config for email templates path Adding config for anomaly tables database Adding logging Correcting test triggers
new 6eb85e8 Made constant final
new 702c763 Refactored anonymous class into inner class
new b0ed4de Multiple changes to thirdeye-reporting: Adding email-templates via config
new 24a73f4 Greatly improved memory usage of segment creation, removed creation and retention of large amounts of boxed primitives, removed memory inefficient cache, removed inefficient binary searches, added missing assertion for unsupported data type, added logging of pre-index creation per-column statistics, refactored forward index creator interface
new e913a6e Fixed issues while updating tables
new 08ad0a7 add streamlib dependency
new 87c5695 add HyperLogLog estimation to distinct count, the function is called 'distinctcounthll'
new 7b8d62c various tests for 'distinctcounthll' aggregation function
new 64d9af1 refactor code, fix some code review comments
new d852792 t-digest and unit/sentinel tests
new 21cc3bf OfflineClusterIntegrationTest for quantile
new 93fced7 multi-value column support for distinct count hll
new 140cd65 add licence headers, add various quantiles, add more comments for TDigest, and add double value support for quantile methods
new b64c9f7 fix precision that sometimes causes tests to fail
new 9e612dd change quantile function name to percentile
new 9fdafbe missing license headers
new acda38a Muliple changes to thirdeye-anomaly: Add previously reported anomalies as feedback to the anomaly detection function Removing exploration depth limitation in thirdeye-anomaly. Move anomaly reporting UI elements to thirdeye-reporting. Fixed wrong count issue in anomaly report generation.
new 7549f00 Misc. cleanup: Cleanup swagger annotation, not use hardcoded '/tmp'
new 9deb7fd Misc. cleanup: Cleanup swagger annotation, not use hardcoded '/tmp'
new e14afb8 Multiple changes to thirdeye-reporting
new b4e0945 Multiple changes to thirdeye-reporting
new 14e888d Pull stream library version into parent pom, allow overriding helix version through the command line
new ffab151 Multiple changes to thirdeye-reporting
new 665508d [PINOT-1887] AvroRecordReader should use FieldSpec.getDefaultNullValue() instead of its own
new 1e67b7a [PINOT-1886] Do not cast boolean to a string when indexing
new 74774fd THIRDEYE-523 thirdeye-client module
new 4423973 THIRDEYE-548 : Fix MYSQL underlying timezone issue for anomaly-table
new 0219217 PINOT-1894: Query console path requires specifying index.html
new b10b5d4 THIRDEYE-525 : NPE in SlideshowViewThirdeyeDaily_aggregation
new 44db870 THIRDEYE-525 : NPE in SlideshowViewThirdeyeDaily_aggregation
new 2e9468d THIRDEYE-525 : NPE in SlideshowViewThirdeyeDaily_aggregation
new b2e5538 THIRDEYE-518 Make clicking intra-day time more intuitive
new 0659fcc THIRDEYE-514 A user should be able to turn off dimension heat map filtering
new c0fd5ce THIRDEYE-514 A user should be able to turn off dimension heat map filtering
new 983c6ae THIRDEYE-514 A user should be able to turn off dimension heat map filtering
new 470ab2f Removed unused mmap'ing of dictionaries during segment creation.
new d2414ec PINOT-1876: Remove PinotHelixAdmin class
new 24f8f7e Fix javadoc compile error
new df6bd68 comment out the assertions that should only be opened on debugging
new 6878126 remove debug code
new 49724d4 THIRDEYE-534 : Integrate default anomaly detection algorithm
new f0f420e THIRDEYE-530 : Index Metadata breaks thirdeye-query if input file is empty
new 99a6b80 THIRDEYE-530 : Index Metadata breaks thirdeye-query if input file is empty
new c25ce83 THIRDEYE-537 Create R client for use by new anomaly detection algorithm
new 89368b6 [THIRDEYE-498] Space efficient storage for leaf nodes
new 17b4bcd THIRDEYE-537 Create R client for use by new anomaly detection algorithm
new a231592 THIRDEYE-551 : Refactor thirdeye-anomaly to use thirdeye-client and remove dependencies to thirdeye-dashboard
new b938dab [PINOT-1905] Fix fatal when reading null for multi-valued column
new 2e3690a THIRDEYE-437 : Intra-day responds to aggregation granularity better
new f994d54 THIRDEYE-437 : Intra-day responds to aggregation granularity better
new 68b156c THIRDEYE-Reporting: Timezone conversion, thirdeye uri alignment, ReportsResource broken
new 182ac8b THIRDEYE-Reporting: Timezone conversion, thirdeye uri alignment, ReportsResource broken
new 5e39e30 THIRDEYE-559 Custom dashboard breaks when there are not 5 series to display
new 287d1a6 THIRDEYE-558 Heat map filter / info buttons are not clickable
new e89b10e THIRDEYE-552 : Hadoop poll phase should wait until data is readable
new d58ca7e THIRDEYE-552 : Hadoop poll phase should wait until data is readable
new 9802b2c More cleanup on Controller api's.
new 89cde81 [PINOT-1872] Fix casting a boolean into string
new 5fa2e39 Dashboard dimension groups, intra-day ordering
new 73bbae5 Fix for dimension grouping
new 1657c4f Show heat map cells that are 1% volume or more
new 3f10743 Fixed incorrect end time in metadata
new bd1485a Fixes multiple derived metrics on dashboard bug
new 87c5365 Fixes bug in thirdeye-query
new 3733f78 Clean up for ABI review
new 39f0721 [PINOT-1931]: Support for clustered mode ZK in pinot-admin
new 5f42501 Write git revision used to build the artifact into the JAR manifest, log the git revision of the code on startup
new 4096a3a THIRDEYE-580 : Ads reports being generated with 0 numbers
new a455f4c THIRDEYE-580 : Ads reports being generated with 0 numbers
new 4e053d0 multiple changes to thirdeye-dashboard and thirdeye-anomaly: THIRDEYE-533 : Integrate anomaly reporting with dashboard THIRDEYE-567 : Transition thirdeye-anomaly from detectionLag to polling thirdeye-server. THIRDEYE-568 : Parallelize exploration in the driver (improved thread management) THIRDEYE-569 : Fix null pointer exception in fanomaly when omitting anomalies from training data. Added FlowControlledDefaultThirdEyeClient to thirdeye-client. Added dimensi [...]
new 5551752 [PINOT-1940]: Added pinot-admin command to convert avro schema to pinot schema.
new 979d45e changes to thirdeye-anomaly and thirdeye-dashboard: add automatic end-to-end setup walkthrough to thirdeye-anomaly fix bug in dashboard where the series-dimension view is showing anomalies change kalman function to be more robust to holes in data
new 0c1727d Enhanced AvroSchemaToPinotSchema to also take avro data file as input.
new 0422ef6 THIRDEYE-581 Metrics that are reserved words (e.g. 'open', 'close') cause the system to break
new ff729c3 Handle command line parse exceptions separately from all other in pinot-admin.
new 0ac3587 loose precision threshold to make tests passed more easily
new dfbdea4 fix random seeds to make sure tests won't break
new eba1d7a fix minor numeric bugs in tests
new 5691e2b only fix random seeds in tests
new fe43153 Fixes off-by-one error in thirdeye-dashboard
new 0a27123 Changed thirdeye-anomaly back to fat jar, removed slf4j impl
new d7cd950 Revert "Fixes off-by-one error in thirdeye-dashboard"
new 30e87af Fixed metric quote on dashboard
new a946d8d THIRDEYE-589 For the latest day, current / baseline off by one
new eceba1d PINOT-1955 Fixed tenants appearing as tenantName_O instead of tenantName
new 6663ce4 PINOT-1977 Fix broken logging in RetentionManager
new ef4e774 Separate data access and exploration logic in AnomalyDetectionTask. Rename mode enums to conform to code standards. Add stl decomposition in using a R.
new d10fe42 THIRDEYE-528 : thirdeye-reporting should not depend on other deployable modules
new df46527 THIRDEYE-349 : Implement end-to-end integration test for thirdeye
new 819e15d THIRDEYE-349 : Implement end-to-end integration test for thirdeye
new a82af16 THIRDEYE-349 : Implement end-to-end integration test for thirdeye
new 2c6dc45 THIRDEYE-349 : Implement end-to-end integration test for thirdeye
new ded1446 THIRDEYE-349 : Implement end-to-end integration test for thirdeye
new ee9dd17 [PINOT-1909]: Added feature to be able to delete a schema
new a61aa70 THIRDEYE-590 : Cleaned up R and Java code for removing seasonality, and added twindow parameter search.
new a710f38 Disabling generation of aggregation / dimension stats.
new 3a533fc [PINOT-1983]: Fix for issue where query retuns 'bound must be positive'
new 30722d3 [PINOT-1978]: Fix for NPE in Retention Manager
new 60302ae Changes to thirdeye-anomaly: clean up kalman (formerly fanomaly) package add framework for scan statistics function add command line arguments
new 71d040c THIRDEYE-592 : thirdeye-reporting should check for missing data segments
new b5f2526 THIRDEYE-592 : thirdeye-reporting should check for missing data segments
new d3a936e THIRDEYE-592 : thirdeye-reporting should check for missing data segments
new 873481e scan statistics version 0
new 1061ba9 new trace context
new a299329 add base operator to handle default trace and loggings
new 3567d82 bug fixes
new b2e68b2 add trace test in QueriesSentinelTest
new aae68a0 more bug fixes
new 26f06a2 activate trace context in server jobs, enable trace tag to be passed from client to broker to server
new 032c807 add traceId for each trace to show parent-child relationship
new 5b8c3a9 fix json return result issue
new a6fa3d1 adding license headers
new f3320a0 fix bug when request is null in trace
new 482433e use new digest lib for percentile estimation with speed improved
new 4a291c0 always enable trace as a temporary solution
new 5786da1 fix bug that causes the real reason of query execution exception not pass to the client side
new f77f668 remove unnecessary imports and reformat the QuantileDigest class
new c3d8918 disable trace temporarily
new e6aee99 add trace enable function in controller now the client can choose to enable trace or not
new 7d044f5 minor fixes
new 120d5b7 minor fix
new 65a1651 Test commit
new aef7db8 Removing test file
new 55faf77 PINOT-1978 NPE fix when an offline table has no validation config
new 5a502a2 PINOT-1828: Set correct return status for controller-api's
new c1f8f09 Initial implementation of StarTree index in Pinot.
new e1e277f Fixed TestSchema (changed default max leaf records)
new f606503 make sure to compile under java 1.7
new 5c61d38 Final cleanup
new 908b5e1 changes to thirdeye-anomaly: integration of scan statistics + performance enhancements disable twindow parameter search in stl-wrapper add support for overlapping monitoring windows add support for simulated online mode with arbitrary timeranges THIRDEYE-593 : Add thirdeye-anomaly function management server.
new a365190 PINOT-1978 NPE fix for retention
new 936cbda PINOT-1985 Unit test fix
new 09e01f2 PINOT-2012 Fixing IAE due to integer overflow when loading large segments
new b802c9e Removing internal reference
new d636ee9 Move dependency into parent pom
new 25b639a Adding new pinot-admin command to show cluster info
new 79ffd6a Fixed whitespace in license header to fix the build's license header check
new 7793edb Removed logging spam, added logging to debug retention issue
new 4e2e130 [PINOT-2023]: Split log4j properties for pinot-admin/controller/broker/server
new b72609e Improved memory usage of pre-indexing stats collection by a factor of ~1.63
new 21cd31b PINOT-2019 Fix unit test due to missing file in path
new 68876ff Removing file that shouldn't have been committed
new 5fce8f8 Implementing feedback changes for reports from Ads
new 91b05b9 THIRDEYE-592 : thirdeye-reporting should check for missing data segments
new e4e7ec7 THIRDEYE-592 : thirdeye-reporting should check for missing data segments
new 7dd4ad9 THIRDEYE-592 : thirdeye-reporting should check for missing data segments
new 166b951 THIRDEYE-592 : thirdeye-reporting should check for missing data segments
new 953114b changes to thirdeye-anomaly: add java-stl with support for incomplete seasons, periodic param, bugfixes refactor AnomalyHistory interface refactor AnomalyDetectionTask inheritance add prototype for running anomaly detection function in management dashboard add default configuration for scan statistics fix timestamp indexing bug in scan statistics add bootstrap option in scan statistics
new 594ca3c PINOT-2020 Log the fact that a table's retention is not configured instead of throwing a NumberFormatException
new 10bb930 Fix for broken unit test
new 08855e2 multiple changes to thirdeye-anomaly: separate anomaly detection driver from task. add minimum monitoring window method to AnomalyDetectionFunction remove overlap monitoring window command line option order analysis of dimension combinations by descending contribution expose stl trend bandwidth as property in scan statistics function add option to report only the endpoints of anomalous intervals add partitioning schemes for driver
new 6309f5a PINOT-2038 Re-use generic record when reading avro files
new df08fb9 Improved StarTree construction memory usage
new d150ea5 THIRDEYE-598 Adding data table and replacing heatmaps with treemaps
new 1c7c535 added per column inverted index create config during segment creation
new 7b1eaf1 adding custom funnels feature for growth demo, there are a couple of TODOs on me, I will get to them after the demo
new 48f8c7e Making funnel view optional
new 816a57a PINOT-2037 Make new antlr4 grammar the default
new 356b408 changes to thirdeye-anomaly: allow detection on multiple collections in thirdeye-anomaly at once. cleaned up SQL exception handling set more reasonable default max scan interval
new 6313b75 adding alises to metrics in funnels
new ea72b62 THIRDEYE-596 : Reporting email's ftl styles should be injected using java resource
new d0acc64 THIRDEYE-596 : Reporting email's ftl styles should be injected using java resource
new 718cbda THIRDEYE-596 : Reporting email's ftl styles should be injected using java resource
new 5308cd2 adding display toggle functionality with collapser element to areas and metric, dimension sections; rename containers on heatmap.ftl to follow original naming structure; small layout improvements
new eb1d458 Fix stl-java's handling of outer-loop robustness weights.
new 5d49dbb make trace results in order, make trace context more robust on error
new 8c10550 make trace info return even exception happens
new e3cd680 [PINOT-2072]: Fix NPE in ShowClusterInfo command, due to unassigned segments
new db88788 [PINOT-2074] Adding explicit 'delete' api for deleteing segments
new b68c486 removed unnecessary collapser from funnel-heatmap.ftl, fix for losing current dimension query on sidenav submit
new 5ab5c23 removed unnecessary collapser from funnel-heatmap.ftl, fix for losing current dimension query on sidenav submit
new e73b3c7 uniform color coding applied to datatable, add plus symbol to table title delta percent and value
new 6b9d599 PINOT-2032 Ignore segments in error state when waiting for external view to reflect ideal state changes on segment refresh
new 4e7013d Added flag to ignore segments in error state when waiting for external view to reflect changes
new da135c9 Track direct byte buffer allocations and log allocations/deallocations
new 5adcf26 Always use MmapUtils.unloadByteBuffer to release buffers, as to avoid direct memory leaks
new bc2ce31 removed unnecessary collapser from funnel-heatmap.ftl, fix for losing current dimension query on sidenav submit
new deeffd9 Fixed close (commit will be squashed)
new b9805af treemap cell positive value background color changed to blue, current query stays in the fold
new 47fc09b Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
new afe41ac Merge branch 'dashboard_current_query_styling'
new c6d0528 Fixing funnel parameter append logic
new 3b7211b THIRDEYE-599 : Add support for ratio of metrics to thirdeye-reporting
new 3401c16 THIRDEYE-599 : Add support for ratio of metrics to thirdeye-reporting
new 7a2ae2e THIRDEYE-599 : Add support for ratio of metrics to thirdeye-reporting
new 686c4af THIRDEYE-599 : Add support for ratio of metrics to thirdeye-reporting
new 83fa8f5 iTHIRDEYE-599 : Add support for ratio of metrics to thirdeye-reporting
new fc1d646 THIRDEYE-599 : Add support for ratio of metrics to thirdeye-reporting
new 3b51f05 THIRDEYE-599 : Add support for ratio of metrics to thirdeye-reporting
new 3b7f355 THIRDEYE-599 : Add support for ratio of metrics to thirdeye-reporting
new 9e9c63e THIRDEYE-599 : Add support for ratio of metrics to thirdeye-reporting
new 6994d15 adding unit test for stl java code
new 6e18a15 [PINOT-2094] Fix StarColumnListAst to have a modifieble list instead of an unmodifiable one.
new 5ed09b5 Fixed RealtimeFileBasedReaderTest that was broken
new c57460e funnel_heatmap header form layout
new eaa242e create and style funnel thumbnails' behavior, adding funnel header form behavior (validation, submit, value prepopulation) except for cummulative btn
new c693764 fix stl code
new 1c7882c Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
new 07b1414 dashboard minor styling changes; display funnel table above the fold, Change Funnel's title to Dahsboard's
new e403f8b dashboard minor changes; display funnel table above the fold, Change Funnel's title to Dashboard's
new d1611b1 adding funnelheatmap.js
new 93e656a Merge branch 'funnel_heatmap_dashboard'
new a42b79e [PINOT-2131] Fixed a bunch of memory allocation issues that caused too much GC
new 6434d8b thirdeye-detector service to run / manage anomaly detection functions
new ce11d94 Don't ignore PQL exceptions in integration tests
new 79be76d THIRDEYE-604 : Create compaction workflow for bootstrap
new 29b2d19 [PINOT 2094] Fix MmapUtils to use thread-safe map
new 151c4b3 [PINOT 2095] Make TimeRetentionStrategy constructors consistent
new d5205cd [PINOT 2095] Use valid time retention values in integration tests
new e8e693e thirdeye-detector bugfixes
new 1b3e8f8 Filter anomalies by the dimension combination
new b7831ec thirdeye-detector bugfixes
new 22e7430 Javadoc fixes
new 2510c06 Track mmapped files, fix JVM crash when double-freeing a buffer
new 197301a Add upload/refresh/delete integration test
new 1b3515b Add performance benchmark harness and dependency on JMH
new 9519d2f Optimize readInt for values < 16 bits by processing an entire 32-bit int in a single chunk, remove Ints.checkedCasts
new edc233d Use mutable longs for aggregations
new 5385cdd Add special cases to handle specific group by scenarios (1 column with a small dictionary, fast aggregation when the entire block matches all rows)
new 99a1a41 Logic to enable / disable anomaly detection algorithms
new 329a7de Adding missing license headers
new 42f81d4 More debugging utilities for thirdeye-detector
new 7eee10b Remove file that was added by git-review dcommit (see TOOLS-72585)
new fb351ca Multiple changes to StarTree index
new 2df6aa5 Fix ClassCastException in IntermediateResultsBlock
new e703aa7 [PINOT-2095] Fix a memory leak in RealtimSegmentImpl
new 5de1983 THIRDEYE-605 : Create thirdeye transform job in bootstrap
new 871cb7d Bugfixes in thirdeye-detector
new 075d88a scan unit test
new bcf6dc5 Fix unit tests that depend on long type for count aggregations
new 5c7923c Bugfixes in thirdeye-detector
new 533bb75 Retry deletion if it fails, using exponential backoff
new 97ca474 fixing the query console table list
new 242da28 fixing the query increment/decrement refrerence counters
new 19957e9 fixing the query increment/decrement refrerence counters
new 10dccc2 adding a new method returnSegmentReader in the offlineTableDataManager interface
new b67cf70 minor refactor
new efd24d2 minor refactor
new 7735c7a [PINOT-2161] Support for applications to be able to set their own defaultNull values.
new 29d0ec0 [PINOT-2181] Fix segment enable/disble/drop runtime
new 1207139 THIRDEYE-606 : Add ability to set custom job config in transform job
new 750f0be THIRDEYE-607: Adding reduce phase to transform job to merge map outputs and tackle NSQuota issue
new 96d08cb THIRDEYE-606 : Add ability to set custom job config in transform job Added source to transform job so that we can do unions
new 601c37b THIRDEYE-606 : Add ability to set custom job config in transform job Added ability to read source, so we can read multiple sources and perform union
new b3d962b Switch MmapUtils log level to debug
new 09837f1 Disable PQL2 integration test, as it fails in a limited set of conditions where the old query parser would incorrectly merge query predicates due to using lexicographic comparison instead of integer comparison (eg. WHERE TotalAddGTime BETWEEN '4' AND '103' AND TotalAddGTime BETWEEN '24' AND '30' or WHERE SecurityDelay BETWEEN '10' AND '15' AND SecurityDelay < '3')
new a4d378a Disable PQL2 integration tests for inherited methods
new 3781c47 [PINOT-2161] Fixing the call to setDefaultNullValue to take an object instead of string. Also, added a couple of cases for setting defaultNullValue via schema.
new 05a846c Added line brushing
new 6896439 scan unit test
new 5e04c89 Merge: 6896439 05a846c
new 4ee8d10 broker debug endpoint to spit out the routing table
new 1722ec9 modifying the Start* and Stop admin commands to allow starting/stopping multiple servers,brokers and controllers
new 55fb0ba modifyging QuickstartRunner to allow starting multiple instances of server,broker and controllers
new b2a483b changes to Quickstart and RealtimeQuickstart w.r.t the new QuickstartRunner
new 9d984a0 adding a hybrid quick start
new f1fc531 Bugfixes in thirdeye-detector
new 8fcd709 PINOT-2197 Increase default timeout for refresh to 120s, fix issue where refresh external view timeout was always 10s even when configured otherwise, remove server-side HTTP timeout.
new 92e3af9 fix one error in the unit test
new 24ba3cd Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
new 06e809f [PINOT-2234] Remove watches created by broker and server for the nodes under PROPERTYSTORE
new 042696a Adds GroupBy support for star tree index
new 59ef978 making realtime segments numdocs and time threshold configurable, default values are set to 5 million and 1 hour
new aeb0d6d THIRDEYE-607 : Add ability to add custom config to join phase job
new 42ae0bc THIRDEYE-607 : Add ability to add custom config to join phase job
new 30a53da PINOT-2205 Add REST API endpoint for pinot version
new 4a2ec21 [PINOT-2243]: Adding shutdown api HelixBrokerStarter.
new 4d83e1b [PINOT-2243]: Adding shutdown api HelixBrokerStarter.
new 637688b Bugfixes in thirdeye-detector
new e5e93a4 Fix version endpoint returning a Content-Type of application/xml instead of application/json when client specifies Accept-Content-Type: application/xml
new ab00b57 Fixed broker debug endpoint returning empty results, made endpoint set Content-Type of application/json, made output actual JSON instead of a JS array, removed unused imports
new c13fd5b Replace tabs in POMs by two spaces
new e0adb2d Fix misindented dependency
new fcdd16e fix json typo
new 78138db PINOT-2246 Set controller API timeout to one hour
new 26bf0a0 PINOT-2246 Set file upload timeout to one hour
new cfebf8e new dashboard views funnel overview, heatmap, timeseries
new 6eb7f5d time-input form as config-for partial, css fixes on timeseries, adding funnel loader
new ff46132 adding conditional elements to config form, refactor removing metric view
new daaac78 fixes on config form behavior,adding configform.js,fix for timeseries styling, adding timezone to timeseries
new faa4e33 baseline inputfield present on heatmap and timeseries view
new 32f405c funnel heatmap click event listener
new e75e0dd adding the configs JSON endpoint
new e414e54 funnel heatmap cell click eventlistener, default view tabular
new 84881a4 funnel heatmap header eventlistener
new 73493bd fix for funnel heatmap header eventhandler
new 3b2a14b config form changes:naming changes, dropdown moving average, table styling changes
new f2378aa display selected timezone on funnel heatmap, ajax path based on collection, styling and naming changes
new 8e7ba34 Refactor FunnelHeatMapView to use FunnelHeatMapRow. Update funnel-heatmap.ftl to display tooltip with baseline + current values.
new 9bdbbb1 Funnel heatmap always has a default WoW overlay and cannot be changed,on the overview it has to be seleceable on the details tab only. Funnelheatmap time column hover and click eventhandlers. Timeseries legend position changed. Time display 'hh:mm z' on Overview tables.
new d4a6103 Calculate cumulative rows for funnel-heatmap. Refactor funnel-heatmap table body to macro.
new 2bf6cef Cumulative rows for intra-day. Update logic for funnel to handle null case and properly comment out macros.
new 45a0e4b cumulative feature, tooltip layout consistency
new 67dab3b adding tooltip on the column names in the funnel view
new d1245e7 adding tooltip on the column names in the funnel view
new f24f5b1 Clean up time series legend labels (remove extra *)
new 1d1b9b2 Revert "Clean up time series legend labels (remove extra *)"
new 3b34a12 bugfix in funnel-heatmap for missing ratio values.
new ff18768 Clean up time series legend labels (remove extra *)
new 8ca0cbb Adding a default view which simply shows all the metrics for this dataset
new 4b1e099 Pass currentUTC to funnel-heatmap and standardize time display between funnel-heatmap and metric intra-day tables.
new d991a36 Refactor (MetricTableRow + FunnelHeatMapRow)->MetricDataRow.java, extract common js function renderTimeCell to thirdeye.js
new e54c76d Include MetricDataRow.java
new 2ac15ca adding dimension-header
new 36eb2b8 Dimension query filter styling changes
new e76293d Update funnel heatmap to use input time
new 7f9577f Refactor and standardize MetricDataRow extraction from query result for funnel heatmap and metric intraday view.
new 8570b0b Refactor cumulative metric data computation to ViewUtils.
new 242746c display dimension query selector in config-form.ftl
new 42a88a1 Add default funnel for funnel heatmap view, based on collection metrics.
new 6f23a7c Initial server endpoint for all dimension values in collection with given time range.
new 79c2963 Dashboard changes to provide available dimension names in template.
new cb4281e Revert "Dashboard changes to provide available dimension names in template."
new e5b94ee Revert "Initial server endpoint for all dimension values in collection with given time range."
new d8a6e45 Provide dimension names in dashboard via querying server with group by (instead of server-side changes).
new a0a0a7a Fix dimension name display to be stored in all dimension views
new b4d3579 adding the option of selecting multiple dimensions, styling config form, funnel heatmap cell click eventhandler fix, default select all metrics and WoW moving average
new 3ef3c47 allowing to query multiple values per dimesion
new 6b9bfe4 minor layout changes on config form
new d23fe2b remove default moving average
new 45894a7 WoW default baseline
new 206c2be removing metric-intraday view,adding funnel heatmap details tab
new 8676698 removinghardcoded data from funnelheatmap, drawing click event data from the DOM
new f54ef07 fix on funnelheatmap cell
new 0098f04 Update FunnelsDataProvider to accept intraPeriod argument instead of hardcoding to 24hours/1day
new 1848adf Modified dimension retrieval to use threshold and limit number of returned results. Followups still pending (metric function, threshold value).
new 2cb57ce adding metric timeseries and small layout changes
new 69b2e7b [PINOT-2262] Set mmap handles to null after the file is unmapped.
new 5c11219 minor styling changes
new 0cecf46 convert UTC time to user selected timezone on funnel title
new ecd4af7 Hardcode day buckets to PDT on overview page
new b8db15e Remove Ratio subheader from overview funnel heatmap
new 45e08d6 [PINOT-2143] Added pinot-admin command to change the state of the table
new 1dca477 PINOT-2268 Fixed random routing table only having duplicate versions of the same routing table
new c393c8e fix for dahsboards without funnel config
new 42d5e69 [PINOT-2286]: Fixing NPE in inverted index creation.
new 43e5653 Fixed bug with Overview page when derived metrics are used.
new f39f127 [PINOT-2287]: Fix for inverted indices not being created at segment load
new af94bce Created DashboardConfigResource w/ endpoints for metrics, dimensions + aliases,values
new 210bc92 DashboardConfigResource: change endpoint to /dashboardConfig instead of /dashboard-config, fix path param for getDimensionValues
new 0cde036 Added TestDashboardConfigResource with tests for all methods except getDimensionValues (complexity)
new 3372640 Merge pull request #56 from gilmichlin/fix_json_typo
new dd3b058 Merge pull request #31 from fx19880617/master
new 56826f1 github-issue-61: Incorrect secondary ordering
new 4735cee Adding standalone tool to generate startree based on rollup output
new c4a0bb1 adding reset.css, clean css unused classes, adding breakdown.ftl, removing multi-time-series.ftl from dashboard
new 942db43 Use UTF-8 for broker responses
new 78f7b6e [PINOT-2290] Update to use commons-compress-1.9 commons-compress-1.9 fixes byte-array garbage generation problem that 1.4.1 and earlier versions had. This caused too much GC activity when loading a segment on the server (while serving queries), eventually leading to the server disconnecting from zookeeper.
new 6c3a369 Optimize dimension + metric view generation in DashboardResource. Update ftl templates to remove unnecessary dependencies on metricView (except for multi-time-series) and added new configs for dimension + dimension alias in DashboardView
new 24a4a17 Rename several unused views and java classes for later deletion. Remove MetricView object from DashboardView.java.
new 9e2d87c removed: derived metrics input field and logic, baselinedate inputfield and logic, adding Wo3W to Comparison selector, adding WoW to all the views, removed unused templates:intra-day,fufnnel, time-series
new 0ffeb05 Remove unused metric views.
new d1f6e0c adding metric selector to Contributors view populating metrics form the URI, restyle configform
new 6a7fa27 styling changes on config-form
new ec92aff [PINOT-2308]: Fix default value of tenant isolation
new bfcc56e PINOT-2047 Show segments active in Helix by default when listing segments for a table, don't show fileUploadTemp and schemasTemp in list of segments
new 1af35d5 adding time selector to configform, adding derived metrics
new 62c9381 Add Contributor dimension view and data provider as well as basic table templates
new 6aebc8a adding RATIO() metrics to URI on funnel heatmap events
new 0ba5f9e Bugfix for missing baseline data in contributor view
new 9459f85 Contributors: Reformat + add Total row
new c34d988 merge
new d360e6c handle ratio on funnel heatmap fix
new 7988ebe [PINOT-2118] Add extra HTTP header that indicates the controller host and version in all responses
new c1519a7 metric selection populated from uri in config form
new 0132574 PINOT-2318 Add server, broker and controller metrics that return whether Helix is connected to ZK or not, added a controller metric that returns whether this controller is the leader
new 0ffd7bc PINOT-2320 Generate NOT IN predicates in query generation
new 4f8ba98 styling contributors view, adding details/summary tabs, calculating color and translate to tz
new 3e2b07f styling changes on contributors table
new 7a401cc fit all 24hr on mac screen
new 51b6a3b Refactor ContributorDataProvider data structure. Include contribution threshold for dimension values to filter out insignificant values. Update total row to hide/show with cumulative button.
new b8fc8f3 Update ContributorDataProvider to roll up dimension values that are filtered out and add them to the OTHER category.
new 8935953 THIRDEYE-497 : GROUP BY-based top k rollup
new 82f9e70 THIRDEYE-497 : GROUP BY-based top k rollup
new 33d6aa1 THIRDEYE-497 : GROUP BY-based top k rollup
new 982f169 THIRDEYE-497 : GROUP BY-based top k rollup
new 4738d81 Bugfix for cumulative row calculation
new 15c0562 Set tenant isolation enabled to false in perf benchmark driver
new c0b0507 [Pinot 2200] Make broker timeout setting name consistent
new 4e69b1c Add missing license header to fix failing build
new 85204d1 [PINOT-2324]: Support upload of already tar gzipped segment
new 6413644 add date to hourly display
new ddd72a8 fix for funnelheatmap metric labels
new 6572acf Refactor predicate evaluation and fix bug in NOT IN for multivalued columns
new 72278d7 Fix to handle missing data in contributor (default 0 contribution for if no data is present)
new d053be7 contributors table layout width to fit on any screen
new 99f5434 [PINOT-2326] Add SSL vip support
new e0fcfda Merge: 99f5434 d053be7
new 3bb638d [pinot 2031] Avoid failures on select * queries.
new 8be13eb [PINOT-2339]: Scan based query processor
new 9be545b Fix typo: getExternalViewForResouce -> getExternalViewForResource
new fed1263 PINOT-2322 Add metric for Helix ZK reconnections
new d3cc857 PINOT-2328 Use ZK znode version instead of timestamp to determine whether or not to do a routing table update
new 8442737 Fixing the timezone issue for DAY granularity
new fff70b7 Fix baselineOffset calculation to handle DST properly for overview and contributors on day agg
new 78d67db Update dimension value config endpoint to return values that meet threshold for any metric rather only the first metric.
new dd7b09d contributor table total values, styling changes
new e428cd7 display MM:DD HH:MM on contributors table
new 69d073d styling change for contributor table
new 8d071f3 [PINOT-2339]: Scan based query processor: Work in Progress.
new a7acda8 [PINOT-2317] Fix connection validation logic on live instance change
new 265304e [PINOT-2339]: Scan based query processor: Work in Progress.
new 938364d Disable failing upload/refresh/delete integration test until segment refresh is reworked
new 28b5484 Use compile-time enforcement of Pair type arguments to be serializable, as Pair instances are serializable
new 9c88c8b Removed empty test
new 5be3455 Removed unused IndexSegmentImplNew class
new d11f23d Adding missing hashCode method to DataTableBuilder, changing equals method to use EqualityUtils
new 9332955 Added missing @Override annotations
new aa865c3 Removed public access to the router field, added an accessor method for the router
new 5829806 Removed useless System.out.println statement
new 80c59ec Setting StarTreeIndexSpec to be ignored if not present in schema definition.
new 5e01780 THIRDEYE-619 : Join phase should emit list of records from perform join
new ba62975 [PINOT-2339]: Scan based query processor: Work in Progress.
new 17dad96 THIRDEYE-619 : Join phase should emit list of records from perform join
new d932558 [PINOT-2339]: Scan based query processor: Work in Progress.
new 53de3f7 Adding missing equals and hashCode methods to MutableLongValue
new b05cbf2 [PINOT-2339]: Scan based query processor: Work in Progress.
new ca8087f Moved readers in wrong package to the readers package
new acc7f1c Fixed potential integer overflow due to a computation being done on integers rather than long values before being coerced to long type, turned long variable into int
new b4c0602 Removed unused Kafka-related classes
new a4c4300 Fix Pql2CompilerTest comparison for NOT IN clauses when duplicate elements exist in the element list
new eac3a94 Fix NOT operator comparison during Pql2CompilerTest
new d3e1fc1 [PINOT-2339]: Scan based query processor: Work in Progress
new d9184ba [PINOT-2339]: Scan based query processor: Work in Progress
new 31754fb Add metrics for direct buffer usage and mmap size in pinot-server, add additional logging on allocation failure.
new e47a4ab Add metric for number of mmap'ed files, add logging of the count of mmap'ed files on mmap() failure
new d9dada4 Log controller port in perf benchmark driver
new 6aba3e2 [PINOT-2315] Fix handling select * across heterogeneous schema
new dbc1f98 [PINOT-2339]: Scan based query processor: Work in Progress.
new 73e514c [PINOT-2339]: Scan based query processor: Work in Progress.
new 47426b6 [PINOT-2371]: Copy of RB: 612583: Add serialVersionUid field with old value
new 9ba3975 Fix Contributors view to support dimension filters by returning empty data for pre-selected dimensions
new a3f10c6 Clean up input date standardization/merge into ViewUtils
new e61ff20 Contributors table will display N/A for the total values for metrics with Ratio() form, timepicker US format, styling changes
new 232be52 Remove selected dimensions from contributor dimension selector
new be279f0 Bugfix on contributor warning log message. Update SQL logging to for contributors.
new df90d44 Refactor template variable population to use DashboardConfigResource
new 97c8df8 Create IntraPeriod and Granularity enums. Split HeatMap calculation into smaller optimal queries.
new 81bfaa4 Undo HeatMap query split and return to full time range query due to inconsistent query response behavior.
new a617462 fix WoW btn display
new 181d19d Dashboard REST endpoints: Adding grouped endpoints for dimension info, metric info, and schema (dimension + metric info)
new 963a1f0 Rename 'End Date' -> 'Date' in config header for dashboard
new fda6979 Add N/A columns for current/baseline if no ratio is available
new b199563 [PINOT-2339]: Scan based query processor: Work in Progress.
new ad56f00 [PINOT-2339]: Scan based query processor: Work in Progress.
new a41a5f1 THIRDEYE-618 : Handle null nested fields in ThirdeyeAvroUtils
new 3067beb THIRDEYE-621 : Add ability to set configuration options to hadoop jobs through command line
new 2d385d7 Copy DropWizardApplicationRunner to thirdeye-detector. See #b3794de for reference
new fe34142 Exclude log4j artifacts from anomaly-detection pom
new 5cbc246 Specify jackson version for anomaly-detector
new 47d383a Update jackson version to match dropwizard dependency for detector. Bugfix in detector email configuration deletion
new 791762a [PINOT-2401]: Fix for UploadSegmentCommand deleting files it did not create
new ae17e9d [PINOT-2401]: Fix for UploadSegmentCommand deleting files it did not create
new 72d8818 Adding a rebalancer for Pinot V2
new 5b65bc4 Adding command to support rebalance table
new 4d5ad2f [PINOT-2350] Fixes for bid-suggest issues
new d906c0e Cleanup logging in our unit tests
new 58a35c6 Add set method for broker host in PostQueryCommand
new 95add6b Fix compilation error in ControllerTenantTest
new 764949d Adding support to rebalance a table. Update 2
new 60a7ec8 Reverting the change made earlier since it was not needed
new 9f3c420 [PINOT-2339]: Scan based query processor, and query comparator
new df3a5ff [PINOT-2398] fix for logging an exception in broker
new 6d5e24b Added funnels REST endpoint for dashboard
new 8cb70c4 add select all checkbox and restructure total on contrbutors page
new db18b89 Configure funnel endpoint to auto-generate default funnel spec
new 6ca0846 fix for firefox: define click event on contributors page; styling changes on contributors table
new fa4d3d7 THIRDEYE-628 : Bootstrap should be able to read custom data formats
new 13dde0d THIRDEYE-628 : Bootstrap should be able to read custom data formats
new 5132946 617227: THIRDEYE-628 : Bootstrap should be able to read custom data formats
new 0bac170 THIRDEYE-628 : Bootstrap should be able to read custom data formats
new fc81568 THIRDEYE-628 : Bootstrap should be able to read custom data formats
new 74677e9 Make pinot-perf depend on pinot-integration-tests so that we can reuse the test harness setup logic in performance tests
new 0a4641c Add doppelganger configuration file to gitignore
new 57f7857 Add methods to configure Kafka log segment size and log retention size
new 89ab384 Initial commit for bitmap changes
new 4656c36 Removing the usage of getMatchingDictionaryIds to filter segments, this is expensive for Not In and Neq operators
new 5edaaa0 [PINOT-2339]: Added table config as input to support inverted index creation
new 4d527b2 Remove the log statement from Or operator close() operation
new f48543e Update ThirdEyeDetector anomaly function factory to use a properties file mapping function type -> class name
new 6f38621 [PINOT-2412] Added a tool to fix idealstate in zookeeper.
new 52a1790 THIRDEYE-622 : Write and read multiple startrees in bootstrap - First step to partition output of rollup
new 80e128d THIRDEYE-622 : Write and read multiple startrees in bootstrap
new 97ce8da THIRDEYE-622 : Write and read multiple startrees in bootstrap
new 900e4e3 [PINOT-2339]: Added distinctCount for scan based query executor
new 58ce34e Make number of rows required to flush a realtime segment configurable
new a522cf0 Add realtime perf code
new 171e665 Refactoring Readers and Writers to support format upgrade
new 1fc2a74 Refactoring MSelectionOnlyOperator to use SelectionFetcher for removing redundant work during row collection. SelectionFetcher is used for querying rows from given blocks and schema. By giving blocks and schema, SelectionFetcher will underlying initialize different iterators based on block and column type and provide the ability to collect a Serializable array as a row for any given docId.
new 9789ff4 Refactor duplicated integration test code into utility methods in BaseClusterIntegrationTest
new e676998 Fix issue where the last row of a realtime segment is discarded (also reported and fixed by @izzizz at https://github.com/linkedin/pinot/pull/66, thanks!)
new 1c6a2fb Make realtime indexing loop more robust, add flaky realtime provider integration test.
new eb96e25 Fix compilation error from bad refactoring
new fff33cf [PINOT-2339]: Runtime improvements and supporting group by on MV columns
new 9f9f95c Fix NPE after swapping a realtime segment
new e727b39 Added comments and cleaned up the code for the realtime segment manager
new 28b1850 Removed empty unused class
new 26385d8 Renamed PinotRealtimeSegmentsManager class to PinotRealtimeSegmentManager
new e385896 Use HelixHelper.updateIdealState to update the ideal state instead of HelixAdmin.setIdealState
new 9e6edd2 [PINOT-2339]: Gracefully skip segments that do not contain columns in the query
new f4cc0bf [PINOT-2381] Duplicate bytebuffer before bulk-copying data
new f8e7dac [PINOT-2423]: Adding plugin to build independent jar for query comparison
new e7e0304 Adding v2 forward index format
new f27f320 Config form Go btn styling change and fix of WoW dropdown for Firefox
new bc4fd24 Fixing the import bug with previous commit
new 00a9a78 [PINOT-2339]: Fixed QueryResponse to match BrokerResponse from Pinot
new 6d15112 Cleaning segment download error handling code
new d2490ba [PINOT-2339]: Making segmentsDir optional
new fad95af [PINOT-2339]: Changed info to debug for messages
new 7079c7c Generate Pinot schema name and the output filename for the Pinot schema based on the input filename
new f57b949 Adding operator names to every operator in filter phase to help debug perf issues
new e545a98 [PINOT-2443]: Adding Perf Report Generator
new 5a442fa Adding default handlers to transition from ERROR state to OFFLINE for segments and broker resources
new e087a6d [PINOT-2443]: Added column name in report
new 7bc6544 [PINOT-2433] Adding timing logs in broker
new 864c67e Fixing the star tree queries test
new 88f9e5f Allow setting Kafka consumer properties in table configurations, update RealtimeClusterIntegrationTest to set the value of "auto.offset.reset" to smallest
new 1e43bb8 Remove dashboard start page and replace with redirect to default funnel view. Add collections config endpoint for dashboard.
new 635d368 adding collections to dashboard views
new 62c8fe6 display collections on dashboard
new b7cc5d0 select collection dropdown option on pageload
new 103151b Adding label 'Dataset' to collection selector
new dd49d93 Replace landing page with automatic redirect to first collection
new 2e68541 [PINOT-2465] Use bitmapped filter op only when the filter type is not range.
new 3aefa21 [PINOT-2441] Plumbing to get the number of null values for a field when creating segments
new a638291 [PINOT-2420] Gracefully handle limit 0 clause in BReusableFilteredDocIdSetOperator
new 517c9ab [PINOT-2439]: Reduce number of ZK watchers on Pinot Controller
new 7c6055f Dashboard: update FunnelsDataProvider to safely handle server connection issues when generating default funnel specs
new 0b5f632 Cleaning up Deprecated code
new ee38270 Fixed realtime integration test
new 60e4b05 Detector: Add anomalyIds query parameter for metrics graph ui and update email report to include individual link per anomaly
new c3d5785 Detector: Additional code to try to determine application port for email report manager, + log4j file
new 92c6d94 Reduce broker messages on exceptions.
new 00489ab Turning of printing query plan for every query
new 08a2a0a [PINOT-2473] Optimize RangeOfflineDictionairyPredicateEvaluator.
new da8b759 Refactoring block related impls
new 3789d4d Fix for the failing tests
new 86d5fbe PINOT-2478 Allow specifying the Avro schema name for schemas that are named differently from the Kafka topic
new 4e176fb [PINOT-2484]: Check for instances that are not enabled in Routing Table Instance Pruner.
new 98c4708 Deleted unused classes
new df9dcc6 Bugfix in DefaultThirdEyeClient with derived metric projection types
new 2a29ade Set up tenants for hybrid integration test instead of using the default tenant
new eeae20a [PINOT-2436]: Improving broker runtime for updating TimeBoundaryService
new f00fb88 [PINOT-2413] Fixing segment refcnt logic to increment/decrement under lock.
new dae60ed Logging messages when updating segment state
new c0623d5 Merge branch 'master' of git://github.com/fx19880617/pinot-1 into xiang
new ff981d9 Adding support to maintain context while scanning fwd index, Refactoring selection and ordering phase to not create unneccessary objects
new 38d6557 [PINOT-2435] Setup helix to never disconnect from ZK
new 00b3dde Fixed tests to not use old API
new 46c05ab [PINOT-2484]: Fixing return value in original RB
new 682c57f [PINOT-2345] Use default value of 0 if the timeout is not set for broker/server
new 81d5cf3 [PINOT-2345] Use default value of 0 if the timeout is not set for broker/server
new c08f912 Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
new 32a5f91 [PINOT-2484]: Fixing return value in original RB
new 7a49da2 Using SelectionFetcher to get raw row instead of old way of doing iterator initialization for collecting every row.
new 75c14fc Reducing the amount of java objects created during query execution
new 1eaedb8 [PINOT-2505]: Log hostname of client from which the segment is being uploaded
new ade7b5a Write the tenant isolation setting even if the node already exists instead of ignoring the setting write silently
new bf0c6ef PINOT-2502 Update routing table on any znode version change, not only on znode version increasing. Fixes an issue where the broker routing table would not be updated if a table is dropped then recreated with the same name.
new 83cd9b0 [PINOT-2509]: Improving segment generation runtime by eliminating un-necessary steps during row indexing.
new d7c7b69 Fix for RoutingTable test: Test needs to provide actual instance configs
new 72c7102 Fix for broken MultiValueIterator
new e243455 Fix for broken MultiValueIterator
new 93eeff8 Fix for RoutingTable test: Test needs to provide actual instance configs
new 314efdc Refactoring bitmap index creator to be more memory efficient
new 0c2b51b Handling the case where totalNumEntries is zero for single value case
new b63a3b7 Added methods to drop tables in integration tests, to make it easier to test certain scenarios locally
new 6ab0172 [PINOT-2451] Changed the segment refresh to be message based instead of toggling states
new a7d8081 Using Pair instead of int array to store start/end of sorted ranges
new 26c43de [PINOT-2451] Fixed segment refresh logic when servers are down. Added a unit test
new da9b31e [PINOT-2515]: Correctly handle UTF-8 encoding, fix for query comparison
new af1b409 Add sanity check for time value and units during segment retention, add logging and associated test
new d0b87d3 Removing dependency on javafx
new 3ae7f9c Adding license header for TimeRetentionStrategyTest
new ecfe52c Avoid NPE when indexing a null multivalue
new 537212a [EKG-85543] Fix NPE on segment refresh when no instances exist for the table.
new 6b8cd65 Adding algorithm to speed up intersection among sorted ranges
new 4f51151 Adding license header
new efac08a THIRDEYE-640 : Custom funnels should appear before default funnel
new 9ccaa6a Fixed multi-value inverted index creator
new 4d431c9 Fix metric columns not being serialized properly in the realtime pipeline
new 5b70dc6 THIRDEYE-644 : Add ability to delete/create funnels config files via curl call
new 32ec1a2 THIRDEYE-644 : Add ability to delete/create funnels config files via curl call
new 78ebfea [PINOT-2554] Set new refresh protocol to be default unless specified otherwise in zookeeper.
new 61766b8 [PINOT-2552] Fix null value generations when transforming avro records
new 1a26ea5 Formatting all thirdeye module java classes according to thirdeye/thirdeye-style.xml
new 1472fd6 Don't ignore test failures when building
new 88108e3 Fixed ClassCastException in OrBlockDocIdSet
new a115d89 PINOT-2553 Fixing issue with the AND operator for certain queries having a non-flattened operator tree
new 1607270 Adding utilities to debug operators/docIdSets
new 61297b7 Fix broken unit test for realtime metrics which have a dictionary
new 09679a0 Adding missing license header
new 4dfd1c5 THIRDEYE-645 : Dashboard fails to start if funnelsConfigRoot folder does not exist
new 30edde2 Validate that realtime rows have no nulls in them before indexing them
new ec1262d Added debugging tool to print values given dictionaryId
new 59583ab Adding code to create inverted index columns on realtime segments
new fd9409d Detector: additional change to try to pick up app port number. Update function scheduler to handle failures and log out warnings.
new 1eda962 Enhanced DefaultThirdEyeClient and refactored thirdeye-dashboard to use the client.
new 02937da Minor code cleanup in DoubleDictionary and FixedBitSingleValueMultiColReader
new 5e9ed69 Refactor thirdeye client and client factory to separate caching layer and flow control implementation from DefaultThirdEyeClient
new 63c28e8 THIRDEYE-627 : Explore use of kafka api on etori to check completeness of data copy onto data/tracking
new 72a412c THIRDEYE-627 : Explore use of kafka api on etori to check completeness of data copy onto data/tracking
new 39ebe80 THIRDEYE-627 : Explore use of kafka api on etori to check completeness of data copy onto data/tracking
new ed33bee On high qps, brokers are running out of connections to the servers. So the brokers were throwing exceptions since the max queueed connections exceeded 30. Increased the number of connections per server to 100 to allow for more qps.
new 1949641 Fixed unit tests for multi-value index creator
new d60a907 Reset max connection to 30, since we limit the number of threads in the server
new 3842b1d Thirdeye: move SQL generation code to client implementation and replace with ThirdEyeRequest where appropriate. Separate ThirdEyeRequest into request + builder.
new 2c41f41 PINOT-2583 Recompute realtime segment assignment on table metadata changes
new 90de2b3 Add deserialization metric, fix bug where total query time would be computed twice on the server metrics
new e86403f Revert java8 features to make pinot java7 compiler compatiable
new ab93b12 Fix the rebalancer code so that it can be invoked from command line correctly.
new b0e46ba Changed back bit offset positions to long
new 1d3e851 [PINOT-2606]: Change printing of routing table from info to debug
new 3769a1c Set ZK watches only for realtime segments in progress
new 6d15287 Add Apache license file to distribution package
new 401fb27 Add maven plugin to do github binary releases
new f28bf6e Adding totalDocument count and total Segment count to metrics
new 649061b Adding comments to new public functions in validation metrics
new e39eda6 Updating the javadoc comments
new 8aa7876 Log missing intervals when running segment validation
new 199c86c Disabling test TestStarTreeSegmentCreator
new f9b2c57 Add zookeeper reconnect metric to the Pinot controller
new 169a3aa Run realtime segment assignment logic on cluster leader change
new 30cfb13 PINOT-2614 Add REST API endpoint for updating realtime schemas
new 5501d60 Properly load total docs property from zk metadata
new 0f01727 Select first schema with non-empty record set for reducing server responses
new a5ea0bd Removing sys exit from PinotAdministrator
new 58d6ce7 Fixes to reduce flakiness in integration test
new 823aef5 Fixing zkClient dependency
new 51bdc82 Removing getTotalDocs method in IndexSegment api, replace it with segmentMetadata.getTotalDocs
new 9a020ad Minor fixes for tests that broke due to previous commit
new df7bb0a Renaming totalDocs to totalRawDocs
new 0f50bbf Adding more wait time for retention time manager test to complete successfully
new 2973b04 [PINOT-2635]: Added time validation in segment push
new ab7e0a3 [PINOT-2635]: Regression fix caused by original checkin
new 3b70237 [PINOT-2635]: Updating Chaos Monkey input data to generate valid date
new d5832ac Rewriting star tree index generation logic
new deee474 Fixing compilation issues with previous commit for jdk 1.7
new 31bb508 [PINOT-2635]: Fix ChaosMonkey generate segments with validate time range
new 32c9afb THIRDEYE-651 : Pinot thirdeye integration: Add ThirdeyeRecordReader in pinot to read aggregation output THIRDEYE-652 : Segment_push and segment_create jobs in thirdeye bootstrap
new e2cf2c6 THIRDEYE-651 : Pinot thirdeye integration: Add ThirdeyeRecordReader in pinot to read aggregation output THIRDEYE-652 : Segment_push and segment_create jobs in thirdeye bootstrap
new 75dce45 [PINOT-2635]: Rename variables and refactoring to original checkin for segment time validation
new b36b99b Removing the short circuit in DocIdSetPlanNode when filters are missing in query
new 65b70a1 Fixing test cases that broke due to previous commit b36b99bfa1c2aecdfbc32c2f55b2f2f46cc951d9
new ca8f664 Fix build for Kafka 0.9
new b66bfd3 Add missing "conf/" to -Dlog4j.configuration to the generated quick-start-realtime.sh.
new 0fabff2 [PINOT-2645] Performance fixes for operator flattening and AndDocdid iterartor
new b807471 [PINOT-2661]: Sort dimension names in metadata.properties file
new dd87bd8 Fix quickstart when using non-default ports
new 70b2a4d Support more Avro types in the realtime integration tests, allow setting a maximum batch size for Kafka writes
new dd1a424 Use ZkStarter instead of ZkServer in zkclient to start Zookeeper, fixes issues with zkclient 0.7
new 2676f9e Merge pull request #85 from mardambey/master
new ef2e5a2 Fix hidden dependency on jackson
new db67bd5 [PINOT-2645] Added a BrokerRequestOptimizer
new 8b9df53 Fix Server and Controller log message to include table and segment names, reason for rejection
new 13b1d87 THIRDEYE-654 : Modify SegmentCreationPhase to set config for star tree index
new 9247970 THIRDEYE-654 : Modify SegmentCreationPhase to set config for star tree index
new 93d3db2 THIRDEYE-654 : Modify SegmentCreationPhase to set config for star tree index
new 1ca64f4 Fix tracing to print out in json format so that we can parse it correctly.
new 4496446 [PINOT-2642]: Trim instanceResponse to appropriate size for TOP N queries
new 59933fb Vectorize sum aggregation function
new acc0662 Changing Jackson version to 2.2.2
new 345571d Changed default implementation in ImmutableDictionaryReader to use typed methods
new e7a0b29 THIRDEYE-655 : Add exceptions list to topk rollup
new ba29b7e THIRDEYE-655 : Add exceptions list to topk rollup
new 4c142f7 THIRDEYE-655 : Add exceptions list to topk rollup
new 46ffff7 Bug fixes for star-tree index creation. Introducing the concept of rawdocs and totaldocs in segment metadata
new ce104f4 Fixing iterator to be compatible with jdk 1.7
new 22d0f64 ThirdEye Client+Dashboard: 1. Added ThirdEyeMetricFunction abstraction to separate argument expressions and derive raw metrics. 2. Added PinotThirdEyeClient and factory. 3. Added CollectionMapThirdEyeClient and ThirdEyeClientConfig to instantiate multiple clients from a file and map requests/collections to the appropriate clients. Removed serverUri parameter from dashboard since it is no longer required. Added config endpoints for reloading clients and viewing conf [...]
new 6e33254 Splitting totalRawDocs into totalDocs and totalRawDocs
new 87e1979 Add default implementation of getHelixClusterName in integration tests
new 9b1372f Change log4j level to info, remove duplicate log4j property keys
new c25df8f Add metrics for controller internal errors
new c21af77 Creating more granual error codes for controller errors
new 248f318 setting start/End docId appropriately for all filter operators based on star-tree v/s raw index
new 1d60ac5 Fixing the test cases which was not setting endDocId appropriately
new dcbae1a Various integration test refactorings to remove duplicated code, add methods to delete kafka topics and realtime tables
new ce39500 Log contents of the row when it fails to index
new 157ad9f Adding links to Pinot presentations on slideshare
new b20cdbe Adding hybrid cluster scan integration test
new 7dc1b24 TE: migrate detector to use config-based CollectionMapThirdEyeClient. Add snapshot of detector_admin.py
new c83048f Wiring in star tree index operator in Filter Plan
new 8c02401 Fixing the logic to check if star tree index can be applied
new 7059759 Minor fix to star tree index operator to handle the case where column appeared only in group by but not in filter clause
new ad73d57 [PINOT-2654]: Automated query comparison for star tree index
new 6e99ff7 Fix when a predicate appears on the same column in both group by and filter by
new 3955c0a Replaced String.join (from Java 1.8) to StringUtils.join to compile on java 1.7
new 0f3c91d THIRDEYE-674 : Minor fixes to make bootstrap compatible with selfserve azkaban template
new 3d38a8d Using composite operator instead of OrOperator to merge star tree index doc id ranges
new 67ecf7f Code cleanup based on intellij analysis
new d4570e4 Shorten realtime segment names
new 63b9c87 Compute document count for realtime segments
new fbfeead Removing unused hashset
new f82917d Calculating doc count for realtime segments
new aab80ee Addressing some review comments on cacluulating realtime doc count
new 363687f [PINOT-2677] Re-factored RealtimeTableDataManager to be like its offline counterpart
new d18f963 Fixing minor compilation issue
new 62baa5a ThirdEye: update thirdeye-client pinot dependency versions to be configurable
new 519afed Fix infinite loop in RangelessBitmapDocIdIterator
new 4acccb9 [PINOT-2716] Fixing memory leak in realtime index creators
new 29a26d1a [PINOT-2699]: Star Tree Integration Test
new e9fac4b Fixing infinite loop in BitmapDocIdIterator
new 184117c Star Tree Integration test fix
new c9b7c6c Misc Cleanup: Changed log level from info to debug for certain messages, added splitOrder, maxLeafRecords in segment metadata
new a20d6a3 ThirdEye: Add ability to hardcode collections for a pinot client
new 2f4b4cb Remove the unused InstancePlanMakerImpl.
new aeed07c Code cleanup: Renamed variables to be more intuitive
new 86722df Replacing original reference results file with scan based output, as it contains the all results instead of trimmed
new 4ff6d5d Lower async-http-client version dependency and move it to maven property
new 7f7b2f8 Pom fix for async-http-client property
new 3bfca1e Refactored ColumnMetadata construction to use builder pattern and a static initializer
new 759f6d5 PINOT-2721 Fix quote parsing in string literals
new 61ad60f Fix integration tests requiring Kafka when connected to VPN
new 2bb7a21 PINOT-2718 Display column range on dictionary creation
new 26539d7 Added more hard-coded queries for star tree integration test
new caa3728 ThirdEye: Pinot client bugfix for missing data. Updated 'No data loaded' message for datasets missing data. Update client reset endpoint to reload funnel configs.
new 1253f5b ThirdEye: Test class bugfix
new 35cb688 Initializing global ControllerMetrics
new d3d4cc1b THIRDEYE-677 : Create azkaban project for pinot flow of thirdeye
new a4c55c3 ThirdEye: bugfix for metric function generation in detector and reporting. Update documentation for UserRuleAnomalyFunction and add warning to Kalman + Scan.
new 7dcc4e6 [PINOT-2730]: Fix issue in dictionary creation when sorted order of padded strings becomes different from that of padded strings.
new 5bb16fb7 THIRDEYE-674 : Minor fixes to make bootstrap compatible with selfserve azkaban template
new 1a7e69b Replaced -force and -dryRun options for rebalance command with single -force option
new 360a29f Replaced -force option for rebalancer with -exec to make it consistent with other PinotAdmin options
new 926751f dashboard heatmap and dimension view enhancements
new b435b5b Optimized bit writes
new bb391ab Add additional debugging info to mmaputils
new cf5af46 Adding additional log messages to realtime, general cosmetic changes
new 42e3d5e [PINOT-2765]: Skip creating star node for specific dimensions
new 42338b8 THIRDEYE-695 : Join Transform phases configs should not be tied to the schema names
new 36935d7 Adding changes to setDimensionSplitOrder
new aefdb82 Build Fix, added null check to avoid NPE
new 1c43359 [PINOT-2725]: Remove old PQL parser
new 29cea66 [PINOT-2730]: Fixing bug in the original checkin to handle different sorting order of padded strings, adding a testcase
new ad0063a ThirdEye-detector: Update anomaly detection job to use scheduled start time instead of system time. Migrated UserRuleAnomalyFunction to loop through available timestamps rather than choose off of provided current / baseline.
new a4c6a43 Catch exceptions in integration test setup
new 0d6edd9 [PINOT-2782]: Protected against integer overflow in response size, if query contains a large TOP N
new 6f11af7 [ PINOT-2784]: Fixing timeUsedMs in BrokerResponse to include time of all steps in query processing, not just server time
new 042035b [PINOT-2785]: Skip predicate evaluation if dictionary does not contain values that satisfy predicate.
new 64219c9 Changing broker response to be printed at debug level instead of info
new 5fb7a86 THIRDEYE-697 : Detector: Configure email reports to optionally disable sending if no anomalies are detected
new bb99f08 THIRDEYE-697 : Detector: Configure email reports to optionally disable sending if no anomalies are detected
new fcee52b Get dimension values by querying only start and end days.
new 032ff4f Improve realtime segment creation/commit logs and metrics
new 234c63a This branch is part of the fix for Jira: PINOT-2592, which is checking for schema mismatches with rows from the inputs. In this branch, we move the transform of the generic row which has been read in to conform to the Pinot schema, modifying the value of the column from the input type to the type in the Pinot schema.
new 058277f Clean up routing table debug endpoint
new de933ae [PINOT-2795]: Re-implemented BrokerResponse to use native objects instead of JSON
new 841c41b Reuse Kafka consumer/iterator instead of shutting it down and creating a new one each time, allows recovery of transient zk failures when committing Kafka offsets
new 03c3ac1 Fix to support change in used brokers for thirdeye
new 63c8553 Fix to support change in used brokers for thirdeye
new 6b61dd9 [PINOT-2795]: Added ReduceServiceRegistry class to configure picking up appropriate reduce service based on brokerResponse at runtime.
new a96b237 Moving handleRequest method from PinotClientRequestServlet to BrokerRequestHandler
new 8262393 THIRDEYE-710 : Add default __COUNT metric with value 1 in pinot segments
new 7f5ff0b THIRDEYE-710 : Add default __COUNT metric with value 1 in pinot segments
new 171c7cb PINOT-2798 Refactor server metrics, use server/broker metrics for all metrics instead of direct calls to the Yammer metrics API
new 35cd612 Add default hostname to ChangeTableState class
new 0654c37 Add metric for offset commit exceptions
new b34cbed Cleanup: Added JsonIgnore annotations on BrokerResponseNative fields that do not need to be serialized/deserialized
new d46d0bb Bug fix in cleanup job, that failed job if METRIC_INDEX dir is absent
new 8b72352 ThirdEye: Bugfix in PinotThirdEyeClient to correctly handle groups. Small email fix for anomaly emails.
new 3a3524e Change pinot-admin memory size
new b101013 Bugfix in cleanup job that didnt retain pinot tar segments
new 485fb16 Set server metrics instance in unit tests, fixes broken tests
new f4ddb4a ThirdEye: Introduce shouldGroupByTime parameter for ThirdEyeRequestBuilder to allow for optimized dimension retrieval query in PinotClient
new 8e62b18 Thirdeye: Update dashboard funnels provider to print simple error on invalid funnel configs
new 04a0ba1 Bug fix: BrokerResponseFactory was not returning the correct response object
new 64b1c62 Set broker respone type based on request Object. Make reduceService local
new 03b4d07 Added missing call to set exceptions in response
new 6cfcd58 Thirdeye: Refactor existing ThirdEyeClient#execute implementations into BaseThirdEyeClient. Add mock client + factory implementations and tests for Pinot + Mock factories.
new 0fa5ecb Replace query for whole range with 2 queries with small ranges at start and end for heat map view.
new bfea7f8 THIRDEYE-712 : Daily push to pinot should delete overlapping hourly segments
new 30bac67 THIRDEYE-712 : Daily push to pinot should delete overlapping hourly segments
new c2db71d THIRDEYE-712 : Daily push to pinot should delete overlapping hourly segments
new 22c6abd Use mergeQueryResults in stead of mergeQueries.
new 82e1ad6 Return default value of 0.0 for average when no rows are selected, for backward compatibility
new d5d6ddc THIRDEYE-712 : Daily push to pinot should delete overlapping hourly segments
new 87b5206 THIRDEYE : Fix to make rollups an optional phase
new ff620e1 add perf mode into query comparison
new 7f426b7 [PINOT-2737] Fix default value for top N
new ec948cd Make query console (/pql?) only send queries to broker nodes in liveinstances
new 08a9e2c ThirdEye (client+dash): add topCount parameter to ThirdEyeRequest and override in heat map view. Add getExpectedTimeBuckets to ThirdEyeClient and existing implementations.
new adc782e Sort list of tables in /tables
new ab496e6 [PINOT-2839]: Bug fix for incorrect StarTree aggregation results
new eaf7475 THIRDEYE-723: Change Heatmap logic to query client for total result + calculate 'Other' category
new 276d4eb [PINOT-2806] Prevent large bitmap creation for equals queries
new 8379752 Thirdeye (dashboard): Bugfix for funnel view to handle missing data by returning empty map rather than throw REST exception and break page load.
new 1f51a4d Write realtime segment completion status to ZK after we're done committing offsets and requesting the Kafka consumer to be released, document failure modes of the realtime code, add better logging
new 374aaa6 THIRDEYE-733: Dashboard heatmap should request for full bucket interval
new 856f189 Hybrid scan comparison test
new 71fd57a THIRDEYE-732: Update request intervals to be inclusive start, exclusive end
new ee65fb2 THIRDEYE-732: Rename ThirdEyeRequest(+Builder) get/set date methods to indicate inclusive/exclusive
new e7c96e3 Use JDK 7 compatible-APIs for the hybrid scan test
new 21d8796 Remove import for unused JDK 8 class
new f66a245 PINOT-2810 : Add support for bucketed time granularity (eg 10 MINUTES)
new 7b28a7f PINOT-2810 : Add support for bucketed time granularity (eg 10 MINUTES)
new 03c2d59 PINOT-2810 : Add support for bucketed time granularity (eg 10 MINUTES)
new 67f192b PINOT-2810 : Add support for bucketed time granularity (eg 10 MINUTES)
new 4fbb043 [PINOT-2757]: Added functionality to skip dimensions from materialization
new 32a76a5 Refactor Swagger stuff to go into pinot-common instead of pinot-controller, so that we can reuse it in pinot-server and pinot-broker for API reference
new bafe221 Adding missing license headers, to be squashed
new 3780e42 PINOT-2804 Add pinot server debug endpoint
new b9f2f69 fixing kafka hlc routing table not handling ERROR status
new a08cd43 Merge pull request #88 from linkedin/fixing_kafka
new 3cdba0b Fixing sortedDocIdIterator cannot be rewinded after dictionary created, during realtime segment persistence process.
new 82755fd Merge pull request #89 from linkedin/fixing_realtime_persistence
new bb749e1 Add sorted column to realtime and cluster integration tests
new 790bdfc [PINOT-2757]: Added StarTree metadata class, also queries containing non materialized dimensions should not be served by star tree index
new 296a878 Adding support for min max range query
new 937b10e Fix QueryComparison Failure
new 2c1780e Merge branch 'master' into range_query
new 6554c00 Merge pull request #92: Adding support for min max range query.
new e2b85f3 Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
new 7f7854b Bug fix in native broker response for selection on multi-valued non-string columns with order by
new ee53a9a Refactoring MProjection operator to avoid excessive hashmap creation
new 1dacaf0 Simplifying the logic in BReusableFilteredDocIdSetOperator
new 15fb802 Fixing bug with previous commit and further optimization to reuse docidArray across calls
new a781366 Fix delete segment API on swagger which used to deletes all segments
new 19e7989 add version tag into swagger
new 5387382 Use "Exception" instead of "exception" for data table metadata key
new fd83d2a THIRDEYE-738 : Ratio metrics break the heatmap
new 6f6227b [PINOT-2875]
new 472521a Added extra log in tests to indicate the number of records in the mismatch
new 9859192 [PINOT-2779] Including more datatypes in flights data for testing
new 5b5e920 Merge pull request #93 from boy-uber/master
new 1883ebd Enabling INFO logs for more classes in integration tests
new 2ac24b6 Thirdeye: Pinot client bugfix to handle empty dimension value for last string
new c040bf5 [PINOT-2883] Add an INFO log if inverted index is not loaded for a sorted column, add a WARN log if choose a multi-valued column as sorted
new 7a25b71 Adding health check servlet and expose server status to be checked by health check servlet.
new 7b91f27 Clean shutdown kafka consumer during sever shutdown/table deletion.
new ba7be79 [PINOT-2779] Add more test cases for hybrid cluster test
new ea6534b Fixing the issue that new created realtime table won't subscribe children change.
new cefe360 [PINOT-2634] New interfaces to support memory buffers larger than 2GB.
new 6010d89 Added SegmentDirectory interface to to the persistent segment data
new bbfe585 [PINOT-2886] Validating table config and schema before starting realtime event consumption.
new a0755e9 Fixing flakiness in startree integration test by waiting for external view to be synced with Idealstate before running queries
new 0afe329 Adding travis support and extend the wait time for setup in integration tests.
new 01ae742 Merge branch 'master' of https://github.com/fx19880617/pinot-1 into merge
new 6024284 fix StarTreeClusterIntegrationTest license
new 9cb2925 This branch is part of the fix for Jira: PINOT-2592, which is checking for schema mismatches with rows from the inputs. In this branch, we modify the transform of the generic row to convert between pairs of types for the input data type from the reader and the Pinot Data type. We also count errors per column and rows with errors. For primitive types we convert if it is allowed by Java. For convertions from primitive types to arrays, we create an array with one element. C [...]
new 6841583 Adding realtime segment metrics
new 816c21d [PINOT-2891] fix hasInvertedIndex always true in metadata
new 795c18e [PINOT-2634] Integrate segment directory and PinotDataBuffer with segment loader and index readers
new fb0c5b1 Fixed allocation size for HugeByteBuffer and proper deallocation of buffer in tests
new 67582d0 [PINOT-2840]: Re-write of aggregation group by operator, for performance improvements
new c2c1848 Revert "[PINOT-2634] Integrate segment directory and PinotDataBuffer with"
new 9a5a7c0 Revert "Fixed allocation size for HugeByteBuffer and proper deallocation of buffer in tests"
new 8d7ce7e Metrics changes to add segment replication/error count metrics
new 27145b3 Adding host name configuration method to Kafka starter
new 41f65d0 Allow specifying the log directory when starting Kafka
new 83b9a8f Fix UploadRefreshDeleteIntegrationTest Failure, should use mkdirs in setOutDir in SegmentGeneratorConfig
new 3d4838f Changes to use updated names from pinot-core in SegmentGeneratorConfig
new 1b6f7c2 Add controller metrics for segments without replicas.
new 715ee4d [PINOT-2840]: Addressing comments on RB 691380 for aggregation group by re-write, after it was subbmitted
new 37538b1 Fixing usage of ThreadLocal
new 151dba3 Revert "Revert "[PINOT-2634] Integrate segment directory and PinotDataBuffer with""
new b516f9b Revert "Revert "Fixed allocation size for HugeByteBuffer and proper deallocation of buffer in tests""
new 1550ce9 [PINOT-2634] Ensure that segment resources are correctly relased after use
new 4fb7ae1 [PINOT-2634] Use PinotByteBuffer as PinotDataBuffer implementation
new f2c4d87 Fixed putDouble method in HugeByteBuffer
new b929d11 Minor changes to schema class
new 38bfe04 [PINOT-2840]: Contd: Re-Write of AggregationGroupByOperator: Framework enhancements to support Avg/Range aggregation functions
new cbff054 [PINOT-2905] Make Pinot Work with Travis
new 7d9207e RAP-8039: UMP job to build star tree and push to pinot
new 520379d RAP-8039: UMP job to build star tree and push to pinot
new e76887a RAP-8039: UMP job to build star tree and push to pinot
new e8778e7 Fixed recursion in hasIndex method
new d11f065 Fixed Preconditions to handle zero sized buffers while creating buffer view
new ca2d9ab Added tool and pinot admin command to compare ideal state with external view
new 0863119 Added more informative usage with examples to VerifySegmentState
new 57dc6e1 [PINOT-2840]: Re-write of AggregationGroupBy Contd.: Memory usage and misc cleanup
new 5edbd70 Added test for automatic conversions between types. Modified some conversion routines. Added support for transformation in real time case. Added metrics for rows with errors and conversions.
new b4cd82c Put partitionId into segment name if there are multiple partitions in one replica
new 18fea68 Merge branch 'support_realtime_segment_partition'
new 6bc3d99 RAP-8150 : Create topk job in thirdeye-hadoop library
new 8e1c264 RAP-8150 : Create topk job in thirdeye-hadoop library
new 0cb953a RAP-8150 : Create topk job in thirdeye-hadoop library
new ef9b6c2 RAP-8150 : Create topk job in thirdeye-hadoop library
new a883084 RAP-8150 : Create topk job in thirdeye-hadoop library
new 2048d2e RAP-8150 : Create topk job in thirdeye-hadoop library
new 7e88dc5 RAP-8150 : Create topk job in thirdeye-hadoop library
new 615903c Fixing data type mismatch when assign default null value from schema.
new 27c1c74 RAP-8217 : Add ability to set startree configs for jobs via properties
new 9e59189 RAP-8217 : Add ability to set startree configs for jobs via properties
new a59b584 Disabled unit tests for HugeByteBuffer and LBuffer
new bcdf1cb Disabled PinotDataBufferTest and removed stale test
new 1be5bdb Increasing test timeout to 3600 seconds from 1800 seconds.
new 55bb0fa Reduce the memory allocated in segment store unit tests
new f8c3e06 Abstract segment fetcher layer out, and copy existing logic to HttpSegmentFetcher and LocalfileSegmentFetcher.
new 0c1bf75 Add minmaxrange and count group-by function
new e7fbc6a Adding config to switch between old/new implementation of AggregationGroupBy operator
new 8a03dc4 Fix multiple index reading in readOneSegment, SegmentInfoProvider
new a8044ab Skip instances with no instance metadata during realtime segment assignment
new e9c31f2 Adding a tool to view star tree index
new bc651ba RAP-8246 : Create replacement job in thirdeye-hadoop
new 509e1d9 RAP-8246 : Create replacement job in thirdeye-hadoop
new 05eb08d RAP-8246 : Create replacement job in thirdeye-hadoop
new 5bfeca2 RAP-8246 : Create replacement job in thirdeye-hadoop
new 7040c97 RAP-8246 : Create replacement job in thirdeye-hadoop
new d887879 RAP-8246 : Create replacement job in thirdeye-hadoop
new 1d79b0e Fixed inverted index loading logic for the SegmentDirectory interface
new 7171f75 Close the file after loading the data from file to memory in PinotDataBuffer
new c558be1 Clean up PinotDataBuffer
new be675eb Adding Timer support into common metrics
new ea1c71b Fix int overflow bug group-by key generation uncovered by test
new 0bafce3 [PINOT-2959] Handle default value for a boolean
new 3a3ac2f Fix some functional and performance issue in SegmentInfoProvider
new a27324c Fix ChaosMonkeyIntegrationTest Failure
new 446017f Making Pair classes backward compatible for ser/de
new f5469ac Temporary disable RealtimeSchemaChangeIntegrationTest
new 195a09f We fix the case of empty dictionaries. This issue was caused by having a single empty string as the only element in the dictionary. We now fix this by making the minimum length of strings in the dictionary to be 1 so that the empty string is padded with a single pad character, so that the dictionary is no longer empty. Added a test for single empty string. Also added a test for single "null" string. For JIRA:PINOT-2947
new d4a6a2d [PINOT-2976] Undoing changes for default value of boolean.
new db87fa9 Add Exceptions Check in QueryComparison
new cbd5ff8 RAP-8312 : Topk job should accept top k values for multiple metrics
new f9cf8d1 RAP-8312 : Topk job should accept top k values for multiple metrics
new b3646db RAP-8312 : Topk job should accept top k values for multiple metrics
new 0d8b7d7 RAP-8312 : Topk job should accept top k values for multiple metrics
new 81f5b8a RAP-8312 : Topk job should accept top k values for multiple metrics
new 261dc54 RAP-8312 : Topk job should accept top k values for multiple metrics
new e83f105 Always use native brokerResponseType
new d5e4b9c Fix ValidationManager bypass all OFFLINE validation bug
new 820bbdf [PINOT-2912]: Aggregation GroupBy Rewrite: Cleanup: Merged SV/MV group by executors, eliminated map based GroupKeyGenerator and ResultHolder.
new 60e8e7b [PINOT-2983] Fix Travis failed tests
new 25b0c49 [PINOT-2885] Throws RuntimeException when trying to do aggregation on multi value column
new 812d7fe Added segment format converter to support conversion to single file index(v3) format.
new be55ae3 Handle bad segments with zero/negative cardinality during aggregation group-by execution
new 45670cb Fixes the conversions of Boolean to/from String in PinotDataType.
new e3c38be Fix overflow in ResultHolderFactory, add max capacity to PairArrayBasedResultHolder
new e48e2c0 RAP-8441 : Thirdeye jobs should not depend on a separate schema file
new d630ae0 Add distictcount distinctcounthll percentile group by functions
new 21320de Adding a test for multi-value reader/writer
new 8f43143 Apply segment format conversion if BOTH server and table configuration are set
new c4880d9 Adding config flag to dump statistics for ThirdEye Aggregation phase job
new af97dfc This rb contains the fixes to make the schema and avro file provided by a partner to work. Specifically, fixes the datatype of a timefieldspec to use the outgoing datatype. Removes the update of the schema with the schema from the avro file. In this way, the types from the avro file do not change the datatypes from the avro schema.
new 82c0be0 Correctly close segment after conversion and pre-processing.
new a7be58b Make segment directory close call safer
new b6a795d RAP-8469 : Derive metric types from avro schema
new b8d8859 RAP-8469 : Derive metric types from avro schema
new ab3f31c travis only build master
new 6b92a7b Reduce number of rows in MultiValueDictionaryTest, it's killing Travis
new 6c493f1 THIRDEYE-774 : Expire Hourly segments will fail if table name has underscores
new 2d3a472 Calling correct method to calculate offline segment document count
new f420747 Merge: 2d3a472 6c493f1
new 68fad77 [PINOT-2995]: Re-write of Aggregation Operator for 'sum', 'min', 'max' & 'avg' functions.
new 651474b Modify MultiValueDictionaryTest to use random, reduce the tested row numbers
new f74ddbb Add QuantileDigest aggregation function. Implement all the last aggregate methods in AggregationFunction.
new 1d7b65c [PINOT-3009]: Aggregation Re-write followup from RB 713246: Refactoring ResultHolder into AggregationResultHolder and GroupByResultHolder, and other misc cleanup.
new df3677b Disable the new aggregation group by implementation, it was accidently enabled in the previous push
new dfb498d Disable the new aggregation implementation, it was accidently enabled in the previous push
new 4a53cca Convert time column in PlainFieldExtractor
new a3fd665 Add _raw dimensions to skipMaterializationForDimensionsSet
new 759c225 Segment creation should skip materialization for _raw dimensions in star tree
new c5bb82b Segment creation should skip materialization for _raw dimensions in star tree
new bb1331a Segment creation should skip materialization for _raw dimensions in star tree
new d850ac0 THIRDEYE-776 : Create aggregation phase job in thirdeye-hadoop
new 9a3e7fa THIRDEYE-776 : Create aggregation phase job in thirdeye-hadoop
new a1f8901 THIRDEYE-776 : Create aggregation phase job in thirdeye-hadoop
new 1072d4d Make the new aggregation and group-by implementation work with all tests.
new bcbab67 Support startree index in the new segment format.
new eb9bae8 Modified V2 to v3 segment converter unit test to avoid timing dependency in the test
new fcab742 RAP-8469 : Derive metric types from avro schema
new 11fddc8 Replacing old thirdeye code base with new modules (thirdeye-pinot and thirdeye-hadoop)
new 1e04b02 Log transport connection pool configuration values
new 1ebd143 In segment metadata, set hasInvertedIndex field always true.
new 14e6ddc In CreateSegmentCommand, enhance the check for format to avoid null value, and fix potential false positive warning.
new fc928f5 Making BrokerReduceService registry synchronized (based on comment on RB: 678177
new beed064 fix legend coloring, fix legend positioning, no popup on ajax load, metric label styling, Fixing contributor view sort bug
new 4ed8f5f Fix PinotClientRequestServlet to handle special characters.
new e38a8f2 PINOT-3023 : Create PinotSegmentRecordReader and PinotSegmentToAvroConverter tool
new d445ff4 PINOT-3023 : Create PinotSegmentRecordReader and PinotSegmentToAvroConverter tool
new 1277aac PINOT-3023 : Create PinotSegmentRecordReader and PinotSegmentToAvroConverter tool
new 5b7fe84 PINOT-3023 : Create PinotSegmentRecordReader and PinotSegmentToAvroConverter tool
new 47a3b16 PINOT-3023 : Create PinotSegmentRecordReader and PinotSegmentToAvroConverter tool
new 2d05597 PINOT-3023 : Create PinotSegmentRecordReader and PinotSegmentToAvroConverter tool
new 89d7698 PINOT-3023 : Create PinotSegmentRecordReader and PinotSegmentToAvroConverter tool
new 2113445 Fix legend color
new 90a4b20 add anomalies tab, heatmap header change, update dataset on home
new 59c6a9b tabular: click cumulative cell will return cumulative timeange, adding helper message to tabular
new a26d9d1 Fix Segment converter precondition to allow zero sized buffers
new d876e38 Adding method to stop all Kafka consumers
new e7afb19 Adding config support
new b5a38e2 heatmap with d3
new 89c3d11 Disable ChaosMonkeyIntegrationTest
new 8a30040 Creating BaseThirdEyeApplication to be used by both Dashboard and Detector
new cde0811 anomalies table
new ed1f453 anomaly ui
new 8eae37c add ranges
new b47a12f Bumping up versions of jackson to prevent conflict with ump
new 5459c16 THIRDEYE-790 : Create backfill job for thirdeye-hadoop
new 8b26965 fix for anomalies view timeseries legend, Fixing heatmap contribution change view
new af9c882 Re-implementation of MCombineOperator for faster inter segment combining of group-by results.
new 9706371 Adding support for Metric Expressions in TimeOnTimeComparision
new d04d842 [PINOT-2779] Added a command-line based test-invoker for running hybrid cluster test for different use cases
new 253e75f refactor anomalies, bugfix for timeseries, heatmap css change, rename js files
new 8a5e5f4 Adding support for ratio in timeseries
new 5dc9d89 Trimming filter query parameters
new c58c5a8 Add assertion when two strings map to the same padded string in the dictionary for a column.
new d626db8 In new aggregation, resolve some java style warning, remove explicit casting
new 8c5cfea Remove excess logging for segment's replication
new a3d3e35 Enhance QueryComparison exception check, print proper failure message
new 0021ae0 Sort segments displayed in the routing table debug endpoint
new 130eecf timeseries legend fix, error handling:checking for falsy data, display error on chart area or form
new 2c7c7a3 code cleanup, apply changes to tabular timeseries legend
new ce2f2c0 Add metadata at the segment level about number of type errors/conversions/nulls
new b937407 Fixing the number of decimal points shown on dashboard
new 6c06a3d Reduce logs in the broker
new b85bdaa Fall back to old group by algorithm when cardinality product of group-by keys cannot fit in long
new 434ae00 Adding support to pinot-client api to dynamically select pinot broker based on table name
new a32d97f Fixing tooltip and color shade in heatmap
new d6a0aeb Move SegmentFetcher to pinot-common from pinot-server
new 22b6836 Adding detailed logs for kafka consumer and broker response.
new 1b033e2 maxTime and additional error handling
new 9c6f5be fix for filter removal,support for max time and fix color coding on dashboard
new 5928b56 Changes to dashboard for efs
new 9ae20f9 Changes to dashboard for efs
new 1dc6591 Changes to dashboard for efs
new cfcf550 Changes to dashboard for efs
new ea86e53 Fixing configs to pick up changes to connection and idle timeout settings correctly
new 0e2dc9d Changes to support new config structure
new 807bf98 Fixing issue with DynamicBrokerSelector
new c44a76c Minor refactoring in caching collection schemas
new d2ff241 Added collections cache with whitelist/blacklist, periodic load of collections, endpoints for cache refresh
new bde2091 fix tabular heat-map-cell coloring, add max min time display remove old files, styling changes
new 2fd228d maxtime related changes, aggregateGran fix, tabular date display based on granularity
new 2dc586a Fixing log message for star tree generation
new 97df477 separate form component requests
new 0454f36 timezone support consistently, error msg when for max time
new 0db858d Add more checks for realtime segment assignment, catch exceptions that may prevent other tables from getting segments assigned, add metric for segment assignment errors
new 31a6b95 Getting the command line runner to capture and re-use scan based query results
new e211383 Add Codecov Support
new 6d5a0cf Move groupby directory under aggregation
new f9a97a5 Fix perf issue in Star Tree index generation
new cca8e05 Fix anomaly detector to handle derived metrics. Add adhoc file-based endpoints for testing both functions and emails (functions will still require a pre-existing function to associate anomalies). Add new endpoints to detector_admin.py
new 0eb19e1 Fix anomaly detector to correctly instantiate pinot client w/ caches
new e14e8b7 Changes to support __COUNT in expressions, derived metrics in collection configs
new dddd42b Changes to support __COUNT in expressions, derived metrics in collection configs
new 10a0588 Move metrics away from constructor to fix initialization issue
new 8e61c6e Anomaly UI fix to retrieve anomalies based on selected time range.
new e668b08 config aggregate, timeseries axes, time selection options based on max time
new 264165b Fixing pinotthirdeye client to use right broker according to table name
new 904e788 Infer the time column parameters from the schema file
new 411c9f7 Handling ratio in dashboard
new 93e51bc Handling ratio metrics in COntributors View
new 9c7d22b day granularity related changes, dashboard option error handling and changing default dashboard styling changes
new 8dcd9d5 Handling ratio metrics in COntributors View
new fc20785 [PINOT-3087]: Use dictionary based aggregation/group-by implementations for realtime segments
new dc951bf [PINOT-3094] Allow column names with dot while reading indexes
new 8b984d9 [PINOT-3090]: Fix for issue where Star Tree returns incorrect results for group by queries when group by columns are not materialized.
new 785008b [PINOT-3089] Modify QueryRunner to make it able to test the system QPS with multi clients
new ceb3080 Minor changes to backfill workflow to handle cases encountered while testing efs and sitespeed
new 2f31a32 Refresh collectionsCache and maxDataTimeCache periodically
new 82952c6 Refresh collectionsCache and maxDataTimeCache periodically
new 8151822 Topk and Derived phases should be able to function as noop in case of null configs
new 73ccdb4 Moving cache initialization to CacheRegistry initialize
new a7155a8 Clean up MCombineGroupByOperator
new 17a05a4 Use background thread to check Zookeeper state every 5 minutes. Modify metric to count number of tables with fewer replicas than requested. Add metric for number of segments in error state.
new 1222594 tabular view to contributors fix, help icon, dataset data granularity DAYS sets the default timerange to 7 days
new 8e8da99 fix for timeseries line colors in FireFox
new 1b317bf daily granularity selection refactored, removing max date from ui-kit date picker calendar
new 6ab8563 [PINOT-3091] New server side trim function for new aggregation group by
new 4d05255 [Pinot-3102]: Skip StarTree based query execution when there are predicates on 'metric' columns
new ff2b9b7 Extract ThirdEyeResponse interface, BaseThirdEyeResponse, PinotThirdEyeResponse.
new f040b81 Rename ThirdeyeCacheRegistry to ThirdEyeCacheRegistry (naming consistency)
new 2c291bf Use ThirdEyeResponse interface instead of Pinot impl where applicable
new 21c1f0c Ignore unknown keys while mapping json table configuration to java object
new 9ab17ce fix for bug: manually switching tab while waiting for ajax response loads the charts on the current view
new 36f9e6e fix for filter bug when stepping back in the history
new 5345032 Passing filter set from heatmaprequest to timeontimerequest
new 10ac6fe fix for today date time overwriting max time in selector, hardcoded help icon link
new a1559f1 fix for today date time overwriting max time in selector, hardcoded help icon link, anomalies table layout fix
new efc1a89 Email report with metrics & filters support. Add link to Anomalies dashboard.
new 9074442 hiding anomalies
new 87c683b hiding anomalies
new 41142d1 hiding anomalies
new 509959b hiding anomalies
new 28d18a9 Minor fix in derived_columns job, which was failing in ump integration tests, but works fine on grids
new 91061ed Add pinot-benchmark into contrib
new 05d6738 Update anomaly detection job to handle NaN, automatically set function-specific properties in anomaly results. Increase time series handler timeout to 5 minutes.
new 48d6e46 Remove ThirdEye img logo in anomaly report emails
new c73fd97 fix for timeseries header
new afb4326 Fix dictionaries and indexes for column with entries which have different original and padded values.
new 6083419 Handle strings in time column gracefully.
new 0ea1445 Adding toString method for Result Set Group
new a0a9870 fix for Firefox heatmap tooltip, filter in title on contributors, tabular delta chart changes, utility refactor, header performance optim
new 844edc0 Controller statistics for replicas and errorstate for non-leader should be set to 0.
new 5dc1843 [PINOT-3101] Fix connection leaks on server restarts and connection timeouts
new fccdf45 ThirdEye Anomaly Detector: Add emails on failures for alerts, function executions, and function creation failures at startup
new e64a4fe Adding filter cache, dashboard cache, and easier startup setup
new d1dedb4 Adding filter cache, dashboard cache, and easier startup setup
new 170caf8 Add a listener only once for leader change in Segment status checker.
new 888e9b6 Handling __COUNT through CollectionConfig instead of always skipping
new 253fbb6 Fix the flakiness of sortedRangeIntersectionTest
new dd4d3a0 Adding support to group by aggTimeGranularity on client side
new c2b00e0 Ignore unknown keys when mapping json configuration objects.
new 58fbeea Moved license header in TenantTest to the top
new 0a3983d Start the server admin component on server startup.
new c76816b Added methods to read total segment size on disk.
new 581fa40 [PINOT-3151]: Fix incorrect min/max value in metadata for star tree segment.
new 119ba02 Adding bin related files to gitignore (#130)
new a21c29f Update user rule anomaly message to include percent change (#135)
new b758743 Fix the order of start time and end time. (#132)
new 610c841 Add build status image to the README.md file
new 8d4206a ThirdEye anomaly: analyze top-level/global data only if exploreDimensions is not set (#139)
new 1ad0d57 ThirdEye anomaly: add default/minimum window sizes of 24 (hours) and 7 (days) for anomaly alert graphs (#138)
new 94beba5 dimension dropdownon tabular, refactor js, tabular header tooltip, contributors delta chart area color (#133)
new eca0507 THIRDEYE-740 : Config driven support for custom dashboards like sitespeed (#144)
new b3f24af Fix issue related to window delays for anomaly detection job & email report job. (#143)
new 60c5832 Created a simple utility to read multiple HTTP URLs in parallel using (#142)
new c10c86e Added server REST API to provide table sizes along with individual segment size information (#131)
new 84a0dd4 [PINOT-3104]: StarTree: Enhancements and performance improvements. (#141)
new 9da299a Let Travis ignore changes not related to pinot code (#140)
new 4d240db [PINOT-3176]: Enhance StarTreeIndexOperator to support all predicate (#147)
new d8f0584 Fix that Travis will end with error status when the commit is not related to pinot
new c0e2017 Remove precondition check where it can be triggered per record, as it can impact performance. (#149)
new c41e68e Optimize for doc level function in group key generator (#150)
new d88e20a In DefaultGroupKeyGenerator, for single value group by column, fetch dictId per block instead of per doc. (#153)
new b138bbf ThirdEye: Update User Rule anomaly message to display with percentages. (#148)
new 06224c1 Remove array creation in Avro message decoder, replace magic numbers by symbolic constants (#155)
new 6283f66 Add support for documenting HTTP responses
new 4217ffb Change instance status API, fix instance listing (#159)
new fb6d682 Adding 10 secs delay for broker to shutdown. (#136)
new 5bc5bb8 Move Travis status badge to be with the gitter chat
new 42e85f8 Add instance creation API (#162)
new 5de4fae tabular view cumulative table related change (#154)
new d88f0a6 Fix Travis script so that if the commit range is invalid, exit with error state. (#168)
new dae8a98 Star Tree Creation: Added check to ensure that time column is preserved (#165)
new 65a2bd7 Added utility class to convert human readable data size to numberic bytes (#145)
new 02aba40 Add REST API endpoint to get instance info (#171)
new 887fed5 Added controller API to provide table sizes (#156)
new dd6f013 Upgrade swagger-ui to 2.1.4 (#169)
new 70f7288 Improve schema and segment APIs and docs (#164)
new 68d1e7b Disable integration tests that overlap (#160)
new 59ca9c6 Removed dependency on javax.ws.rs in favor of explicitly setting response content type (#172)
new 43e6f5b Implement a separate data fetcher as a higher level abstraction for data fetching (#146)
new 8d86c1e Enabling anomaly detection tab (#175)
new f36361b Refactoring of view handlers, to support parsing of single query instead of one per time bucket. (#173)
new 9293aee hiding total on contributors, daily data granularity preselects midnight data on pageload (#177)
new d73b69b TE: Adding dependency between email reports and anomaly functions for anomaly retrieval in reports. (#176)
new 5efbdc4 Fixed caching issue by adding hashcode and equals method to PinotQuery (#180)
new 548e309 Adding more logging for anomaly detection job to make debugging (#182)
new 99bbbb9 Keep build errors during Travis build (#183)
new 90ce39b StarTree: Address conflict between split-order and skip-materialization (#184)
new 70b98da Support segment upload with URI (#122)
new 94f89dc Adding fix for handling time buckets
new 02e36ce Adding fix for handling time buckets
new b2a9025 Add new realtime segment name format (#181)
new be071e8 Adding fix for handling time buckets
new 54b773e Change realtime segment timestamp format (#188)
new 124d17d ThirdEye: bump up anomaly detector quartz misfiring threshold (#189)
new 135d862 Increase the timeout for TableSizeReaderTest. (#187)
new 75310ca Remove most log output from Travis builds (#192)
new 9d263cc Validate table name on creation (#190)
new 7c90c76 Update license year to 2014-2016 (#193)
new 12c9a15 Add Pinot Hadoop jar version to segment metadata (#174)
new c64f56c Segment Status Checker: Handle case when partition online in idealstate is not present in the external view (#185)
new 86235b3 Bugfixing: wrong query results if order by columns not existed in select columns list (#194)
new 4ed07a6 Use ideal state to compute segment assignment (#191)
new 9aab6a7 Add license header for BalanceNumSegmentAssignmentStrategyIntegrationTest (#197)
new 4119304 Fix for null value metrics of type other than INT/LONG (#196)
new 529c3d9 Fix the failure for BalanceNumSegmentAssignmentStrategyIntegrationTest (#202)
new 69e41fd Travis should fail if the install step fails. (#201)
new 03f5fdf * Shrink start and end as per data granularity when creating between clause for Pinot query. (#203)
new 8ba7c51 TE: Add test cases for PqlUtils between clause (#206)
new 2c1e398 Adding column function_name to anomaly_functions (#204)
new efd18d7 TE: Update timeseries client package to be consistent with comparison (#205)
new 7cc81e6 [PINOT-3198] Part 1 of 2 commits to change SegmentOnlineOfflineStateModel to contain the new CONSUMING state (#208)
new c32945a Add .codecov.yml to ignore auto generated files in code cov (#209)
new ef78c3f Add license header for .codecov.yml (#212)
new 98b4ed8 [PINOT-3198] Part 2 of 2 commits to change SegmentOnlineOfflineStateModel to contain the new CONSUMING state (#211)
new 67088e2 Let Travis only run tests on Oracle JDK 7 (#215)
new 648877d TE: remove unused classes and methods in dashboard (#218)
new cb7f771 Edit .codecov.yml config to ignore auto generated files in CodeCov (#213)
new 53b118e [PINOT-3248] First round check-in for re-factoring segment name code. (#216)
new 6e16852 TE: allow handler requests (TimeOnTime and TimeSeries) to specify inclusive endpoints. Update handlers to support inclusive endpoints and set all dashboard requests to use inclusive endpoint (detector uses exclusive) (#217)
new 391e46b TE Bugfix PqlUtils to use [start,end). Update ToT request constructor to inherit inclusive end date flag. (#220)
new 1c302f3 TE: fix test case to reflect [start,end) logic for PqlUtils (#223)
new cf8d258 THIRDEYE-816 : Table names should be able to display as alias (#225)
new 3e03063 [PINOT-3248] Second round re-factoring of segment name logic (#219)
new 4898cff Handle null external view for table. (#224)
new 13c7af6 Add REST API parameter type documentation (#170)
new cc3d4bd Add optional Kafka message headers in tests (#227)
new c1d3c4a TE: Update anomaly email reports to use collection alias if available (#228)
new 96cc73d Fix FieldSpec to let ObjectMapper handle any order of fields (#221)
new 9413517 [PINOT-3202] Added code to create and verify the kafka partition allocation amongst low level consumers (#222)
new 035af6f [PINOT-3285] Enable Realtime segment versions to be configurable (#231)
new d8e1b22 New Anomaly Dashboard (#229)
new 309ec24 Flip default value of New Aggregation GroupBy engine from OFF to ON by (#234)
new b5dde0f In new group-by, support group by keys cardinality product larger than long (#179)
new b2d4cfb Anomaly function (#237)
new b8a17ba Fix for overflowing cells in Tabular View (#238)
new 6446645 Fix class name in deprecation statement (#235)
new 17882dd Re-factored the StarTreeIntergrationTest to make it easier to add new (#236)
new e6a210e tooltip fix, anomalies view tabular call cleanup (#240)
new 80995b7 simple-anomaly-report css (#239)
new 47c446e [PINOT-3301]: Fix broken backward compatibility when reading older (#241)
new c2a4cc3 Remove unneeded debugging code (#242)
new 14b0d68 Adding fix for filters in tabular view (#243)
new 6bf93e0 Enhanced StarTreeIntegrationTest to test group-by results. (#244)
new 97ced7c TE: Timezone fix for email report graphs (#246)
new c62c816 Matching selection columns ordering for both request and response in DefaultReduceService (#195)
new 1a87d98 PINOT-3296 Handle spaces in segment names (#247)
new 1a0825b Lower memory usage of MmapUtils (#245)
new 3958b72 [PINOT-3310]: Added test to check older star-tree index can be read in (#250)
new bd60664 Improve documentation of routing table builders (#198) (#198)
new ba6e59b Enhance StarTreeQueryGenerator to generate all kinds of supported queries (#249)
new bfe5948 Remove exceptions used for control flow (#252)
new f7a5309 THIRDEYE-815 In heatmap summary, derived metrics (with ratio) should display what the ratio is, currently it shows the cellSize var instead of the actual ratio. (#251)
new 64974aa Handle exceptional cases in realtime consumption (#253)
new 114b046 Enhance QueryRunner to support single thread, multi threads and target QPS modes. (#254)
new bd57f2b Remove debug logs from QueryRunner (#259)
new 5ca6172 Handle BOOLEAN type in FieldSpec (#256)
new a047df3 Added a main method in StarTreeQueryGenerator to generate star tree queries. (#257)
new 8d8e1a5 JIRA832 (#260)
new b0de326 fix styling of heatmap tooltip of long dimension values (#263)
new d5631be Changing ISO time to timestamp (#258)
new 1237791 Create summary in time-series response JSON (#265)
new cd141f8 In SegmentInfoProvider, added time column as a single-value dimension (#264)
new 10027a1 Optimizing performance of splitting on time column. (#262)
new 477413d tooltip position fix, removed time range from anomaly config form, fo… (#248)
new 066f2f5 tooltip width fix (#267)
new 8ba7c3b Cleanup thirdeye pom.xml (#266)
new 63a6416 Add optional support for merging consecutive anomaly results. If anomaly results are not consecutive they are rejected. (#255)
new a964a24 dataset selection dropdown fix (#268)
new e958ab2 Distributed Anomaly Detection (#269)
new e5dc536 Updated ClusterTest to pass different admin ports for each server instance. (#270)
new 13108fb TE: Refactor code to support more metric functions such as 'average' and 'count' (#273)
new eba3488 Adding worker_id of worker which picks anomaly task (#274)
new 851c00d [PINOT-3329] Fix exception in realtime consumer hashcode (#275)
new 6128d35 adding new anomaly function - MinMaxThreshold (#278)
new 78f2f4d [PINOT-3329] Adding a unit test (#276)
new 5b8801c TE: adding API for Anomaly feedback support (#280)
new bafa110 Self service (#282)
new b03461e TE: refactoring anomaly feedback (#281)
new 8faef3c Implement metadata API to support AnomalyFunction CRUD self service (#284)
new 1bdc360 THIRDEYE-850 : Task driver in distributed anomaly detection should be able to run different types of tasks (#285)
new 998fdb7 Added storage quota configuration to table configuration (#166)
new 54d52cc Added missing headers for PR #166 (#288)
new 7eb2ee4 THIRDEYE-853 : Create migration script for new db tables anomaly_tasks and anomaly_jobs (#287)
new b2c2e05 Catch and log exceptions and timeouts in combined groupby operator (#283)
new f4b06e8 not adding OTHER metrics when all values are zero (#289)
new 325bc89 Disabling retention in integration tests. (#290)
new f735a71 Self service ui (#291)
new bb559ee adding self service ui code
new dae6ec7 minor comment change
new 8acb0d0 resolve merge conflict, remove comment
new 746dc8f Merge branch 'te_selfservice_ui'
new 3c0df07 Add low level realtime routing table builder
new c4b914e Exclude LLC segments from HLC routing table (#200)
new e99deff adding UI integration with anomaly feedback API (#293)
new b77b9e2 thirdeye build fix (#295)
new 82fbff2 [PINOT-3204] Plumbing the path for adding a new table. (#279)
new 5c940dd Enforce quota on segment upload (#294)
new 9802ed7 THIRDEYE-854 : Create a manager for DetectionJobScheduler to enable/disable/adhoc detection jobs (#296)
new 64f1e20 thirdeye data layer refactor (#297)
new 111ab50 Te self service update modal (#300)
new 2279375 TE: refactoring DAOs, removing duplicate code (#299)
new ea4b325 Throw exception on ideal state update failures (#301)
new da206f4 Te self service delete fn flow (#303)
new 78dce17 THIRDEYE-856 : Create monitor module in distributed anomaly detection (#304)
new cee43ee Getting addTable to work with both high and low level consumers (#306)
new ff22678 TE: adding guice-jpa provider (#311)
new f0fbb8e Let QueryComparison always use UTF8 charset (#314)
new 5206108 Fixing log messages when rejecting segment uploads due to time stamp or (#316)
new efb94ce TE: Enabling Application start (#315)
new 03e716b thirdeye hql fix (#318)
new 1246744 Te self service update (#312)
new c60240f Make the padding character configurable and change it to the null character. (#286)
new 54fe9d9 Refactor anomaly jobs and tasks to use hibernate JPA (#319)
new a2d33bd Automatically turn on znode compression (#302)
new dbc0654 Fix REST API error messages (#321)
new dfa4a8f TE: removing unused DAO classes. (#320)
new cf3a11f thirdeye db cleanup (#324)
new 03fe64e Te self service min max (#326)
new a700987 adding cron as query param in anomaly function (#325)
new 437ba80 Let Travis print install log
new 4c03980 Inside SegmentPreProcessor, enhance the logic of inverted index creation (#323)
new 7007c03 THIRDEYE-859 : Create report module in distributed anomaly detection code (#327)
new 49d192f TE : dao layer cleanup final (#328)
new 31ffb92 Fixing stale results issue (#329)
new ba1d711 [PINOT-3203] Controller code changes for Low-level consumer segments (#322)
new 94077cf Fixing filters in anomaly endpoints to accept json (#332)
new 4d56e2c TE: adding changes to suppress alerts when data is missing (#331)
new 2adae07 adding cron to create and update anomaly fn, cleanup anomaly table, anomaly fn form layout changes, display properties on existing fn table (#334)
new 2784e56 Fixing null filters (#336)
new a18bd9e TE: fixing db migrations script (#335)
new b6e7ea8 fix for daily CRON, adding default values to create form, error handling with fail callback on delete method (#337)
new 06a6929 THIRDEYE-865 : Make anomaly dashboard point to new database and manager (#339)
new e1c6394 [PINOT-3411]: Fix bug in tenant update that drops tenants while down (#342)
new ffbf070 [PINOT-3381] Updated ValidationManager to consider high and low level consumer segments correctly for doc count metric (#333)
new 053879a TE: adding anomaly merge component (#341)
new 6730676 Added query latency percentiles and other stats in the query runner. (#344)
new 674e009 Adding connection pool properties (#345)
new 84fe485 anomaly function filter update, cron schedule minute fix (#343)
new 831a642 Fix EqualityUtils to correctly compare Collections. (#340)
new c35fd57 adding anomaly merge rest end points (#346)
new faccb73 TE : improving anomaly merge algorithm (#351)
new 23847c9 Use the null character as a padding character so that we will be able to support strings with '%'. (#352)
new a91b011 [PINOT-3200] Adding capability in broker to select HLC or LLC segments for routing based on a selector class (#350)
new f085f62 TE: removing unused code (#355)
new fe8e9ce feedback fix (#356)
new bdba25d Fix IAE in low level routing table builder (#349)
new 74f1af1 Use Kafka message key in integration tests (#347)
new 4063c40 Support array offsets in Kafka message decoding (#348)
new 35ab13b TE : adding new merge strategy (#358)
new 6c87b2c adding changes for anomaly merge ui (#360)
new 082495c Check routing table size in HLC/LLC switching (#357)
new c75bc67 adding persistance for merged results (#363)
new 907aa05 Te self service active state (#362)
new e9a6b6c Simplify FieldSpec logic and reimplement equals and hashcode (#354)
new b35156d Adding transactional to findById for anomalyFunctions (#365)
new e177e25 Adding updated pinot schema methods (#366)
new 6dcac51 Renamed StarTreeDataTableOptimized to StarTreeDataSorter, and added some (#368)
new 1695034 Removing session management in detector (#367)
new f6c033f adding list of hours option to cron create and update (#364)
new cd895b6 TE: adding persistence helper for merged anomaly (#370)
new 900888d Adding fix for pinot schema deserialization (#372)
new a8024aa TE: transaction fix - adding transaction for all entityManager interactions (#371)
new 941e413 Add getOutGoingTimeColumnName() to TimeFieldSpec for third-eye backward compatible. (#376)
new 8383d5a fixes for transactional support (#378)
new 7b48c61 Disable illegal setter for MetricFieldSpec and TimeFieldSpec. (#374)
new bd52a3f [PINOT-3435] Adding segment completion protocol endpoints in the controller Added a utility in the server to find the controller leader (#359)
new af6742c [PINOT-3469]: Add a max allowed value for 'TOP' and 'LIMIT' in the query (#375)
new a66ba25 [PINOT-3464] Set the download url correctly in committing LLC segments (#373)
new 2f9f7a4 optimize single metric selector behavior, timeseries x axis display 'MM-DD HH a' format, self service: remove ^0 from 0[0-9] fromat cron hourlist (#377)
new 123e562 Add table name to the quota error message (#379)
new ac48b5e TE: anomaly merge code with some todos (#381)
new 97ee7c4 Fixes for email report url and deleting entities (#384)
new 61c3b43 [PINOT-3471] Fix errors during checking of storage quota (#382)
new dfb679f TE: Adding anomaly merge executor (#385)
new e0824eb Add preliminary Kafka SimpleConsumer wrapper (#338)
new a020799 [PINOT-3430] Moving ControllerLeaderLocator to core, since other realtime classes are there. (#388)
new 5fa1dbb Initial version of the LLC segment data manager
new 2d2f039 [PINOT-3479] Provide best effort segment size if the segment is being pre-processed (#387)
new 295a936 Added more logging around reading segment sizes from server during segment upload. (#389)
new c272c61 run adhoc anomaly function (#392)
new 5ef8277 adding ui code for comparing merged results (#393)
new 2aa0408 Schema evolution: Default values for newly added columns (#161)
new 51fa4c0 Adding hikariCP (#394)
new eb066a9 THIRDEYE-849 : Move webapp configs to database and create CRUD endpoints for them (#390)
new ba60e49 Switching default behavior for Query Response Limit. (#397)
new 4f52241 Set connection timeout and connection manager timeout for multiget request (#395)
new 69c7f05 Adding weight in AnomalyMergedResult (#396)
new c871606 Make default columns configurable (#402)
new b6a6457 [PINOT-3349]: Adding new interfaces for StarTree & StarTreeIndexNode. (#383)
new 0e61d5b [PINOT-3430] Added server-side support for segment completion. (#401)
new 050ab7b THIRDEYE-873 : Last day's data not seen on dashboard for login (#403)
new 55feb6d fix: on daily data granularity changing the default end date from midnight to last available datapoint (#404)
new 7b50bab fix for ratio metric display on timesereis: changing timeseries tick format (#405)
new 2f4055a Revert "Adding hikariCP" (#406)
new 4eba8da Fixing pinot schema method name (#409)
new e89c6cd Add Kafka leader broker lookup and connection (#407)
new 10eccb3 TE: removing duplicate code (#410)
new 50b70d0 disable collection and metric change on anomaly function update (#408)
new a173c45 Fix range queries using scan based filtering (#391)
new af0ffb0 Use stream provider config to init message decoder (#414)
new acecaf5 Keep connection if connected to the leader broker (#412)
new e0b4275 Read inverted columns config and applied to current serving realtime segment Adding unit test for ScanBasedFilterOperator.
new c669c39 TE: adding code to recompute severity in MergedAnomaly (#417)
new d9da314 Make indexing loop exit flag volatile (#416)
new 4571ee4 Handle having no sorted column in LLC (#415)
new 56dbcb3 Fix the number of replicas in idealstate so that we get multiple servers in CONSUMING state (#420)
new 09f51d5 Fixing severity field for anomalies (#418)
new 7bde232 TE: enumerated column fix for anomaly_functions (#419)
new 60d87bd TE: replacing c3p0 with tomcat-datasource (#423)
new e2992e6 Refactor PlainFieldExtractor and PinotDataType (#411)
new 21bb2b8 Fixed DELETE table to not throw NPE when table has only LLC consumer type (#421)
new ac265bb Change schema call endpoint from /schemas/{tableName} to /tables/{tableName}/schema (#427)
new 251d650 Re-write of StarTree and StarTreeIndexNode. (#425)
new 2d9b53c add hll column in segment with startree (#330)
new 0cfb2ae THIRDEYE-876 : Config driven support in thirdeye dashboard for tables having metrics as a dimension column (#426)
new 841e2bc Fixing NPE when getting max timestamp for realtime only table. (#428)
new eda332d adding anomaly feedback comment field (#424)
new 41f559a Upgrade fastutil to 6.5.16 (#430)
new 441e94f Canonical Schema (#398)
new 49eba44 Changing tableToBrokerUrl mapping, in order to support realtime tables in TE (#432)
new 23470d3 TE : switching alerting on merged anomalies (#434)
new 6eebcfa Add jacoco multi-package support (#435)
new 3fb5eb8 TE: fixing npe (#436)
new b50bda3 disabling chart from anomaly email (#439)
new 0e59468 change aggregation interface from double array to object array (#431)
new c705e88 fix for multiple submission on self service (#441)
new 5738645 Te self service form all fn types (#442)
new 0995e4e Refine the Pinot query for reducing query time. Specifically, assume that a query has a time range from 10PM to 11PM, the query was WHERE time >= 10PM AND time < 11PM. Now is refined to WHERE time = 10PM. The query time is reduced 60% to 90%. (#444)
new 2a628f3 Segment Completion corner cases (#443)
new e364d2e Fixing default top value for queries to pinot (#445)
new d90b99f te: adding changes to provide merged results on the dashboard (#448)
new fe7dc96 Backward compatibility for brokerUrl mapping change (#450)
new 9410d43 Delete duplicate TestUtils in pinot-common (#422)
new fcb11c8 Fix LLC consumption not working (#453)
new 99de48d adding rest end point for anomaly feedback for merged_anomalies (#452)
new 6160dc2 Periodically print query latency status in the query runner. (#446)
new f2570e5 Extract maxTime for a pinot table from query SELECT max(${time_column}) from ${collection}, instead of fetching from segment metadata. (#437)
new e393037 fine tuning datasource properties (#454)
new d37c378 THIRDEYE-849 : Change configStore file to use database (#449)
new c642a09 TE: Not sending repeated mail for merged anomaly (#456)
new 13601fb [PINOT-3578] Fix download path for segments. (#455)
new 120a230 TE: adding connection pool properties (#457)
new 364d56e changing baseline daet update on datepicker selection (#447)
new ceb4f16 clean startree test cases with hll (#429)
new 0614c79 restoring a method for backward compatibility (#458)
new b884d62 removing autocommit setting from db config (#460)
new 431e0e3 Cleanup: Removed BrokerResponeJSON and replaced it with (#451)
new d15e455 Ignore tools, perf and hadoop in code coverage (#463)
new 9b6c9be Te ui dimensions fix (#465)
new 782c699 [PINOT-3542] Fix table deletion to delete the kafka partition assignment (#464)
new 730dc15 filter display fix on existing anomaly fn table (#466)
new 77726cb TE: adding improvements for task driver (#467)
new 9dc2d1a anomalies view timeseries display changes (#468)
new b58295c Improve inequality performance on sorted columns (#433)
new 0dfc5c7 small styling change (#470)
new 886f273 Fetch initial Kafka offset on table creation (#462)
new 9a77909 Fixing maxDateTimeCache in case of simpleDateTime format (#472)
new b3d2550 Adding join, transform and wait phases to thirdeye-hadoop (#469)
new 5c93917 self service function type metadata hardcoded (#476)
new b0d1304 [PINOT-3505] Update committing segmentmetadata from the incoming metadata file (#473)
new 4239de2 TE: fixing chart rendering for selected anomaly (#475)
new 8920e3e adding dbi entities for parallel development of dbi based data layer (#478)
new e441966 Refactoring packages (#481)
new 1364d94 TE: obsolete code cleanup (#482)
new 3ac2e60 te: dto isolation and accessing entities using Managers (#483)
new 8055e38 Fixes in thirdeye-hadoop for standalone setup (#485)
new d383d19 Adding DAO for entities (#487)
new 5e995bc Te anomaly view timeseries update (#488)
new 11096fe [PINOT-3614] Bumping up jackson classes to 1.9.6 (#486)
new 222c4ea TE: adding rest end point for email configuration (#489)
new 0b2ed7e Improve display of Pinot processes (#490)
new dba4f4e Remove warnings about no WAR client connector (#492)
new 7694b50 StarTree: Added capability in server to auto convert star tree binary (#484)
new 17d0adb TE: adding test base class for new datalayer (#494)
new 63717a6 Renaming StarTree v1/v2 to on-heap/off-heap. (#495)
new 3432bce Add controller landing page (#493)
new 2b0cdbc Add console output for all Pinot services (#491)
new 2a85362 Renaming exiting Entity to DTO and DAO's to manager (#497)
new 6ebbcb7 Added QueryContext to pass information about the query to lower layers (#480)
new aaaffd3 THIRDEYE-884 : Restructure the way columns are named in thirdeye-hadoop rollup (#496)
new d59bc95 TE : sql schema fix, replace foreign keys with column indices (#498)
new 5082115 Added property for pinot-controller vip port (#499)
new b617b59 startree hll config and aggregation function in query processing (#440)
new 21642cc TE: updaing status for failed tasks (#500)
new 51306b5 Updated other instances of getControllerPort to use generateVipUrl (#501)
new 0700d09 [PINOT-3631] Upgrade to kafka 0.9 (#502)
new 0d336bd TE: fixing hibernate entities (#506)
new cb69064 Adding new entity layer (#508)
new 9538ee0 [PINOT-3596] Add segment time unit so that retention manager works for LLC segments (#507)
new 016a463 separate hll config and startree (#504)
new bc769b4 TE: fixing entity rename in query, marking an alert task failed rather completed if it fails (#509)
new 2b6be6c anomalies default view (#510)
new 0d9993c refreshing maxDataTimeCache more frequently (#512)
new 29d94b2 Add SchemaUtils class to provide schema related utils. (#503)
new 227b9ab Pull hll derived column rewriting to planing phase (#511)
new 3dbe323 [PINOT-3644] Remove watches on HL segments that are completed (#513)
new 434c152 Te anomaly change chart (#514)
new f93ad25 Fix for handing nUnitSinceEpoch datasets (#517)
new a274380 Changing default top value for pinot queries (#518)
new 13b1c17 TE: Anomaly-detection : fix for derived metrics (#516)
new c36ca82 TE: fixing entity delete for detached entities (#521)
new adb2670 Bump up guava dependency version to 19.0. (#520)
new 2f3fd07 TE: entity delete fix (#523)
new c00d41d turn on run adhoc anomaly with wow,wo2w,wo3w (#519)
new 7c76e6f TE: Fixing time interval for baseline based analysis (#524)
new 3db8cbc Handle broker request deserialization error (#522)
new 3fbb728 Bump up zookeeper version to 3.4.5 (#526)
new d7c7a4d Adding anomaly filtration logic for alerts (#527)
new 2b54d52 fix for property display on self service existing fn table (#529)
new 0575302 New entity layer for bean (#525)
new cebc8b0 te : datalayer cleanup (#530)
new 388ca51 [PINOT-3520] Added unit tests for LLRealtimeSegmentDataManager (#528)
new ad6708f te: fixing build issues (#533)
new 5ac3b29 Move Helix related method from Schema to SchemaUtils. (#534)
new 290333f Remove integration tests code coverage (#531)
new e35bf07 Te : anomaly severity recompute - fixing anomaly dimensions filter (#535)
new 75b0d5a Pretty print table REST API responses (#539)
new f5234c7 [PINOT-3677] Fix LLRealtimeSegmentDataManager to handle time-based constraint and numRows based constraint (#538)
new 2406f7a Refactor SelectionOperator related service. (#307)
new dad5889 Unify the package for Json related operations: Remove the remaining dependency to codehaus.jackson and move to fasterxml.jackson. This unification prevents the errors that are induced when processing Json objects using the mix of codehaus and fastxml's class, e.g., reading a json object of codehaus using an object mapper of fasterxml. (#544)
new 57f2e48 Replacing Java ser/de with custom ser/de in DataTable. (#532)
new 500e0e7 test cases for new entity layer (#540)
new c61b1c1 TE: not updating alerted flag if an anomaly did not qualify alpha beta rule (#542)
new 18ed9b1 TE: refactoring base test class (#548)
new 1477092 Bug fix in dao layer, id was not set when returning the entity (#551)
new 2d2b98c TE: fixing json ingore props (#554)
new 55b5823 Remove dependency on third-party HLL lib for segment generation without HLL (#549)
new 0a647a3 Added metric to catch any Java de-serialization in data table. (#550)
new 1701019 THIRDEYE-900 : End to end integration tests for distributed anomaly detection (#543)
new 496a6c7 [PINOT-3678] Fix the download path in consuming to online transition (#545)
new f7f91c2 Adding fix for handling missing data when grouping consecutive anomalies (#555)
new feef0ef hide off runnign adhoc and minor display changes (#553)
new c687def Reset the receivedStop flag after the thread is stopped, so that the catchup happens correctly. (#557)
new e67c6d5 Fix issue when creating StarTree segment with large dataset causing JVM crashes (#560)
new a6b9665 Trivial javadoc format change for SelectionOperator
new c47a22c Create a KafkaBrokerWrapper to use java reflection for kafka8 and kafka9 compatiability issue. (#556)
new 0943459 [PINOT-3691] Handle controller failover correctly in the server. (#561)
new 90f398a Fixing test cases with implMode=jdbc (#559)
new 152d270 TE : adding alert self service (#566)
new 50c8234 Fixing incorrect buckets shows in case of simple date format (#564)
new 7fb34c6 Make Avro to Kafka stream command configurable (#547)
new 1e93ce7 Re-architected server query execution to hand over query to the scheduler for execution. (#536)
new 5bfb24e Adding fix for bug related to anomalies showing incorrect 50% drop (#567)
new 21ed8df Add a new pinot-admin command ValidateConfig to validate table configs and schemas (#562)
new 3614e78 [PINOT-3696] Make segment commit time configurable. (#565)
new 2875234 Minor fixes to generic dao (#570)
new d28f795 te: refactoring timeseries data fetch (#569)
new a75d800 Adding timezone support through config, to ensure simple date format works on servers without default PDT (#568)
new e07f2d2 adding adhoc function analysis (#571)
new a3fdbbf TE: fixing timerange optimization (#573)
new f3c69ae Renamed USER_RULE to WEEK_OVER_WEEK_RULE (#574)
new c4b5158 Te anom thumbnails (#572)
new c4a802c Fix javadoc for nested list items
new 4d946dd [PINOT-3709] Fix ConsumingToOnline transition when segment creation takes too long. (#576)
new 54b6c86 Adding config driven support for implMode (#579)
new 10f02ef Fix for bug in email manager for jdbc impl (#578)
new 63d1985 Adding tool to migrate data from old tables to new tables (#575)
new 6d49781 Use existing IndexLoadingConfig from RealtimeTableDataManager for (#581)
new a33b4d6 Te anomaly fn params (#577)
new 90413aa TE: email config refresh after submitting (#580)
new e2548f1 fix for missing baselineVal and cuurentVal properties (#584)
new 648161d Refactor QueryGenerator and query comparison to support selection queries. (#310)
new 10d32d7 merged anomalies timeseries changing filter logic (#587)
new d060d82 Add unit test for SimpleConsumerWrapper (#479)
new c7dbfcd Add timeouts to SimpleConsumerWrapper (#537)
new c691e8d Fix compilation issue with Kafka 0.9 (#590)
new ee2610b In SchemaUtils, explicitly call releaseConnection() to prevent connection leak (#588)
new e1c9fc9 Increase simple consumer test timeouts (#591)
new 5f9496a fix for existing anomaly function query, for metric selector display, for anomaly time range display (#592)
new 3c02db2 adding error error messages for empty responsedata on contributors (#589)
new 3cec840 Enabling OffHeapStarTree by default: (#593)
new e78cd40 Adding fix for time zone conversion in anomaly detection jobs (#597)
new 1470bbc zooming on timeseries (#596)
new 7f1625b TE: Adding default value for window delay unit when a function is updated (#600)
new 95482b6 Add Swagger UI JSON editor support (#594)
new 375226c [PINOT-3466] Add metrics to get the percent of requests routed to LLC vs HLC (#585)
new bc8ccac Enable Controller failover logic (#598)
new 259cd1f TE : adding email migration script (#602)
new 4811da5 Replacing sysout with LOG.debug (#603)
new 80b1045 [PINOT-3205] Added validation to make sure that there is at least one segment in CONSUMING state in each kafka partition (#601)
new 592864b Bugfixing for fetching realtime segment metadata (#605)
new 04afb39 Adding utility for cleaning up and regenerating anomalies for datasets (#604)
new 7388bfc Modified Balanced assignment strategy to skip servers that are not (#607)
new 513798a TE: removing netty from transitive - inclusion (#608)
new 129b7b0 1. Handle null list of files while providing segment size. (#583)
new cb86dc4 Adding fix for missing first datapoint on tabular and timeseries (#612)
new b74ea50 THIRDEYE-905 : Write a cleanup and regenerate anomalies tool (#610)
new fc7f349 Adding check to validate connection from datasource (#614)
new 70b706c anomalies view aggregate granularity (#611)
new ef1fe60 Correctly set http status on controller restlets (#613)
new cb0d9b1 TE: reverting netty exclusion from pinot-api (#615)
new 92036c9 TE: removing hibernate layer and unused code (#618)
new 15b1da3 Replace assertEquals by assertTrue (#616)
new cf0b1da Add metrics to LLC consumer (#586)
new f706797 Fix to handle empty IN list (#621)
new 886dfea Update realtime consumers on index config upload (#619)
new ff27062 THIRDEYE-908 : Redesign how schedulers pick up and schedule functions from db (#622)
new 805f7a6 Adding fix for anomaly generation getting skipped in case of 1 granularity bucket (#625)
new 368d6a8 self service manage alerts (#627)
new 59594cd Adding timeformat field to Schema (#626)
new 965c7ae adding thirdeye admin tab with json entity editor (#628)
new 48fa01e Enable command-line tests to take an option to run tests using LLC (#624)
new 590858d fixing alert formatting for function details (#630)
new 316c9a7 Fix swagger-ui not working when running from JAR (#623)
new d899d7d Fix unit tests to load segments from correct directory (#620)
new dd71693 cleanup flot.js, move utility.js, get-form-data.js, handlebars-methods.js to commot, removed unused css classnames and not rquired todo notes (#631)
new 2e9b993 Delete old format files after converting segment to v3 format. (#609)
new a47717d thirdeye webapp improvements and cleanup (#632)
new 7dbb5e4 THIRDEYE-910 : Metric as a first class citizen of Thirdeye (#634)
new bca3a4a anomalies thumbnails function info (#635)
new 5e53a05 Right now we keep all the segment files to data directory.This contains a lot of tmp file when this table is uploaded with url only protocol. (#636)
new 8109c38 Support aggregations and aggregation group by on multi-value columns (#599)
new 5c67617 Adding thirdeye admin tab with json config editor (#638)
new a6d6b72 [PINOT-3743] Remove all traces of LLC when a table is configured with only HLC segments (#637)
new 2e88a22 Add routing options to BrokerRequest (#629)
new b2f5217 Use maven batch mode for Travis (#617)
new 03bdcee optimizing te workers to pick task (#639)
new 5700acb Add additional LLC metrics (#642)
new b72a702 Adding a tool to migrate all datasets and configs to new design (#643)
new bbea840 Metric centered dashboard
new 44c3b97 Add debug-option to enable/disable using star-tree per query. (#647)
new 6f7562f Vertical multidimensions drill down on time series and anomaly detection (#641)
new 1714b2f Added a utility to reduce number of replicas for an offline table (#646)
new e241efe TE: adding fix for failing tests, Changing feedback type on anomaly ui (#644)
new d70a99e Metric centered dashboard changes
new 72bce86 Replace committed segment in the data table (#650)
new e28da4a Metric centered dashboard changes
new d8ef3ef Adding fix for delete overlapping segment logic (#652)
new f36ecb5 TE: adding anomaly detail page (#651)
new b7e9b24 Fixing known anomalies filtration (#657)
new 3c03d54 fixing ui charts for minute level granularity data (#658)
new 21977f3 Handling race condition among workers (#660)
new 163ddaa te: fixing time range in the email (#661)
new 75c95cd Metric centered dashboard changes
new 4ad2858 Remove log message that warns on Java ser/de. (#662)
new 82c5904 Add README link to BOSS 2016 (VLDB) presentation (#649)
new 6770a5e Fixed heatmap cell size for cells whose value drops to 0. Fixed percentage change of heatmap cell whose value raise from 0 to any value. (#640)
new ea1eccb Exclude pulling in netty from org.jboss.netty group (#664)
new b13e616 Added more details to query context to track query specific information including timers. Updated code to use TimerContext from query request to record times for different phases of query. (#606)
new 78141ed Pql for nonadditive (pre-aggregated) data (#659)
new 328d9fb TE: adding anomaly funciton filter/dimension details on the anomaly detail ui (#665)
new 3e89bf6 adding additional log info (#667)
new 50ad3b4 Remove the shade under anomaly charts. (#668)
new 5c5de06 Look for metadata file in different versions of segment directory (#645)
new 9e5d3a6 fixing aggregation ganularity on UI (#670)
new fc7b3f2 Extend Anomalies Tab for Supporting Non-additive Dataset (#666)
new f8f341d Dashboard refactor
new 75b6f47 anomaly function api fix (#671)
new 0a632ca Fix the following anomaly chart bugs: 1. baseline and current values in different scales. 2. The rounding setting that induces weird numbers in the chart. (#672)
new 8d4d2f3 Fix anomaly detection keeps working on the first monitoring windows since it is started. (#674)
new e46f0cb THIRDEYE-935 : Add support for simpleDateFormat in thirdeye-hadoop (#677)
new 4cb12f4 Metric centered dashboard
new fa3d75d Adding caches for metric, dataset and dashboard configs
new 0518c58 Setting timeunit explicitly in case of simple date format, instead of relying on config, so that config can be used to construct correct schema (#679)
new 0bc4d04 removing duplicate code (merge in WoW Function), adding some js improvements (#680)
new 39a4983 Using dashboard, dataset and metric caches to speed up queries
new 5200001 Fix detection job runner for adhoc detection. (#683)
new d71ba69 storing js libs locally (#685)
new f19d041 saving jquery lib locally (#686)
new d037b63 Fix display of decimal numbers on anomaly charts (#687)
new ad77c2b Fixes from comments
new 5d77e0e Fix the issue where the loader (spinning wheel) locks out the entire page when it is loading. Now the loader only locks out the loading area. (#688)
new 64186b3 Fix query issue for non-additive dataset, which throws java.util.ConcurrentModificationException (#689)
new 8e23062 Improve load time of anomalies page by 2x (#690)
new 43bbec8 Fixing end to end test
new 68b966b Indexed Dynamic Priority Queue. (#694)
new 82f2d06 Merge branch 'master' into metric_centered_dashboard
new 10f528a Merge branch 'metric_centered_dashboard'
new 8678561 Merge branch 'master' of https://github.com/linkedin/pinot
new 2a22521 Minor fixes to metric centered design (#697)
new 85215d9 Avoid join() with same thread on segment commit (#693)
new 2f27895 Generate timeseries for anomaly according to its function (#698)
new d89b2dc Adding ability for load metrics tool to read from collectionSchema in absence of pinotSchema, adding whitelist (#700)
new 00047a7 Adding mode in regenerate anomalies tool to run for every consecutive window in range, instead of the whole range (#701)
new 1cb9b44 Fix NPE and return 404 if schema is not found (#696)
new 2f6e34a [PINOT-3882] Bumping up zookeeper version to handle large number of watches (#695)
new 9452494 Endpoint to view raw anomalies by functionId or dataset+metric (#703)
new bb2d390 1. Generate unique key using window start time for adhoc detection jobs. 2. The monitoring window slides the distance according to the cron specified in anomaly funciton. (#705)
new a64de7d Adding fix for heatmaps breaking in case of derived metrics in metric centered dashboard (#706)
new 3da432f Added metrics for number of tables and IS znode size. (#699)
new 3a3a0cf Fix for issues handling special UTF-8 characters. (#704)
new d8ba3ae Remove anomalys dependancy on dimension key (#708)
new 19e7ac7 Fix the filter set does not show on frontend; remove the unused method that confuses model mapper. (#709)
new 0bf391d Do not include recently pushed segments in calculating number of replicas (#692)
new 9fcaa5a Add pause when waiting for EV change (#682)
new a9f4d9c Tool to measure performance of lookups on String Dictionary. (#710)
new 6a01234 Quick bug fix: Temporary disable adaptive baseline values in anomalies tab (#715)
new 3e34082 Disable helix flapping connection detection on server (#713)
new af23ca0 Handle errors in Dynamic Broker selection (#717)
new 1b1c220 Removed legacy PQL compiler leftovers (#716)
new f95bc35 Eliminate flaky behavior from Indexed Priority Queue unit tests. (#718)
new 170e579 1. Fix memory leakage due to unclosed PreparedStatement and ResultSet. 2. Fix the end time keeps expanding after each iteration of generating the set of tasks for an anomaly function. (#721)
new 9a99c6b Updating http client version (#722)
new 25e4774 Adding rest end point for ingraphMetricConfig (#711)
new ab19e5f Limit number of groups in Aggregation-GroupBy query results. (#676)
new 346ae13 1. Replace dimension key with dimension map in logs 2. Fix broken tests 3. Fix alert runner handling empty anomaly results (#724)
new 31d0b2a Add table config-based routing table selector (#702)
new 24c0ce7 Minor FilterQueryTree improvements (#720)
new 67545aa Refactor request optimizer to be more flexible (#726)
new 528d38a Fix direct memory usage tracking. (#723)
new 254d96d Add max weight limitation to ResultSetGroup cache (#729)
new f5fe143 Fix time series of OTHER dimension always show 0s in anomalies tab (#727)
new b674498 Add passive refresh for max data time (#730)
new 3a4b34d Adding tool to automatically load new datasets from pinot into thirdeye (#733)
new 2ab1384 Redesign monitor to not generate many tasks per run (#735)
new 45c3c1b Minor renaming for autoload service (#737)
new 66a6552 Presentational timeseries for anomalies tab (#736)
new 0892999 Move views.properties from detector-config to webapp-config (#738)
new 9454abe Add multiple or equality predicate optimization (#728)
new 98b5128 Handle incorrect pinot schemas gracefully (#739)
new 34d9da8 TE: Create multi-dimensional summary for week-over-week change of metric (#317)
new a2df0f6 Fix exception in test (#740)
new 408849a Converter for Columnar segment to Star Tree segment. (#743)
new 9efc31e Bugfixing for PINOT-731: zk path parser in broker starter. (#732)
new f316066 Adding end points to configure metrics,datasets (#744)
new 6e4139e Fix dimension map compatible issue (#745)
new ab7d0ee Adding support to configure derived metrics (#747)
new a797812 Add internally constructed alias for MetricConfig (#748)
new 832dae3 Minor fixes to dervied metric create/update (#750)
new 7b0feac New changes for ingraph onboarding (#752)
new ba040b9 Add tools to convert Pinot segment into another format. (#749)
new 2b61471 Fixing sql schema index sizes (#753)
new c7a2a9b Optimize tests for selection queries. (#707)
new 610e6c4 Adding endpoint to configure external datasources (#756)
new a5371a8 Fix the presentational time series for OTHER dimension (#760)
new 910ccd7 Adding webhdfs client to upload segment through webhdfs api (#746)
new 1cc9393 Remove spuriously emitted LLC routing metrics (#758)
new c621a9a Add optional clause sanity checks to SELECT (#734)
new 4f9434c Improve query engine benchmark setup (#742)
new 1d804e3 Minor fixes to thirdeye-admin (#759)
new 8490641 Allow changing optimization settings per-query (#741)
new 986e780 Populate correct known anomalies to anomaly detection functions (#754)
new 30e39ed Add backfill function to detection scheduler (#757)
new e06e889 Modify autoload tool to autodiscover metricAsADimension case metrics (#761)
new 26c7d7e Remove trailing nulls ('\0') from string values during indexing. (#755)
new 7967e32 Minor ui fix (#764)
new e3bff7d LLC routing table selector fixes (#763)
new 0fded25 Update default dashboard everytime a metric is added or deleted (#762)
new 670d77d Build an executable jar from QueryRunner for performance testing. (#766)
new 35cb8d9 Revert automatic addition of metrics to default dashboard (#769)
new 63225fd [PINOT-3999] Fix file collision when multiple LLC segments get downloaded in the same msec (#770)
new 27140ec Enhance merged anomaly calculation (#772)
new cb4927f Load creation metadata from the index directory tree (#767)
new b607ec2 Fix sequential gap between merged anomalies (#774)
new 65775a0 Add log for debugging missing buckets (#775)
new fbac029 Move Pinot server APIs to jax-rs (#751)
new f145bdf LLC routing table ratio bugfix (#768)
new 544e1b4 Making Porjection/Aggregation/Group By independent of Datasource/IndexSegment classes (#773)
new 4520789 Refactoring anomaly results page (#780)
new 553d576 Remove unused classes (#779)
new 078456e Fix broken tests for merging raw anomalies (#783)
new 4b228b3 Fix random seed in flaky test (#786)
new ba234e0 Add ExecutionStatistics into Operator to track the execution statistics (#771)
new 1c35a1a Optimizing anomaly results pageload (#787)
new 272c008 Refactor anomaly merge logic and utility endpoint for computing anomaly weight (#785)
new 10387f2 TE: Fixing build failure caused by merge (#788)
new 73bca05 Enabling anomaly details page (#789)
new fa2e85e adding more config object under json entity editor (#791)
new 41775d4 starting cache loading asynchronously to reduce application start time (#792)
new d781387 Minor update on detail anomaly tab (#794)
new 029862d Adding externalDashboardURL field to metricConfigBean (#793)
new 030a451 Performance improvement for StarTree index generation. (#776)
new 1ef38cc Serialize/deserialize idealstate when copying (#781)
new d0b7cd2 Populate external url template in the metric for merged anomaly (#798)
new e3fd7ab Fix the calculation of weight for week-over-week anomaly function (#801)
new 9cc1d49 Split Travis tests in two parts (#802)
new 8210882 Use Travis cache for builds (#803)
new befec9d Add metric for routing table update time (#799)
new faf3dcb Add API to sync brokerResource with Helix tags (#782)
new 341dc1a Display test execution duration (#804)
new 807449d Use explicit jersey package versions instead of dependency management (#800)
new 28ab0ae Fix log messages and exceptions seen during deployment (#777)
new 4e3fdd8 Changing externalDashboardTemplate to a map of sourceName to template (#805)
new 8745a97 Input validation in schema builder (#796)
new 1d6cc50 Remove references to _OFFLINE in pinot datasets, to ensure realtime also works (#807)
new f15ae57 Reduce anomaly detection latency (#808)
new cec341d Fix the issue of duplicate alert (#809)
new f459db8 Fix Exception when getting a schema for offline table (#806)
new 385b563 Update information of merged anomaly only when there are multiple raw anomalies (#811)
new a7b5a59 Don't derive TableSizeResource from restlet server resource (#813)
new e330099 Starting template for new UI (#814)
new 345bd8f Clean up old aggregation related operators. (#810)
new 1e5a98e Added Table views API to read external view and ideal state from controller (#797)
new 70a172d Remove pinot files from cache even tests failed (#815)
new 27fbeb3 Fix IndexOutOfBoundException when doing group-key trimming on non-comparable objects. (#821)
new 1252b86 Add custom setting of alert filter for metrics (#818)
new 05588a9 Fix path name used for deleting expired segments (#820)
new 05aa54f Fix realtime segments to not build inverted index on sorted columns (#823)
new ca60753 Add PerfBenchmarkRunner as a tool inside PinotToolLauncher (#817)
new 8fa003e Exclude jsr311 lib from swagger jersey2 library (#819)
new f5ab863 Fix non-server components do not pick up temp directory (#824)
new 5e4d9b8 Add support LLC-specific config overrides (#822)
new fbc9007 Added server REST apis to read tables, segments and segment metadata (#784)
new 6bda7b7 One more stab at fixing the dependency hell (#825)
new cf364d3 moving entity editor page under thirdeye-admin (#812)
new 69432fb Pin jersey-media-json-jackson to 2.24 to address build issue (#826)
new 9d6d2cf Cleanup unused code. (#831)
new 237c0a8 Add metric for allocation failure count (#829)
new 85f2416 Add metrics for time spent and timeouts in broker waiting for a connection. (#833)
new fb0d999 Add retry to segment push. (#765)
new 1e90637 Move alert filter from metric level to function level (#828)
new 819d29b Reorder addition of pinot table views API (#835)
new 03698b7 Remove hardcoded Thread.sleep() in unit tests (#834)
new 54defa2 Remove unused local variables (#832)
new 9139dfc Added server api to read segment metadata (#830)
new 9e0b38d Routing table rebuild optimizations/rewrite (#795)
new ae6b904 Handling DST issues (#838)
new 4d073fc Inside CombinePlanNode, fix plan node timeout check. (#836)
new 06150c6 Adding analysis ui page (#837)
new c674c54 Handle removing index for V3 format silently. (#839)
new 6dc324d Adding link to external dashboards on anomaly thumbnails (#827)
new 56cbef6 Adding changes to include reports group by metric/dimension in email alert (#841)
new e1603b8 Fixing anomaly feedback update, future time offset removal from line chart, anomaly weekly report generator script (#843)
new 3197cda TE: use scaling factor to rescale metric before anomaly detection (#840)
new 9291aca Changing email subject to bring metric name first (#844)
new 7b8b69c Adding js files (#847)
new 0f2e6d5 [PINOT-4124] Update idealstate correctly when adding a table to brokerResource (#845)
new 5c0c8a2 Add database of override rules for holidays (#846)
new 2e1b5cf fixing line charts (#848)
new 254f799 Fix the error message for difference summary when summary cannot be generated successfully (#852)
new f46e494 Add endpoint for CRUD of override configs (#855)
new 7dc6368 Adding support for history and hashbang (#851)
new d283776 Adding custom anomaly report generation tool (#853)
new 47f8e94 Add scaling factor logic to detection and merge task runner (#856)
new bc9a8cb Get the URI debug/memory/offheap to work (#857)
new 0ba685c Minor changes to ui-styling (#860)
new 37260d3 fixing dimension report alert (#862)
new cb866d8 Fix DST problem in TimeOverTime, Heatmap, and TimeSeries view (#859)
new 83b3512 Fix FastHLL throw NPE. (#849)
new 4839e0e Re-design Projection operator to perform actual projection. (#850)
new 484d23f Enhance the exception type in PqlQueryResource (#864)
new 0bfc8d8 Allow creation of segment without starTree when HLL columns are providedx (#863)
new a989780 [PINOT-4062] Auto-create segments in CONSUMING state if we do not see one in a partition (#858)
new 1c88832 Use Kafka topic/id/zk as consumer manager key (#866)
new c3d72d7 Add configurable connection/fetch timeouts (#842)
new 3c74305 Restore perf benchmark log level to info (#816)
new 9e24911 Change holiday scaling model: Apply scaling factor to baseline (#867)
new 0c24c01 Adding flag to handle external metrics not part of a dashboard (#872)
new 83620da adding DataResource with empty method end points (#873)
new a559497 fix PerfBenchmarkDriver postQuery (#870)
new 9132139 Setting default fabric group options in admin ui (#875)
new 350677d adding more methods in data resource and some alert template fixes. (#879)
new 381c11a [PINOT-4171] Support more readable ways to enter segment consumption time (#878)
new bd954b2 Adding autocomplete plugin (#874)
new cf3f085 Add support for per-segment realtime flush size (#884)
new 4ea13bf refactoring alert template to include report data with link to dashboard (#885)
new 4807a2f Adding fix for default rollup thresholdon admin (#883)
new 14e63ce Add override config to entity editor (#886)
new 20e4a88 Wrap Kafka protocol exceptions (#882)
new d24a6d2 Add codecov badge to README (#888)
new fd5f24f Create shaded jar for pinot containing all dependent jars. The shaded jars will (#876)
new 2d46f1e Extending dimension summary report to include total and share columns for all sub dimensions (#891)
new 0a953c4 Fixed incorrect compare on errors during consumption (#890)
new cedfdfe Fix auto-creation of segments (#889)
new 7dcaa7e Set default supported segmnet format version for servers to v3. (#881)
new 23def50 Add QueryRunner into PinotToolLauncher (#887)
new 72bfca4 Enhance broker side exception handling. (#868)
new 5461e9e Add partial result broker metrics (#877)
new 14f157d Make toString on segment names work as expected (#894)
new c36a5b5 Optimize get string without padding. (#895)
new 2900e14 [PINOT-4169] Part-1 of auto-fix for non-consuming segments (#892)
new 59ece38 Support configuration of aggregation function arguments (#871)
new a15c624 Refactor: Modify BlockValSet interface to support filtered as well as unfiltered (#896)
new 86f7e85 [PINOT-4169] Part 2 for auto-creating non-consuming partitions (#898)
new 405adcf Controller delete segment will directly drop segments from ideal state (#900)
new 5594772 Remove delay between deferred routing table updates (#897)
new 7fd0084 Greatly reduce object allocation in realtime path (#880)
new 1d32bfb [PINOT-4167] Auto-update LLC Kafka partition assignment if any parameters change (#903)
new 4e3f618 [PINOT-4172] Fix LLC segment cleanup when conumer type is changed (#905)
new f6a0c27 Make the wait for a server connection as a phase. (#901)
new c5f925a [PINOT-3788] Stop consuming when realtime segment is full (#906)
new 5b1403f Enhance SelectionOperatorUtils/Service to allow rows with type compatible data schema to merge. (#902)
new 1b1ede3 Fix the typo of 30000 seconds timeout in combine group-by (#909)
new 8cb6e48 Add backward compatible for the new selection only logic. (#910)
new 6147cb0 Fix time boundary sometimes not being computed (#908)
new f5bd62a Add routing deferral latency metric (#904)
new 4c3238d Reuse GenericRow instances in realtime indexing (#907)
new dd4e3b0 Add a metric that reflects the highes offset consumed by a server for a partition (#911)
new 5933627 No-dictionary indexing support for single-valued string columns. (#899)
new eac7d18 Append OFFLINE/REALTIME into the server name stored in ScatterGatherStats (#912)
new de876b5 Group By support for columns without dictionary. (#913)
new 4409627 Fix PerfBenchmarkRunner main method (#916)
new cafcf6d Enhance QueryRunner to report statistics for the current report window. (#917)
new 99c7b6e [PINOT-4244] Handle kafka exceptions during consumption (#915)
new 1a56786 Set the flush threshold on a per-segment basis (#893)
new 04693ff Cleanup: Fixed in-efficiency in multi-value dictionary fetch. (#921)
new d35cc3e Fix closing of index without dictionary. (#923)
new 79fef91 Add tests that covers all aggregation functions (except FastHLL) (#922)
new 993756e Old aggregation function clean up (step 1) (#919)
new dcdc418 Clean up the System.output inside the tests. (#675)
new e726728 Fix VarByteSingleValueReader throw IndexOutOfBoundException when all values are empty strings (#928)
new c6f8ad5 Trim trailing nulls in ForwardIndexCreatorTest. (#927)
new 637fb51 Added a broker debug endpoint for getting the timeboundary for a given table. (#924)
new afbecc3 Te entity editor improvements and daily report scheduler (#920)
new 24a4d40 Remove dependency on pinot SegmentNameBuilder for segment name generation (#929)
new df297ad Remove deprecated methods (and their usage) from SegmentNameBuilder. (#925)
new 0d0cbcc Clean up old aggregation functions (Step 2) (#930)
new 04d7eaa Fix aggregation group-by garbage collection issue. (#933)
new 1f37fa7 Add raw indexing (no-dictionary) support for single-valued primitive data types. (#932)
new e159a6b Add routing table builder for large clusters (#918)
new a3cdc6c Clean up old aggregation functions (Step 3) (#934)
new ddc2110 Add data directory availability/latency metrics (#937)
new 19ae3ed Enable large cluster routing for large clusters (#936)
new 74a8c2a [Pinot-Controller] Fixing the segment delete method so that if the deleted segment already exists, we overwrite it. Previously, it would throw a FileExistsException. (#931)
new 4760de7 UDF and Expressions support. (#648)
new 9573627 Fix handling of transient kafka errors in the server (#941)
new c1f793e Add table quota configuration as part of table configuration json conversion (#940)
new 8e53fbc Adding log messages during validation run (#943)
new 7965c69 Refactor: Change AggregationFunction api's to take BlockValSet instead (#942)
new b220f13 Adding new AlertConfiguration, API for sending adhoc anomaly report emails. (#945)
new f221a47 Make v3 format as default segment format (#938)
new 92711de Remove old logs that made it back due to merge conflict in 4c3238d (#946)
new 225ee76 Support parsing of expressions (udf/transform) in groupBy clause. (#949)
new 5e222ca Add configurable RT flush sizes in tests (#947)
new 70c8f1b [PINOT-4312] Fix NPE in ValidationManager (#952)
new c104452 Fix the method to search for file paths across different segment formats (#954)
new 0cca9a2 Fix typo in method argument name (#953)
new 4790740 Log segments with no replicas. (#950)
new bfa4c70 Move transform related interfaces from pinot-common to pinot-core. (#957)
new c3903d8 Skip writing null storage quota config to table configx (#955)
new 5b1ec33 Updating the validation manager to rebuild broker resource (#951)
new b47edfb Revert "Updating the validation manager to rebuild the broker resource if we …"
new 137a841 [PINOT-4325] Fix auto-correction logic for consecutive failing segments (#956)
new fa887ee Cleanup: Remove ProjectionColumnBlock class. (#960)
new e1a53b5 Updating the validation manager to rebuild the broker resource if we add a new broker
new 5a1e8e0 Fixing unit tests
new 26785a5 Thirdeye add onboard endpoints in dashboard resources for anomaly fucntion cloning (#961)
new eb500d1 Re-factor auto-creation of segments (#965)
new 4d0d5f0 Move DataTable related classes to pinot-core + Old aggregtiaon function clean up (Step 4) (#939)
new 077cad5 Fix the problem of retention manager not pick up the latest table config (#962)
new cbd63de Adding AlertV2 module for thirdeye (#964)
new 3b3c6b1 Log an error and update metric when zookeeper updates fail (#968)
new b9f60fc Old aggregation clean up (step 5) (#944)
new 04f3b97 Added MoveReplicaGroup command to pinot admin commands (#966)
new 011302d Data completeness checker skeleton code (#969)
new 700b64c Update broker resource rebuild endpoint (#970)
new 5900baf Add internal API to get the service status (#963)
new bab738d Adding Alert V2 improvements. (#973)
new db401f6 Add UI for Spline Regression anomaly function (#975)
new 0992ba7 adding compare mode / color coding etc for custom data reports (#978)
new 51783e3 Adding type parameter to delete segments logic (#979)
new 392d8d5 Add Transform support into query planning and execution phase. (#967)
new 1d8b016 Merging new thirdeye ui to master (#980)
new 41ea24b Package re-arrange: Move new aggregation related classes from operator to query (#984)
new 92f9504 Refactor rebuilding helix entry (#983)
new d5188e7 Generate v3 segments by default (#981)
new 57dac2b Add explicit dataset suffix to queries, to avoid realtime being queried by default (#987)
new 0636938 New Framework for Anomaly Detection Function (#974)
new c778cdd Handle UNKNOWN_TOPIC_OR_PARTITION as a temporary exception instead of permanent. (#985)
new 14f5a7b Remove cruft left behind if commit fails (#986)
new b03da9e Refine wrapper class for new wow anomaly function (#993)
new 025b4b0 Bugfix to handle broker resource state transitions to offline or dropped (#990)
new f74346e Optimize logic to refresh watches for realtime table and fix error handling (#988)
new 040ece8 Cache totalDocs and totalRawDocs in SegmentMetadataImpl, instead of (#989)
new d68f4b9 Limit number of servers hit per query for LLC (#991)
new 1663801 Fix scatter gather stats (#977)
new 5ca7ffa Self-Service UI (#997)
new 7616530 Eliminate getSimpleName() call for each operator during query execution. (#996)
new 6396242 Data completeness checker module (#992)
new daa3ac4 Add ModelProvider interface and ZeroRemover transformation function (#999)
new 4870731 Changing the list of datasets used by checker (#1001)
new 694fb66 Fixing broken end to end test (#1002)
new 4e369a2 Add configurable thresholds for routing tables (#995)
new 8544ccf Remove redundant data member from BaseOperator class. (#1003)
new 18b4731 Prefetch mmap data during segment load time (#1000)
new a79a8c3 Add additional metrics around server query performance (#994)
new f462737 Fix service status NPE (#1006)
new e9871a2 Enabling intraday reporting on thirdeye (#1008)
new 46afe88 Apply alter filter when showing anomalies on ThirdEye UI to improve UX (#1009)
new 5d56b36 Fix routing table unload from broker (#1007)
new 6fdb0a2 Support prefix when creating tar.gz files (#1011)
new ba34e66 Minor bug fix to new WoW anomaly function (#1012)
new 592555b Fixing segment list API (#1010)
new d4222e6 Log exceptions and safely close stream while creating tar.gz files (#1013)
new 9c22392 Remove old WoW anomaly function and AnomalyTimeSeriesView (#1017)
new c6b7693 Fix DST issue in query result parser (#1018)
new 00100ac Add a tool for fetching thirdeye data, and transform to CSV file (#1005)
new 8fd6fd3 [PINOT-4406] Optimize idealstate updates when deleting segments from same table (#1014)
new dba9849 Make the search window of merge anomaly end-exclusive (#1024)
new e7b22c6 Added exception to log message if controller fails to find instances for new segment (#1021)
new 14dcb34 Disable prefetch for zero-length pages (#1026)
new 0f1cc86 Integrate optional completeness check before running anomaly detection (#1019)
new b6ef259 Adding EventDataProvider API interfaces (#1028)
new 49ec1b8 New Input Parameters for Spline Regression Self Service (#1030)
new 9290416 Segment Deletion API (#1029)
new 2178667 Filter optimization to avoid unnecessary predicate evaluation. (#1027)
new 02d7480 Yyuen/hash service fix for back button (#1020)
new 1100f36 Fix brokers reporting STARTING for a long time (#1025)
new d4c6e28 Disable anomaly detection on OTHER dimension (#1031)
new 1e988af Refactor DAORegistry for testability (#1033)
new 7719bee Retry schema fetch on failure (#1034)
new abf5b83 Adding db access classes for external events (#1035)
new dbc4d86 [PINOT-4446] Revert back to exponential back-off and introduce locks to serialize idealstate updates to same table (#1036)
new 85e212f Cleanup: Remove fieldspec dependency from mutable dictionaries. (#1038)
new 0b82ca0 Adding holiday event data to thirdeye (#1039)
new 7bdf385 Fix bug handling negative hash code (#1041)
new a92a227 Onboard/Replay End-Points (#998)
new 367ecf1 ThirdEye: Test suite reset to base state (#1040)
new 09d8547 Add AnomalyMergeConfig to AnomalyFunctionBean (#1044)
new ae0f8f6 ThirdEye: make completeness check fully depend on dataset config (#1037)
new 9009cd6 Yyuen/root cause analysis link (#1045)
new 870a16e ThirdEye: Multi metrics anomaly detection (#1048)
new 9260696 Add anomaly filter to new UI (#1052)
new 7908c75 Fix codecov ignore coverage for certain files (#1054)
new 06a45ba Auto-fix segment consumption if idealstate update fails during LLC segment completion (#1049)
new 87b0760 adding holiday event api (#1060)
new 8c55194 [PINOT-4491] Re-throw exception when we fail to get a schema (#1057)
new 3fbe684 [PINOT-4425] Controller should not assign segments to disabled server instances (#1047)
new 52af617 Hookup FixedByteChunkSingleValue indexing. (#1059)
new 4d1312d Calculate percentage of replicas which are up and emit a statistic (#1065)
new 7078ca3 Performance improvements for no-dictionary group key generator. (#1050)
new 3a0ac29 Yyuen/anomalies tab styling (#1058)
new 6f171ff Add avgBaselineVal and avgCurrentVal to MergedAnomalyResultDTO and RawAnomalyResultDTO (#1061)
new 8763f8b Yyuen/root cause analysis time range logic (#1067)
new cedbbb4 Introduce multimetrics to new anomaly function framework (#1069)
new a8de4d4 Implement total count threshold transformation function (#1070)
new 38d29e7 Handle case where there are no replicas in ideal state (#1073)
new be476df Bump up jackson version number to fix method notfound during runtime (#1074)
new 41c5838 Bump up jackson version for fixing build (#1076)
new 1f1dd0b Switch to first come first served scheduled query executor (#1072)
new 0d08bfe Changing segment naming (#1071)
new f6108f3 Fix the bug where trailing empty strings in group key gets ignored (#1078)
new 981a87a In Pinot Broker, use a separate SPECTATOR to process non-instance messages. (#1077)
new fe6c0f6 Alert filter autotune (#1062)
new ead9223 Deduping OTHER dimension from pinot response into TE (#1082)
new 5e2d847 Enhance message parser in anomaly wrapper generator to handle misformed message (#1083)
new 0e37e81 Adding event counters as metric reported through jmx (#1079)
new 75d04b7 Decoupling sequence Id and Postfix for segment naming (#1081)
new 0dd1da8 Compute maximum number of replicas from the zookeeper state instead of (#1080)
new 1bcb7f1 Yyuen/new card design (#1084)
new 39edc8a [PINOT-4358] Added code to remove old LLC segments that are marked IN_PROGRESS but in OFFLINE state (#1087)
new 023ed76 Add metric for percent of segments with replicas. (#1085)
new ad6ee97 [PINOT-4432] Upgrade helix to 0.6.7 (#1091)
new fb681e2 Fix the bug that group by metrics can pass startree check. (#1090)
new afce054 Filter optimization to merge range predicates on time columns. (#1086)
new c4bf229 Improve summary algorithm (#1092)
new b73bcdf Enable thirdeye setup new parameters in SplineRegression (#1088)
new c7ba842 Fix a typo inside RangeMergeOptimizer (#1094)
new deadda9 Implement an AlertFilter with up and down threshold (#1063)
new 424a28e fix alert filter testing to use TestNG annotations and allow config dir injection (#1089)
new 8fa3d97 Support controller configuration to add minimum replication factor for new tables (#1095)
new 1f84fd4 Provide config to disable table level metrics for server and broker. (#1093)
new d3a7200 Adding longer timeout (#1098)
new cc8db1f Fix the hybrid cluster integration test and change back the sleep time (#1101)
new 21a2f7e AlertFilterTaskConfig (#1102)
new fa24675 Add support for Server side Time segment pruning. (#1099)
new 2dbd2cf Add table level configs. (#1100)
new 1c0f576 Yyuen/import front end plugin (#1103)
new 534ff56 Fix test failure due to arbitrary test order of TestNG (#1105)
new ad75a89 Return valid empty response if all segments got pruned (#1107)
new a17f675 Temporarily disabling TimeSegmentPruner. (#1111)
new 3a6d9f7 Scheduler enhancements (#1104)
new 49a0273 upgrading front-end-maven-plugin (#1112)
new 13fb43f Fixing test failures (#1114)
new f15963d Migrate min max anomaly function (#1110)
new b6116c3 Adding anomaly event provider (#1115)
new 1b6ef1c Add Travis Support to ThirdEye (#1116)
new 4ffbae4 Adding explicit timezone information to test (#1117)
new e2f1651 Fixing minute format for data completeness checker (#1118)
new cb93e9c increasing threadpool size to improve anomaly search time (#1120)
new 0630566 ThirdEye Search (New Design) (#1119)
new 697d28e Autotune Endpoint Tool (#1096)
new b5aa688 fix pagination speed (#1122)
new 089e5c8 Refactoring backfilling method according to scheduler enhancements (#1123)
new bdb7e8e Adding informed query util class to query informed API (#1121)
new 7d1c7079 new email template (#1124)
new f975fcb Fix NPE and log message when rebuilding broker resource (#1125)
new 1718dc4 Handle tables with no ideal state in SegmentStatusChecker. (#1113)
new 7b996f4 Alert Filter Factory UnitTest (#1127)
new 141c580 Remove old min max anomlay function (#1126)
new db3b199 Enable DetectionJob to send back job id (#1129)
new 604125a Yyuen/investigate page (#1128)
new 1439f4d Add a util to auto add inverted index (#976)
new 7cacc69 Update merge models to compute avg current and baseline (#1132)
new 934245b Implement FunctionAutotuneConfig table and its required class and test points (#1133)
new 87c38a1 Use the smallest value between current and baseline bucket number (#1137)
new 78672e1 Fix the bug where temporary segments are not properly cleanned up (#1136)
new cc6fa1d TE: New UI/email template (#1139)
new ba64d34 Add server side column min/max value pruner (#1135)
new 7abdb18 Refactor anomaly detection framework for adding grouping logic (#1143)
new b88dbcc Extend usage to alert filter autotune (#1141)
new 69a6b98 Implement Synchronous Backfill function
new 6b8b8c6 [PINOT-4603] Re-factor parameter passing in Segment Completion Protocol (#1142)
new 7906fed Disable cleanness check for anomaly result regeneration (#1146)
new 6a82de4 Add min/max value to TIME column for local test. (#1148)
new 1843f90 Adding error handling and graceful continue in case of exceptions (#1150)
new 765f3e1 Fixing bad condition in data completeness checker (#1151)
new f4e8fd5 Update merge anomaly info even it has only one raw anomaly (#1153)
new b249c35 Initiate AlertFilterAutoTune Interface (#1152)
new dc7c51c Remove casts to PlainFieldExtractor (#1156)
new 9a95b5d [TE] Yyuen/post enchancements for launch (#1147)
new 5a520a4 Preallocate realtime segment sorted iteration order (#1158)
new 0ddb04e Refactor segment creation stats gathering (#1157)
new 17bb739 Added PUT request to modify table configuration (#1155)
new ce8e25e [PINOT-4603] Extended the protocol with the lease message (#1149)
new fc014be Add TableRetentionValidator to validate the retention policy in table config, and the start/end timestamp in segment metadata (#1154)
new 4a361a6 Fix IndexOutOfBoundsException while generating minmaxvalue for empty segment (#1162)
new 327df89 [TE] Yyuen/view contribution links to old ui (#1160)
new fa6b541 Send anomaly history and scaling factor to UI display method (#1161)
new f0c5c3e Fix out of range exception when input cube is too small (#1134)
new cd7ceae [PINOT-4637] Configure the average number of multi-values per row. (#1165)
new 2a55305 [TE] Yyuen/last changes for launch (#1167)
new 4f6f7b4 Fix the bug where default column handler throw exception and prevent segments getting loaded (#1166)
new 36c7ffb Disable test for checking completed jobs in end2end test (#1169)
new 19b4dac Fix null exceptin issue when updating merged anomalies (#1168)
new 6afa4fd route logo to search (#1170)
new 1a226bb TE: Improve new UI time series plotting (#1171)
new 86065ba Email template changes (#1163)
new 983b80f Improve anomalies timeseries view (#1172)
new 3b888de Stats gathering spring cleaning (#1164)
new 9734abd Refactor code to compatible with Java 7 syntax (#1173)
new 8efbe51 [PINOT-4603] Added server side code for extending lease for build time (#1174)
new 877e9ab [TE] Fix thirdeye-hadoop travis CI (#1176)
new f74be71 remove all console.logs (#1177)
new fdf5fe3 [TE] New Email Template (#1180)
new cb83310 [TE] unbreak logging in unit tests (#1179)
new 4a66539 [TE] Dataframe (#1178)
new 11985ca Calculate offset for baselines using Period instead of millis, so that DST is handled correctly (#1183)
new efbbce5 hide view contribution link when ingraph link is present (#1184)
new b5df0f6 [TE] More sensical Anomaly region (for daily granularity) (#1186)
new 705a32b getter for metrics registry (#1187)
new bde9b11 [TE] Yyuen/pulling library locally (#1188)
new 2b4e6ad [TE] data completeness resource with endpoints for complete and incomplete dataset segments (#1159)
new 2a231f2 [TE] Extend merged anomaly for unifying anomaly result class (#1182)
new 07e80eb apply font family to graph (#1189)
new 94cd4fe [TE] Yyuen/apply styling to chart (#1191)
new df67c32 Support for partitioned data during segment generation and query (#1181)
new 97bd343 Adding padding to prevent y axis numbers being cutoff (#1194)
new 7ef41fd Using new constructor for LongColumnPreIndexStatsCollector (#1195)
new 794f686 Implemnet MergedAnomalyResultBean constructor (#1197)
new b343291 Add partition function to mimic Kafka's default partitioner. (#1196)
new c7c7ddc [PINOT-4462] Step 1 to add auto-cleanup of deleted segments (#1193)
new 840ffa5 Fix parameter name of WeightThresholdAlertFilter (#1198)
new 2b8b8d3 [TE] Fix null exception when using toString from MergedAnomalyResultBean (#1199)
new 033b8f1 attempt1 (#1190)
new 081faf5 Create top class toString method for all bean and index classes (#1201)
new fde9e5b [TE] update anomaly detection context to include historical anomalies (#1203)
new e6a830a Unpack znodes from client API if necessary (#1207)
new 2f3b35a Send email notifications from Data Completeness Checker for incomplete data (#1206)
new a66e309 [PINOT-4475] Handle SimpleDateFormat correctly when creating segments (#1200)
new e75a7a5 Reuse in-memory dictionary during stats gathering (#1202)
new 51d52ac Updating table configuration should ensure that realtime cluster is setup (#1204)
new 0480275 InitAutoTuneEndpoint (#1185)
new ba2231f Add classifier config (#1209)
new 1b995e1 Updated MoveReplicaGroup to take source files as csv parameter (#1208)
new 7d34b12 Refactor DAO manager impl classes (#1213)
new cb93cfd Changing the SegmentNameGenerator API name from getSegmentName() to the more appropriate generateSegmentName() (#1212)
new 1293128 [TE] Yyuen/link to old ui (#1214)
new 0e21ad3 Add view of baseline of min-max function if it is available (#1216)
new 89b846b Control the creation of shaded jars using a profile called build-shaded-jar. (#1218)
new 69a1d4e Dataframe extensions (#1217)
new 222827b Fixes to PUT table configuration API (#1215)
new 9b21132 Upgrade alert filter auto tune tool to fit new autotune end points (#1210)
new 8df24d7 Clean up controller tmp directory when uploading files (#1220)
new 543325b upate TimeSeries in anomalydetection context to align with anomlay-detection lib (#1221)
new 6045e8c Implement Function Replay Endpoint (#1145)
new 6e34dd5 Add support for metrics from multiple datasets in a derived metric (#1224)
new 97ae157 precision and recall in emails (#1225)
new 3bb7a68 Separate IndexLoadingConfig from TableDataManagerConfig (#1211)
new a31e994 Refine backfill function and replay runnable (#1229)
new 12dccc8 Add support for viewing anomalies with custom ranges, dependant on the anomaly function (#1228)
new 3c9c427 Implement a RELOAD message to trigger segments reload (#1222)
new 78fc5c6 Adding regex predicate support (#1192)
new cdd0714 Support for selection and filter on no-dictionary columns
new 22de286 Added support for no-dictionary based selectino and predicate evaluation.
new aea4c3c Add version of getInt api that takes reader context in FixedBit/ByteSingleValueReader.
new 04c72d1 Adding integration test for NoDictionary columns
new db0dc2f Addressed all review comments.
new 47f4a80 Handle default number of rows when updating flush threshold for LLC (#1023)
new daa5222 [TE] Add grouping stage after anomaly detection stage (#1226)
new 3fc0afb Remove keyword 'LIKE' from PQL grammar.
new a7e0922 Log info instead warning for bad queries (#1237)
new 2e471d0 Log an info instead of an error for data schema mismatch (#1236)
new d2ff096 Fix for status always showing Not Resolved in emails (#1240)
new 8f5b4ee Fix data completeness checker emails notified logic (#1241)
new e315738 Fix the NPE for realtime segment (#1238)
new e923dc9 THIRDEYE-1211 : Backend tasks for Root Cause page (#1244)
new a206f41 Implement offline analysis endpoint in DetectionJob (#1232)
new 211d19a Log info instead of error for NumberFormatException caused by bad query (#1242)
new b9da7d8 When validating schema, log info instead of error (#1246)
new 4bc793a Add support for partitioners in LLC realtime. (#1230)
new d17e905 Implement multithreading time series query handling (#1249)
new 4f03f47 Fix the rest API for getting table/segment metadata (#1250)
new 5ca35f7 Log info instead of error for invalid table config uploaded (#1247)
new 5af9d1e [TE] Add data filter for removing noisy time series (#1248)
new 455f94c Update Pinot Druid benchmark programs (#1253)
new 597bf55 Add backward compatibility for reading old JobBean (#1252)
new 0cab1f1 Change email subject to Thirdeye Alert (#1251)
new 760dc1e Onboard flow endpoint (#1231)
new 9ea9bb1 [PINOT-4726] Remove excessive connection-related logs from broker (#1245)
new ae9b871 te_anomaly_removal (#1239)
new d942958 Add time range when applying data filter (#1254)
new 02d13da Add JSON string serialization/de-serialization apis for SegmentPartitionConfig.
new 8b359a0 Follow-up from partitioning pruner changes (#1255)
new c894f00 Derive partition ranges during indexing instead of relying on config. (#1258)
new a74b3f5 Fix to recompute routing table when instance config changes in Helix (#1260)
new be45e69 Refine TimeGranularity handling for non-additive dataset (#1259)
new aff606a THIRDEYE-1242 : Increase frequency of schedulers (#1257)
new 1e627d0 Max data time endpoint, topk in heatmap, rounding off double values (#1261)
new 712e1fa Adding min and max limits for rounding (#1266)
new 67182f0 Enhance IndexingConfig to have SegmentPartitionConfig. (#1264)
new ab8acf2 [PINOT-4748] Retry on temporary kafka failures in the controller (#1263)
new 0270aa6 Handle invalid content type while reading input schema (#1269)
new 9c12ed7 Fix the flaky Netty Connection Tests (#1265)
new 586158a When loading segment files, load from desired segment version (#1271)
new 066315a [TE] get metric name from anomalyDetectionFunction spec, not self defined (#1274)
new 26dedbb Refind AutoTune Thirdeye Component (#1270)
new d33850f Make max time value consistent across granularity of dataset (#1276)
new e453295 Implement parsing function from and to JSON String (#1272)
new d8417e7 Returning only original data aggregation granularity in case of non additive datasets (#1278)
new 636e7c3 Fix the problem where PinotLLCRealtimeSegmentManager directly load segment metadata from a file (#1277)
new d5128a7 [PINOT-4764] Ignore exceptions in AsyncPoolResourceManagerAdapter during broker shutdown (#1273)
new dc88ec2 [PINOT-4768] Handle exceptions when a table gets dropped while committing an LLC segment (#1275)
new 761710a Temporary fix for duplicated alerts (#1280)
new d564e65 Fix frontend max data time issue for non-additive dataset (#1279)
new bd85e25 Avoid logging exception for bad table configuration (#1267)
new c53fbd2 Fix NPE from getTableSchema (#1284)
new 0b66b23 Logging message for invalid segment API calls
new 72673fb Refine the interruption message when task driver is terminated (#1285)
new d1d4160 For invalid aggregation function name, log info instead of error (#1283)
new 1db9133 [TE] Yyuen/root cause analysis redesign (#1227)
new bde175b Enable AnomalyResource to get AnomalyTimelinesView from MergedAnomalyResult (#1281)
new 6169b42 Gracefully handle create requests for existing offline tables. (#1286)
new e9b38fb Add a blocking shutdown API to NettyServer to ensure server is correctly shutdown in tests (#1287)
new 10e509f Bugfix for server instance shutdown state (#1288)
new 7e27732 Add a ConcurrentReadWriteDictionaryTest to test concurrent read write against REALTIME dictionary (#1295)
new f003f1e Enhance data filter for handling unhealty data (#1298)
new e2fa948 [TE]Initial anomaly report generator endpoint (#1297)
new f624c99 Add external link with dimensions (#1302)
new afc5259 Change error to info for user errors (#1299)
new e134a66 Fixing service status to return GOOD after current state matches with Idealstate (#1304)
new c2e97c2 Change anomaly function's default delay time (#1306)
new c86313a Enhance task driver's execution and termination logic (#1303)
new 1ca6c51 [TE] dataframe extensions for root cause analysis framework (#1308)
new f06ce5f Removed parameterization of K for KeyedFuture interface. (#1307)
new 015300e [TE] test operation semantics for double, long, string, and boolean series (#1312)
new b23be4e Fix the problem of int overflow when trying to generate inverted index on huge number of entries (#1315)
new 7305e05 Add ability to set expected completeness and completeness algorithm for dataset config (#1317)
new 182089f Unify Performance Evaluation Method in ThirdEye (#1310)
new cfd4cf7 [TE] rootcause analysis pipeline prototype (#1309)
new 4bd0205 Reuse JVM for pinot-api and pinot-common tests (#1293)
new 1bd28cb switch system.out.println() to LOG.info() (#1318)
new 9936a47 Throw exception when failed to load segment in both retry and no retry case (#1316)
new e09e1db Fix != in query predicates (#1289)
new 9fff256 Handle bad json format during schema create or update operation (#1296)
new eba3fb5 Fix null alert filter in AnomalyReportGenerator (#1320)
new f087ebb Implement lock-free thread safe on-heap mutable dictionaries (#1301)
new a32eff7 Implement methods for self-onboard daily detection function (#1313)
new 2213882 Various updates to service status (#1321)
new 89a31ff Add ability to read from external pipelines (#1319)
new 9dfaf0d Wait for query timeout during shutdown (#1322)
new 681ca0d EntityUtils.assertType() fix (#1323)
new a19850b Eliminate exception due to json parse error of table config zn record (#1325)
new 58d3b93 [TE] Rootcause refactor (#1327)
new cd040ac Propagate bad query exception all the way so that it does not get logged as an error (#1328)
new 905aad1 [TE] Add alert grouping logic (#1324)
new e19c17c Add URL encoding to handle space chars in dimension name and value (#1330)
new 0a282a0 [TE] Plug in holiday events pipeline to rca framework (#1329)
new b1c9cb5 [TE] contribution analysis fix (#1331)
new adcf8cd improve interactive framework execution (#1334)
new 4dcb7cc Fix the NPE in MCombineOperator when query timed out (#1335)
new 1abf69b Add average change threshold alert filter (#1337)
new 8ab8305 fix pagination ajax call (#1333)
new 1d73239 unified replay endpoints (#1314)
new c25b2b9 Shorten host names in response time logs (#1311)
new a5010f0 [TE] Dataframe selectors (#1332)
new 769fcd7 Improve empty group key handling (#1338)
new 89aa5c3 Add support for Star Tree realtime segment generation. (#1339)
new e017f1f Autometrics config setup (#1340)
new 7f8e925 [TE] Dataframe performance tweaks (#1341)
new 26d0cd9 Derive table name from input segment for ColumnarToStarTreeConverter. (#1343)
new cba0b3f Add ability to load EventDataProviders from external code via config (#1346)
new 1089a2e minor change to single function replay (#1348)
new c701db1 Add a REST API to decommission the instance (#1344)
new 0d91577 Fix druid query (#1349)
new c25c560 Remove deleted segments after N day retention (#1342)
new bed9181 Change single-line whitelist to multilines (#1354)
new ebaea49 Entity to entity mapping table for root cause analysis (#1355)
new 59de82b delete existing RCA framework (#1359)
new c07cf4b minor dataframe extension (#1360)
new 1b1e879 [TE] Rootcause DAG execution framework (#1361)
new e9cb829 [TE] Rootcause analysis entities and utilities (#1362)
new 3601402 fix pagination bug for results num multiple of 10 (#1366)
new 562173a [TE] Rootcause analysis pipelines and business logic (#1363)
new 8d52e8a [TE] Yyuen/add ember (#1336)
new 5d9fb22 Split the query generation piece of BaseClusterIntegrationTest into (#1351)
new 2c3685a Fix segment status checker to report the best available number when no segments are present (#1364)
new 46eedd5 Fix segment deletion manager to handle deleted directory correctly (#1368)
new 9b6719a [TE] Rootcause pipeline generalizations (#1367)
new 98328b8 Fix the function name for getAllTableNames (#1350)
new b11c661 fix bug (#1371)
new 72862d9 Pick the first server to report a row-limit reason as winner (#1369)
new fa68b16 robustness for dimension analysis, URN parsing and pipeline defaults (#1370)
new 97f9669 [TE] Add buffer to holiday pipeline (#1373)
new 4c756bc iso 8601 support for frameoiwrk runner (#1374)
new 96ad290 Modify single function replay runnable to threading (#1372)
new 723dca5 Added segment recovery mechanism to segment reload (#1358)
new 5265452 fix window size variable name typo (#1377)
new fdae195 Prevent checking for the field 'replication' while adding LLC tables. (#1375)
new 1656b64 In netty server tests, Wait for server to start before connecting to it (#1378)
new 0e29a4d resized email (#1383)
new 91922f9 Fix bug causing incorrect values for wow in derived metrics (#1384)
new 6f8ad2b Allow loading offline table schema with offline table name or raw table name (#1382)
new 58018ac Update idealstate when replication is changed for a table. (#1381)
new 9dc7868 Fix the bug where BadQueryRequestException is not properly thrown out in CombinePlanNode (#1385)
new 70ae44c Provide the ability to have different incoming and outgoing specification for time column in LLC. (#1379)
new d7d4a2a Decrease frequency at which Data completeness runs (#1386)
new ccc9be9 [TE] Rootcause frontend draft (#1380)
new bb2b002 [TE] Shade or exclude commonly used jars in other projects from TE Hadoop fat jar (#1388)
new eb5f938 Address comments for outgoing time column (#1389)
new df803b4 Update the highest kafka offset metric correctly so that we can use it for alerting (#1387)
new 303fb39 Shade all jackson packages in thirdeye-hadoop (#1391)
new 9575e19 Rename shaded jar to follow naming convention (#1393)
new dea7ca4 [TE] support static typing for dataframe builder (#1390)
new e764ee7 Changes to handle external client and external metric properties (#1392)
new cc93da4 Shade snake yaml jar (#1395)
new 2fa9861 Refine performance evaluation method (#1353)
new 47f4d1e [PINOT-4885] Changed in-memory byte buffers in realtime to use native order (#1394)
new 64f3ce3 [TE] Changes to make external clients independent of thirdeye database (#1397)
new d8236bb Avoid creating unnecessary garbage during realtime indexing. (#1401)
new dc58353 tolerate (and skip) entity formatter exceptions (#1396)
new 86a71a5 tolerate class not found for formatters (#1398)
new ae69d7e enforce DAO instantiation as singleton (#1403)
new 974f7c4 split holiday event formatter (#1400)
new c260d11 [TE] fix root cause template (#1399)
new e5056ab Initial commit for Pinot Minion (#1352)
new a70f877 Enhance Schema setter and getter annotation (#1406)
new 4812a59 truncate rootcause label to 500 characters max (#1410)
new c241b12 Reuse metrics during realtime ingestion (#1411)
new 8c847db [TE] Yyuen/heatmap tooltip (#1409)
new 396d3ee Refactor TableNameBuilder class (#1408)
new 85aac0b Added support to PinotHelixResourceManager to handle Minion resources (#1413)
new 03220fe rootcause results in data tables (#1412)
new 9f8a528 Shade avro library for fixing dependecy issue (#1417)
new b4d6e29 make mapping type a generic string (#1418)
new 3036857 add basic hyperlink reference entity (#1419)
new f863dd3 Change pom file of te-hadoop to produce thin jar (#1421)
new d7e2e23 Misc changes for reducing garbage generation during query execution. (#1407)
new 62c86f2 [PINOT-4885] Change FixedByteSingleColumnMultiValueReaderWriter to expand header section dynamically (#1405)
new 967b27f [TE] Refined Dimensional alert grouping logic (#1414)
new f20f2e5 Enable offlineAnalysis return list of anomalies (#1347)
new 0192101 Integrate Minion into the cluster (#1420)
new 505b797 Fix incorrectly reported query processing time when all segments are (#1425)
new 80eb685 In PinotTableRestletResource, for client error, log info instead of error (#1422)
new d58cdbe [TE] Implement CondensedAnomalyTimelinesView (#1424)
new 97769f0 Remove empty dimension from email subject (#1426)
new 4dfac09 Fix the setter argument @Nonnull annotation for derivedMetricType (#1428)
new 2f9166c fix splitting logic to allow for filters name with : (#1427)
new d240c8a support dynamic loading of static pipeline (#1433)
new 4d7ac06 [TE] Dataframe join tweaks (#1432)
new 9f09f00 [TE] Implement Anomaly ImpactToTotalMetric (#1404)
new 1983ca7 [PINOT-4885] Changing SingleValue Forward index to be expandable (#1423)
new b305147 [TE] Rootcause runner tweaks (#1431)
new d472d71 Remove the old bimap based mutable dictionary implementation (#1438)
new 07e9e40 Build shaded jar for minion (#1439)
new 965ef8b Shade all dependency jar of te-hadoop (#1435)
new 180f4b3 [TE] Fix NULL pointer exceptions in dashboard (#1441)
new 868ff5b fix warn message (#1442)
new e649b8e Fix NEP in fetch metric tool (#1443)
new dd665a7 [TE] metric correlation ranking pipeline (#1434)
new cd23830 support multiple entity-to-entity mappings from single source entity (#1444)
new ce5b835 Revert anomaly detection input context builder (#1446)
new 1bf01e1 Revert "Revert anomaly detection input context builder (#1446)" (#1447)
new 5b6142d Add loadRawAnomalies flag for all merged anomaly fetches (#1448)
new 5c04710 [TE] - Use client from config, to support external clients (#1449)
new f3e7cde Refactor AbstractTableConfig (No functionality change) (#1445)
new d9dc8c4 Avoid query execution path for simple count(*) queries. (#1452)
new ded6d30 Change concept of client in TE to datasource (#1455)
new 434a4f3 [TE] External email recipient provider (#1453)
new 8875ecf [TE] Add UI and email template for grouped anomaly (#1430)
new 07474aa [TE] support multiple (named) framework in single configuration (#1451)
new b6f395e set datasource (and minor fix to builder api) (#1456)
new 7ccf910 [MINION] Added PinotTaskRestletResource into controller (#1437)
new 2872a66 Merge offline/realtime table config into class TableConfig (#1461)
new 5935d64 Add MetadataBasedAggregationPlanNode and Operator. (#1460)
new f381e76 Add OnboardDatasetMetric entity (#1459)
new a6ab4dd [TE] refactor rootcause endpoint to support related metrics query (#1458)
new 91ee55c [TE] add euclidean similarity measure to related metrics pipeline (#1463)
new 9b2d6bc [TE] Refine recipient provider factory for external providers (#1464)
new 376e859 [TE] Add methods to dataset, metric, dashboard resources for external clients to access (#1465)
new 7b33e62 [TE] refactor rootcause dashboard config, make RCA optional (#1466)
new 78bb3ec [PINOT-4887] Added support for no-dictionary columns in Realtime consuming segment (#1462)
new a13bd5a [TE] Enhancing search by metric name to search by name or alias (#1429)
new bbd4ab9 return possible values for status, metric, dataset and dimensions in the anomaly search results (#1376)
new 6e43c21 [TE] Changes to caches to handle external data sources (#1470)
new 1cc0337 [TE] Yyuen/add more filters (#1469)
new a2d2723 [PINOT-4887] Fixed min/max values for realtime no-dictionary columns (#1468)
new 34d40c0 For whole table reload, send table level message instead of segment level message (#1467)
new e32b96d [TE] spawn pinot connections in parallel on initialization (#1457)
new 7e3aa7f [TE] Yyuen/add filters to investigate link (#1474)
new 40bfc07 Move functionality of completeness checker to interface impl (#1478)
new 40ca7e4 Returning null to fix LLC segment commit when there is no leader (#1471)
new 35821bf Fix broken star tree index viewer due to migration to segment v3 format. (#1475)
new 925083a [TE] Add endpoints to activate and deactivate functions in batch (#1473)
new 1c826ec Exit Pinot when a top-level exception is caught (#1479)
new a259dd1 Adding simple date format to segments config (#1416)
new 8dc9415 Make auto onboard service generic to onboard from all data sources (#1483)
new f6a046f Add license badge to README.md (#1480)
new ab26366 Add Docker instructions for quickstart (#1481)
new 83a9d18 Fixed ColumnarToStarTreeConverter to correctly load the config file. (#1472)
new bf5fb93 Add PQL support for query options (#1490)
new 0265d95 fix pagination (#1489)
new 81c8492 Revert "Adding simple date format to segments config (#1416)" (#1493)
new 9f008a8 [TE] augment path property with postfix (#1494)
new ce95f8d [PINOT-4602] Changed servers to re-use segment files if available, when asked to re-commit an LLC segment (#1485)
new f31de2a Reduce number of DNS lookups on the broker (#1487)
new 1dc9e5d Remove setting for commit timeout to exercise the lease code. (#1496)
new f1d4ceb [PINOT-4887] Ensure that we put in an invalid value for cardinality in metadata, and throw exception when invalid value is present. (#1477)
new 1cbdaf5 Added getTableConfig API into ZKMetadataProvider (#1497)
new 57d068c [TE] Yyuen/adding_ember_mirage (#1499)
new 7014376 Fix the issue where cache does not get cleared properly when setting fields in schema (#1488)
new f327ffc Add TableTaskConfig into TableConfig for task config (#1482)
new 7b97768 Remove LinkedIn internal config from config file and tests (#1502)
new 8481858 [TE]Bug fixing replay end and start disorder (#1501)
new a0700c5 Add classification config to entity editor (#1503)
new 7f50830 [TE] endpoint for anomaly time ranges per metric id (#1500)
new 23d8c6b te_endpoints_anomaly (#1504)
new 6ae8168 Fix HybridQuickstart and standardize all sample configs (#1507)
new a216b78 [PINOT-4889] Preparation for off-heap mutable dictionary for Realtime consuming segments. (#1508)
new b97cdcd [TE] Enable Merge For Two Sided SignTest (#1484)
new a5881d9 [TE] Set historical anomalies in building AnomalyDetectionContext (#1510)
new 82372352 Implement Joda DateTime Utils (#1506)
new 55530a6 Add partiton metadata into SegmentZkMetadata. (#1505)
new 4ea8a89 Added tooltip (#1512)
new bd9bde0 [TE] Fix the decoration for filter set when query mixes non and additive datasets (#1511)
new d295634 [TE] Combine filterset and anomaly dimension (#1514)
new 91378dd [TE]Yyuen/more_ember_addons (#1515)
new 695f4de [TE] Fix week-over-week values for ratio metrics (#1517)
new 3792e3b Remove check for data completeness flag from dataset, and rely only on flag from anomaly function (#1516)
new 1f248fd Fix frontend compilation issue (attempt 1) (#1518)
new 103004d [TE] Issue type classifier for anomalies from multiple metrics (#1509)
new a2e4013 [TE] add search filters to time range endpoints (#1519)
new abd15a7 [TE] Add ad-hoc anomaly detection for classifier (#1520)
new b39188b [TE] Dataframe utils (#1521)
new 18aa410 Refactoring classifier interface and fix typo (#1523)
new 0af7f64 support addSeries from DataFrame with alignable index (#1524)
new ecaa211 [TE] Refactor classifier interface to simplify implementation (#1528)
new b933bcc [TE] cleanup multiple version identifiers (#1530)
new 17878c8 [TE] Dataframe grouping extension (#1529)
new af5304e [TE] tolerate missing data and data source exceptions (#1531)
new 4897d20 [TE] remove default time-groupby (#1533)
new 7d55a81 [TE] Multiple main anomaly functions for classification jobs (#1537)
new 82d1cbb Update DetectionResourceHttpUtils because of the new http query path (#1534)
new 2635f74 Broker side ZK metadata based segment pruner. (#1532)
new 8fcac62 Fix brokerHost argument being ignored (#1538)
new 7516852 parsing the dimension filters JSON (#1535)
new 73286bf Enable batch reload on controller side (#1526)
new 81493ce [Uniform Schema] Use table name to load schema instead of schemaName field in tableConfig (#1527)
new e4d1ca6 expanding daterangepicker max time range (#1543)
new 76f7307 [TE] Decrease the shutdown time of task drivers (#1544)
new 5230f02 [TE] Anomaly ID Presentation Component (#1541)
new 439f830 [Pinot Minion] Add dataDir, metrics, healthCheck into minion (#1536)
new 9e3a3b8 [TE] enable json pretty print by default (#1545)
new 3de6239 [TE] score metrics based on target and candidate score (#1546)
new 19ac437 Refactoring broker routing table generation to select strategy based on builderName in tableConfig.routing (#1539)
new 6e4a60f Split Commit Part 1 (#1491)
new f51cef2 [TE] add generic database-backed configuration store (#1542)
new c6803b0 Fix the LLC_PARTITION_CONSUMING metric (#1551)
new 5c6be72 [TE] Add Issue Type of Anomalies to UI and Alerts (#1552)
new 8885581 [TE] Add filter set to summary algorithm (#1554)
new bcfcb8b [TE] Frontend timeseries resampling (#1522)
new 8c93d9b Refine the function to find merged anomalies in time range (#1548)
new eff50f6 [TE]Unify BaseAlertFilter Thirdeye and Anomaly Library (#1549)
new 634cc60 Fix NPE in Broker when existing table configs don't have any routing config options (#1556)
new a92464d Replica group aware segment assignment strategy (#1540)
new 7d93fe4 Moving routing package from transport to broker (#1557)
new 57a6982 Remove all traces of external data sources (#1560)
new 1ebf5f5 Moving SampleExternalView to broker module from transport module (#1561)
new b18732d [TE] support object de-serialization over string only (#1550)
new df7f3fd Commented out scripts for startree benchmark (#1553)
new c7fdd8b [TE] fix junit import regression (#1564)
new a621e63 Reduce the runtime for integration tests (#1559)
new e1c1ec7 Add back segmentPushFrequency in SegmentsValidationAndRetentionConfig (#1565)
new 696e7a9 [TE} Yyuen/fonts and ember truth helper (#1563)
new fdba41a [PINOT-5062] Handle NumberFormatException when looking at column values during pruning (#1562)
new 54d8cd8 [PINOT-4889] Off-heap mutable dictionary for fixed width columns (#1513)
new b39a5f0 Fix HybridScanBasedCommandLineTestRunner to use the correct row-limit for LLC (#1570)
new b1a38fa [TE] Add auto complete end points for functions and alertconfigs (#1569)
new 09aa411 Extend waiting time for reload (#1572)
new f743919 [TE] Nav bars with bootstrap grid (#1568)
new ff4e028 added component (#1573)
new d722913 redux and container component and rca page (#1574)
new b305c43 [TE] Tie in old ui to new ui (#1575)
new 5e1cb1d functional basic type ahead search (#1578)
new 5b703c1 Fix rebalancer adding extra replicas (#1567)
new 988346c Fix commons-math dependency (#1566)
new 369d406 Clean up logging for the live instances listener (#1495)
new 05b85c1 Endpoints for triggering onboard service for data sources (#1579)
new 1bb622d Nav updates (#1580)
new be49258 [TE] Dataframe performance improvements (#1577)
new a0d5729 [TE] RCA related metrics - add static ranking strategy
new 684dc72 fix anomaly page spinner (#1582)
new cd2a642 Adding Application entity information for alertConfig grouping (#1583)
new c48cec3 Adding feedback status and comments to anomaly details (#1584)
new 525b9d5 [TE] refactor rootcause pipeline event entities (#1586)
new a6e1312 RefineSetParameterBaseAlertFilter (#1585)
new e791f18 JSON decoder improvements (#1589)
new 4f574c9 [TE] Rootcause events endpoint (#1588)
new 0e119d4 [TE] support (almost) arbitrary series names (#1590)
new f8a963f LLC Split Commit Pt. 2 - FSM (#1555)
new de786e7 added feedback comments (#1587)
new 0baa125 [TE] fix bug in sum specialization for long sequences of nulls (#1594)
new 34bbf30 [TE] Dataframe tostring extension (#1593)
new ba097d5 [TE] single value accessors for series (#1592)
new 74e445d [TE] dataframe and series slice tests and fixes (#1596)
new 12d0bcc move anomalies view by function id to AnomalyResource and add option to apply alert filter when query anomalies (#1595)
new 4af2e13 Refactoring to query scheduler to support priority based scheduling (#1597)
new 3036f6f Enhance interface of group recipient provider for advanced usage (#1598)
new c98b455 Reduce the table level broker metrics emitted (#1571)
new 953ea66 Adding partition aware routing table builder (#1558)
new dae8d25 Refactoring LLC APIs and adding FSM tests (#1599)
new aa6e9ca fix spacing in filter name bug (#1604)
new e71332c Refactor anomaly feedback (#1607)
new ca76c19 [TE] Self Serve Nav and Feature Layout (#1602)
new 80aad3b Remove all segments from store when a table is deleted (#1609)
new 340f3dc Adding a context to all the off-heap allocation (#1608)
new b486454 [TE] Monitoring performance counters (#1610)
new 22ddd45 adding admin page for managing entity mappings (#1611)
new 6dacfd2 Ignore isNotified field of anomaly in alerter (#1601)
new 02378fc [TE] Dataframe object series (#1603)
new 00dbe30 [PINOT-4889] Implemented off-heap dictionary work for strings (#1591)
new 8361c5f [TE] raw endpoint for rca pipeline invocation (#1612)
new 6dc469d datatables for listing entity mappings (#1613)
new 1b75dcb created filter-select component (#1614)
new 440d36c adding fix for mapping ui selfservice entity urn parsing (#1615)
new 00c2585 [TE] yyuen/manage alerts show all (#1617)
new 95bc00d [TE] Rootcause entity mapping editor scores (#1619)
new 747a518 Update README.md (#1620)
new 3b04092 Added information about scanned documents in the request log on server (#1616)
new 1c7df69 Add a REST API to fetch crc information (#1547)
new 36a922e Fixed the bug for replica group aware segment assignment strategy. (#1622)
new 2e422dc Invert colors of contribution details and heatmap cells if inverseMetric is true (#1625)
new 4a7a555 [TE] auto unboxing of singleton series values (#1626)
new f6652bb [PINOT-4926] Introduce a memory manager so that we can switch between Mmap and direct allocation (#1621)
new d3bf5a8 Fix for star tree to handle offset larger than int (#1606)
new e70827b Avoid creation of byte[] during DataTable deserialization. (#1629)
new b409da2 adding alert subscriber mapping provider end point (#1630)
new 80612cf Implement an endpoint that outputs the baseline calculated by algorithms (#1623)
new 5c834a1 Refactor and optimize all integration tests (part 1) (#1618)
new b094d80 Use pinot-api response in integration tests (#1631)
new 143d681 Adding default aggregation function as a metric field, so that we don't always default to SUM (#1633)
new 2b812fd Modify the calculation of impact to global (#1624)
new a874f32 Changing scope of fields to ensure they aren't resued over multiple runs (#1637)
new 362d4de Change the default segment format in perf runner (#1636)
new e58f95d Fixed NPE in FileBased instance config, removed segment replace test via OFFLINE/ONLINE (no longer used) (#1638)
new 5fcb61d TE : enabling replay and autotune end points in dashboard (#1640)
new b219491 Optimize ConcurrentReadWriteDictionaryTest (#1639)
new 5950735 Optimize BitmapInvertedIndexTest (#1642)
new be45b35 Optimized TablesResourceTest (#1641)
new 17b8dc8 [TE] Dataframe grouping expressions (#1632)
new 49792e9 Shade Apache hadoop jar to solve runtime error (#1647)
new 9c3ef29 Improve the shading for apache hadoop (#1650)
new bfe5f33 [TE] generic map code test (#1649)
new e2bf1a1 Optimizing PinotResourceManagerTest (#1646)
new 1b5c9e0 Optimized HelixBrokerStarterTest (#1648)
new c7bde83 [TE] double series rounding and normalize (#1654)
new 763fb53 Implement endpoint that surface function baseline to clients (#1634)
new e16e9fc Adding partition aware realtime routing table builder (#1605)
new 6748ebc [TE] Alert Management and Creation (#1644)
new daa459f Throw BadQueryException when illegal/unsupported TimeUnit specified in (#1657)
new eae85df Re-factored RetentionManager to use PinotHelixManager alone, and converted RetentionManager to a pure unit test (#1652)
new 093cea3 Updating the table config cache timeout to 1hr (#1659)
new 452e3ba Optimize ControllerTenantTest and ControllerInstanceToggleTest (#1658)
new 4827807 Cleaning SegmentAssignmentStrategyTest (#1661)
new c68c42c Removed all SentinelTests, refactor fastHll tests (#1645)
new 69dde01 Optimizing routing table builder tests (#1663)
new 4567b99 Refactor AvroRecordToPinotRowGenerator for bug-fix and cleaner logic (#1653)
new bf596a9 Optimize redundant adds into fixed size priority queue for topN (#1662)
new c5231d3 Follow up to PR 1606 to avoid int overflow during star tree creation. (#1664)
new c7e2fbf Fix the flakiness of TableViewsTest (#1665)
new e4ccc48 LLC Split Commit Part 3 (#1600)
new 8619d40 Fixing invalid enum constant in segment get API (#1660)
new c05d585 Avoid expensive String.format for non-double values. (#1667)
new 3f43c37 Fix NPE when accessing broker's routing table debug endpoint. (#1669)
new 18a9d14 [TE] Enable anomaly function to partially update its properties (#1651)
new 872db00 Fix the bug where dropping a HLC table will throw NPE (#1666)
new 070ea85 [TE] Enable speedup for backfill (#1655)
new 2c88c22 [TE] Create Alert Flow Fixes (#1671)
new f789eb7 Fix the bug where deleting HLC realtime table throws ZK exception (#1672)
new a5c5990 Remove metrics from non-leader controller (#1673)
new 58768d8 TE: Adding external resource load in dashboard app (#1670)
new 766b3e9 Modify js according to the new change of anomaly function CRUD (#1674)
new 5c9344a Allow front-end to send out alerts from list of functionIds (#1656)
new 834e761 [TE] Rootcause default scorers (#1675)
new 2e043f3 [TE] overfetch anomalies and holidays (#1677)
new 8e07e6b Minor cleanups for validation metrics (#1676)
new 67c045e Bug fix for inconsistent padding removal between api's in (#1681)
new 31d822a [TE] fix rootcause endpoint usability (#1682)
new 72b1f00 [TE] adapt to backend changes (#1683)
new eb2c301 [TE] Allow job status checking for detection jobs (#1679)
new 91c7e25 [TE] fix js copy paste errors (#1685)
new 466e027 Refactor http request utils in ControllerTest for simplicity (#1686)
new 8e8d560 In create anomaly function endpoint:modify window delay, setup frequency for minutely data (#1689)
new 1db2d05 [TE] rootcause - cross-comparable values for dimension analysis (#1688)
new 497dcbb [TE] Import Metrics Form (#1691)
new 90a8814 [TE] Fix runtime problem of Node.class from org.w3c is not found (#1692)
new f1ac74c Fix on the missing filters on anomaly result in DataResource (#1687)
new db863c3 Rewrite HybridClusterIntegrationTestCommandLineRunner to use different ports (#1690)
new f48c447 Fix the test where segment flush size is too small for real use case (#1693)
new 7472e64 On-Heap implementation of immutable StringDictionary. (#1684)
new bd4cb14 Fix travis (#1695)
new b639ea0 [TE] DataFrame setters and fastjoin (#1698)
new 7ce59b8 Clean up the add table methods in ClusterTest for simplicity (#1694)
new 3d57062 Fix runtime error of thirdeye-hadoop (#1702)
new 21725a9 adding service name in mapping self service (admin) (#1699)
new d6e3766 Fix the flakiness of OfflineClusterIntegrationTest (#1700)
new 9373a4f Fix FileNotFoundException when uploading multiple segments with same name (#1697)
new 5b7b4f7 [TE] Self-serve Logic Enhancement for Import Metrics (#1703)
new 2671977 Suppress unnecessary log for partition aware routing builder. (#1701)
new 0cba293 final feature (#1704)
new 0eb2c0a Improve logging to identify failures during LLC segment commit (#1705)
new 8cc6b7f [TE] log rca pipeline duration (#1707)
new 641dc1e [TE] warn on non-additive dataset rather than fail (#1709)
new be59cbd [TE] RootCause - add analysis region timerange (#1711)
new 8b05b1a [TE] metrics correlation endpoint hotfix (#1713)
new f713c60 TE: Self Serve Unified Replay Flow (#1696)
new abf75b9 Refactor StarTreeClusterIntegrationTest (#1712)
new fe1982a [TE] append anomaly id to anomaly entity label (#1714)
new a9643f9 [TE] dataframe - joinpairs min size fix
new ff293b5 TE Self Serve Import and Create - refinement (#1719)
new 14003f8 implemented feedback (#1718)
new f4b0ba1 fix (#1720)
new 46726ab TE Create Alert Bug Fix (#1722)
new e826dd5 Remove the RoutingTableSelector related classes (#1721)
new ea36144 copy changes and default tab (#1723)
new 494ee78 [TE] Self Serve: adding filters/dimensions to request (#1724)
new b5c3dc8 Fix dependency issue for thirdeye_hadoop push (#1729)
new ea299fc [TE] rootcause - related entities tracking (#1725)
new 649c377 Adding country dimension to holiday events (#1730)
new 26ceaae [TE] human start date and duration (#1715)
new a179204 [TE] Dataframe group multiple (#1706)
new 815f7f3 Update urls in thirdeye to support changes in pinot (#1731)
new 542d16b Add window size as another attribute when fetching anomaly history (#1726)
new 7ab48c8 [Minion] Add Minion use case ConvertToRawIndexTask (#1710)
new 9431997 Enhance timeConvert UDF to support conversion to WEEKS/MONTHS/YEARS. (#1727)
new 14c37fa feedback (#1733)
new 334bdde In integration tests, when stopping Server/Minion instance, delete temp dir (#1735)
new f2e5e97 [TE] Remove mandatory whitelist, parallelize initial cache loads, improvements to admin, cleanup and more (#1732)
new 96ca7da [TE] ignore dimensions whose relative contribution did not change (#1738)
new a18a5c3 [TE] Rootcause frontend tweaks and fixes (#1737)
new 75c10f4 [TE] Self-Serve Create Alert: Prelaunch Fixes (#1734)
new 87a67518 Filter query tree serialization bugfix (#1740)
new d295ebf [Minion] Fix the delete task queue restlet resource (#1744)
new 997f5d3 [Minion] In ConvertToRawIndexTaskGenerator, do not count COMPLETED segment as running segment (#1745)
new 79cd140 implemented feedback (#1741)
new b835459 [TE] timeseries alignment when handling different time granularity (#1742)
new 8b38e74 [TE] support transitive hull for related entity mapping (#1743)
new bbf4ed9 Fix filter values are not shown completetly for non-additive dataset (#1680)
new 2cbf114 implemented feedback (#1747)
new 6d14cdd [TE] change default window delay setting and frenquency setting for anomaly detection creation endpoint (#1748)
new 9c25081 [TE] Rootcause - frontend event and metrics patches (#1749)
new 2e9051a Optimize sorted index (#1753)
new be64974 add tooltip (#1751)
new 9e8aa6f [TE] Self-Serve Pre-Launch Optimizations for Create Alert (#1746)
new 35118b7 [TE] rootcause - related metrics max deviation from baseline strategy (#1750)
new 70de36e [TE] Reroute "/" to "/thirdeye" (#1757)
new 2dc3707 [TE] Self-Serve Create Alert Pre-Launch Modifications (table layout) (#1756)
new 20ed5c7 Introducing Jersey-based REST admin API for Pinot controller (#1739)
new 1fe8827 [Minion] In ConvertToRawIndexTaskExecutor, change the uploaded file name to segment name (#1758)
new acb1340 Improve logs during realtime segment commit in the controller and server. (#1759)
new 2cd02e0 Remove the default segment version, allow loading segment as is without segment version configured (#1752)
new 5ae4167 Fix the bug in timeseries comparison (#1760)
new 276f633 TE: adding base classes to support authentication (#1754)
new da71771 Extend aux recipients provider to aux info provider (#1761)
new fe2a90e [TE] Added a feedback menu (#1763)
new de25573 [TE] Fix grouped anomaly alert cannot be sent (#1764)
new ac954f5 fix (#1765)
new 6bd2739 [TE] Self-serve Create Alert Bug Fix (#1768)
new 6204f63 Enable user input data granularity (#1766)
new ff0a131 Enhance AggregationGroupByTrimmingService to support trimming on non-comparable intermediate result (#1767)
new 6631033 events in table matches events in view (#1769)
new 7fa026f TE Self-Serve Create Alert - adding user selected granularity (#1770)
new 68dfdaf [TE] multi-keyword metric search/autocomplete (#1771)
new 73ca278 When adding segment to offline table, only set SegmentZKMetadata after operation succeeded (#1773)
new 35e99df Fix the exception from HelixHelper.getEnabledInstancesWithTag() when instance config does not exist (#1772)
new 1bc5d10 Fix and optimize star-tree unit tests (#1762)
new 6323498 [TE] add related entities and filter dimensions to label (#1776)
new dddaef8 [TE] refactor and tweak human timestamp display (#1775)
new 5338cd0 Compatibility patch for older http clients, a bug fix in segment state toggle, logging (#1774)
new 414a52e [TE] prevent checkbox column from resizing on content change (#1778)
new 0af8f1e fixing investigate max Date (#1779)
new 0fe67af fixing css selector (#1780)
new 6cbf94c Reverting PR #1773 and add comments to prevent same bug again (#1784)
new f5606a9 [TE] display region on top, investigation below (#1786)
new ad73d75 Cleanup: Remove unnecessary String.split() call when getting (#1785)
new 6c0c537 [TE] Fix delay when users trigger replay (#1789)
new 6b25a7c TE: bugfix, modify windowDelay and windowDelayUnit within function update endpoint (#1790)
new 53dd103 Hot fix for controller admin health check API (#1791)
new f60fb1d [TE] Update_time should automatically update to current timestamp on table update (#1777)
new 9a2751d [TE] gracefully handle null and empty responses (#1787)
new c87c2ac TE: authentication filter implementation with end point whitelisting (#1792)
new 722fa86 [TE] Towards using dataset metrics intelligently for correlation (#1788)
new f9b406e Use delete api instead of toggle, for deleting segments, because pinot will be deprecating the toggle api soon (#1793)
new 999d892 Added support for dimension columns to be no-dictinary columns while consuming realtime segments (#1795)
new ef47c7e [TE] Self serve onboarding: Replay, Initial Tuning, Notification (#1781)
new 36a9c31 Cleanup: Remove unnecessary ByteBuffer.duplicate in (#1796)
new 2fcd9bf Fix annotations (#1798)
new 2bdf963 encoding filters in endpoints (#1803)
new b43229a When adding a table, first check if tenant exists (#1797)
new 1bea88f Fix tables/{tableName}/instances API and other exceptions (#1802)
new 769d467 fix metrics tab (#1805)
new 6b0e8f2 fixing percentage change discrepancy between ui and email, adding created/updated by fields in objects (#1800)
new 9c2533d fix wrong type (#1806)
new c9e95dd implemented feature (#1801)
new bd3c767 Fixing API incomaptibilities (#1807)
new 3847406 Plumbing for realtime consuming segment statistics (#1804)
new 93a679d add frequency as query parameter for endpoint that update anomaly detection function, make time series retrieve endpoints do not roll-up (#1808)
new c9923de Surface function baseline to users, and amend the baseline with anomaly results (#1783)
new 91a621a Self serve replay notification, avoid exposing configurations (#1810)
new e924e69 Enable Jersey API as the default, restlet as backup (#1813)
new bfea0e2 Fixing APi errors (#1814)
new 00ce34b displayStart and DisplayEnd added to hashparams (#1815)
new 46865df Preparation for OffHeap config: Re-organizing parameters to RealtimeSegmentImpl (#1811)
new 083f5b1 Added new priority based scheduler for multi-tenancy support (#1668)
new e6f8abc Fix the package for unit test (#1818)
new 9dedc8d Clean up downloadSegmentToLocal() method in SegmentFetcherAndLoader to prevent unnecessary sleep and wrong marker file (#1794)
new 4b88645 Enhance Controller Validation Manager (#1817)
new b31347b [TE] Fix bugs in EmailResource (#1821)
new f513a7a Avoid unnecessary split/join for in-clauses. (#1819)
new 60aa367 Optimize query parsing (#1812)
new bc44c5b Build fix: PR's 1812 and 1819 were incompatible, causing a test failure. (#1822)
new 5bd55fe Adding a configuration for the length of query logs. (#1825)
new 0508866 Wire-in config and segment stats history for off-heap memory allocation for consuming segments (#1823)
new 962c96a Fix NPE and add a test case for older history files (#1827)
new 1c0629f Move StarTree viewer to jersey based resource (#1830)
new ece02cd Fixed a bug with priority queue selection logic (#1828)
new 562cb4d [TE] re-enable informed events, plus necessary tweaks (#1826)
new 2604e84 [TE] display all events touching range rather than only starting ones (#1831)
new de00c6d [TE] add informed to events-header template (#1832)
new 307b669 Bubbling up error during Avro -> JSON schema conversion (#1820)
new 2b398a3 Fix time granularity in anomalies resource (#1829)
new 92fbeee Refactor DefaultGroupKeyGenerator for performance concern (rename to DictionaryBasedGroupKeyGenerator) (#1809)
new ee9a3da [TE] fix js typo that breaks event end time check (#1836)
new c113e98 Adding end point to generate anomaly report by app name (#1837)
new dab203b [TE] Fix Anomaly Merge Logic (#1838)
new ac17734 Fix the thread-safety issue for realtime inverted index (#1782)
new 01801e8 [TE] frontend - baseline event tweaks (#1845)
new d096d7f Removing all restlet stuff from code (#1833)
new bb710fb adding rca link (#1846)
new 0d6f656 Re-implement the PinotDataBitSet (#1847)
new dc36f55 [TE] rootcause - max-score entity set (#1843)
new 67cf973 Retry segment download only on 5xx error codes. (#1848)
new 9d35ea5 Add list of DateTimeFieldSpec to Schema (#1842)
new 44106be Fix incorrect read of history on restart. (#1850)
new f0797bc Re-implement all offline dictionary based index readers (Based on #1847) (#1834)
new 34e610c Fix the int overflow issue for large index (#1851)
new c1c51da [TE] Distinguish missing data and zero (#1849)
new 7acd443 [TE] Separate time series fetcher for UI and anomaly detection (#1852)
new 4c1fe54 [TE] rootcause - frontend support for dimension entities as filters (#1854)
new 3ae423e Re-implement all offline dictionary based on the fixed byte value reader (#1853)
new ebff612 Kafka simple consumer fixes (#1855)
new 4058fda [TE] add conditional formatting for small numbers (like click-through rates) (#1857)
new e6addef Temporary disable testOneQuery() in PrioritySchedulerTest because it's failing (#1863)
new 3872a59 Fix SizeBasedDocIdSet (#1859)
new bd7bea4 Add connect and read timeouts for kafka schema registry fetch so that we fail sooner if kafka fails (#1861)
new 18aaa01 Fix file descriptor leak during off-heap allocation for realtime consuming segments (#1860)
new d8a3083 [TE] filter inactive metrics from related dataset metrics (#1865)
new a062000 TE: fixes related to alert subscriber group (#1864)
new a948f84 Remove repo directory from Pinot distribution (#1858)
new 317fb60 [TE] edit MVP (#1867)
new 3f72843 [TE] format 0 as 0.0 (#1866)
new a0fc9e3 fixing endpoints on prod (#1870)
new 7b37bf1 Optimize OrDocIdIterator (#1868)
new 89055bc Changing parameter of message decoders to type T (#1871)
new ed7d9a3 Clean up memory map files for a segment on start (#1873)
new 9243e6f fixing user readable values mapping (#1876)
new 0d7d865 Making LLC flush size configuration usable (#1869)
new c802cda Add metrics for keeping track of off-heap allocation for consuming segments (#1874)
new c31ac56 [TE] Extend autotune to take user configurable parameters (#1840)
new c8b6e00 new screenshot page in ember (#1877)
new 93fa63e Fix priority scheduler unit test (#1875)
new 7dd3eba changed url and script (#1879)
new a9f7793 [TE] setting default for sensitivity (#1878)
new 51c3629 [TE] implemented FE authentication (#1844)
new 56ea991 Fix off - heap mem gauge metrics (#1880)
new 0ba7ded Support for time bucketing with time zone offset for timeConvert UDF. (#1882)
new 88ade7e Revert "Support for time bucketing with time zone offset for timeConvert UDF. (#1882)" (#1885)
new 15dac30 Per request from user, they need an AnomalyFeedbackFilter in the search page to get the unlabelled anomalies or true issues. This pull request is to enable the anomaly feedback filter, and assign an user-readable name to user. (#1887)
new 8be4c2e [TE] Te auth ui fix (#1888)
new d44d16a [TE] Separate granularity of timestamp and data point (#1886)
new 46eaed6 [MINION] Clean up rest APIS and add API to clean up finished tasks (#1889)
new 91c7bf5 Make number of segments that can be refreshed simultaneously (#1884)
new c679c06 [TE] Support Email Sending from DashBoard using backend configurations (#1881)
new 5d7bf09 Allow us to cycle through different kafka schema registry VIPs during retries (#1891)
new d73f478 copy fix (#1893)
new e740e4f TE: authentication fixes (#1895)
new e9e9063 adding redirect logic to thirdeye-admin (#1898)
new 3f89e50 [TE] expand future time series endpoint to support time-range aggregation (#1896)
new 029ba08 [TE] add default cache expiry 1 hour after write
new 7be8ea5 Implementation of HAVING clause (#1728)
new 06ff4ab [TE] add convenience methods to accept List<String> series names (#1900)
new 1bacc43 [MINION] While cleaning up finished tasks, explicitly remove task contexts (#1894)
new 1762793 [TE] enable fragment matching for auth whitelist (#1901)
new f1ccac3 [MINION] Model multiple tasks from generateTasks() into one Helix Job so that all of them can run in parallel (#1892)
new 94a19b6 [TE] Frontend reroute auth fixes, continued (#1904)
new b1ff412 [TE] Yyuen/fix get graph pnj (#1905)
new c1244c9 [TE] fix merge edit error (#1906)
new 92d7ad8 Adding support for other Kafka Consumers (#1872)
new e13045a [PINOT-5314] Fix leak in OffHeapBitmapInvertedIndexCreator (#1902)
new 6cebc1c Allow alert filter autotune to absorb different feature set from user input (#1907)
new ae96a77 [TE] add hyperbola time-range scoring strategy (#1909)
new 651899f Helix 0.6.8 has the fix for correct number of replicas on a rebalance. (#1911)
new af5e5d2 TE: Adding job Id info in anomaly to enable better debugging. (#1912)
new 84eac4d Remove jetty dependency from broker (#1910)
new 386d5c1 [TE] remove duplicate javax.mail dependency (#1914)
new c16258b [TE] add transformations, javadoc (#1916)
new 7955963 [TE] harden alert scheduler against incomplete/incorrect alert configs (#1918)
new 40b45b8 Enable autotune to tune alert filter for a group of functions (#1913)
new 37648aa Reuse buffer in IntMapBasedHolder processMultiValue() (#1920)
new 0576447 Make maxInitialResultHolderCapacity for group-by executor configurable (#1915)
new 349cbd4 In MCombineGroupByOperator, add early termination to release thread resources (#1921)
new fff6097 [TE] Change investigation comparison from Wow Total to Average values (#1922)
new fc762fb Add percentile functions for percentile locations 10, 20, 30, 40, 60, 70, 80 (#1923)
new 47193f3 fix screenshot issues (#1927)
new b2f4448 logout now redirects to login (#1929)
new b9a141a [TE] Self-Serve Edit MVP2 (#1931)
new add9277 [PINOT-5281] Implement relabance API on the controller (#1928)
new 67c976d Add dateTimeConvert udf to convert a dateTime column from one format to another, and bucket to given granularity (#1926)
new b8afd74 fixing data encoding (#1934)
new 43ee713 [TE] authentication backend fixes (#1936)
new a05c9c3 [TE] Yyuen/expire_session_token (#1938)
new c2d24c5 [TE] frontend fixes 20710926 (#1937)
new ce2b7c9 [PINOT-5350] Protection against file system timeout when fetching segments (#1933)
new 0feed98 Fix the metadata for empty data table (#1935)
new 3c7ba3b Remove stray test data (#1940)
new b037018 [TE] move threadlocal to auth filter (#1941)
new 016a3bc Migrate broker api to Jersey (#1930)
new 8733cee Adding configurable parameters for kafka consumer factories (#1925)
new 01242f9 Tool for backfilling datetime column in pinot segments (#1890)
new 2c9e3e0 On broker side, handle empty response from server (#1939)
new 93309dc Add support for star tree segments with udfs (#1946)
new 06b43d0 Simplify the logic for ScatterGather (#1947)
new 333409c TE Self-Serve Edit Alert: bug fix THIRDEYE-1703 (#1944)
new 0dee04c Adding an API to validate table config (#1856)
new 9487bcc [TE] Self-Seve Alert Create: Graph Dimensions (#1948)
new ab5e0ff [PINOT-5369] Fix segment list API (#1950)
new d39b4cb [TE] Enable HttpUtils to pass Authentication tokens (#1949)
new 37f4af3 [TE] help link update (#1952)
new 3629f72 [TE] upcoming holidays support (#1953)
new 9035784 TE Self-Serve Enhancements (#1954)
new 228175f Add HDFS segment fetcher support (#1919)
new 27a67f4 [TE] Update default SPLINE_REGRESSION_VANILLA properties (#1957)
new 8c08e81 Improve the time column error message (#1956)
new 8f999c5 TE Self-Serve Edit Fixes (#1958)
new 3934d1b Update sample queries in README.md (#1959)
new 0d2b463 Enable fetch metric tool to pass authentication check (#1960)
new 3aa64fb don't show y2 values for events (#1961)
new 90ce38d TE SelfSeve Manage Alerts Sorting (#1962)
new b9f5882 [TE] Dataframe multi index (#1945)
new 92ec489 Remove the SegmentMetadataLoader interface and implementation (#1965)
new 52d4b41 [TE] Remove empty entries from query response (#1966)
new 469a15d Remove FileBasedInstanceDataManager (#1968)
new bf75b7a Ignore CONSUMING state in externalview while reporting segment status (#1967)
new d18dd11 fix bug and move anomaly region with investigation period (#1964)
new 769128c Make consumer directory configurable (#1969)
new 7b4a968 Load segment fetchers through reflection (#1970)
new 4b6b9cb Enable self-serve for confidence interval sign test onboard (#1963)
new 7b19f9c TE Self-Serve graph data granularity (#1972)
new 3e67e98 [RE] prepare rootcause ranking for service matching (#1971)
new 6916a8b Catch LinkageError when initializing fetcher (#1973)
new 0614d10 Upgrade netty version to 4.1.4.Final (#1975)
new 1e1284c Change default query executor timeout in CommonConstants to 15s (same as default in query executor) (#1979)
new 488f27f Fix memory leak issue on broker side (#1981)
new 9242cba [TE] quietly skip pipelines with exceptions (#1982)
new 492c81c fix expired credentials (#1984)
new ed07d46 [TE] Self-Serve Improvements (graph and email validation) (#1978)
new 4361132 [TE] Build index for raw and merged anomaly table (#1986)
new ca97b59 [TE] Self-Serve UI Fixes: dimension call error (#1985)
new 748a66f [TE] remove stray debug output to console (#1990)
new 0b8461e Fix configuration on mapreduce job (#972)
new bdac580 Remove SegmentId, SegmentIdSet and clean up related classes (#1955)
new 167cf07 Add partition width metric (#1991)
new 440d860 Update partition information of SegmentZKMetadata during commit (#1992)
new 149aada fix the jumping around (#1994)
new bf89056 [TE] Frontend speedup custom c3 (#1988)
new 10e7373 [TE] bump custom c3 version (#1995)
new 19cbf16 Refactor on-heap/off-heap inverted index creator to prevent leaking resources (#1989)
new 26cd93c Pass property store and server metrics into InstanceDataManager and TableDataManager (#1977)
new 291abe4 [TE] Self-Serve Manage Alerts: UI Improvements (#2001)
new 871e98a Primary metric is now unselectable (#2003)
new dde2a8a Fix for a segment having no statistics (#1999)
new 3560946 Improvments to the dateTimeConvert udf (#1996)
new 57efb0f Handle a hash value of Integer.MIN_INT (#2008)
new 014c16e [TE] AllowUpdateFunctionWithOverrideProps (#2002)
new 5cd3e3d Add access control to controller APIs (#2010)
new 1399bb1 Adding an API to validate a schema (#2009)
new 6d2b603 [TE] implement default authenticator to support non-ldap environments (#2012)
new d33da96 Altering mapper and segment creation to be compatible with PinotBuildAndPushJob (#1997)
new 36417ab [PINOT-5138] Provide access to metrics registry (#2013)
new 458c9b0 Adding API to get the controller metrics registry (#2016)
new 1c131af Adding ability to configure kafka message decoder based on factory (#2019)
new 9a5df01 [TE] display exact, localized deployment timestamp in label (#2011)
new fe59c0b fix file upload not working against latest pinot controller (#2006)
new 3150279 Fix off heap inverted index creation on MacOS X (#2015)
new 712802a Add data table to query console (#2014)
new 1360d40 [TE] Frontend dataset mapping support (#2017)
new a46182f Add StarTreeIndexSpec and hllConfig to segmentValidationAndRetentionConfig (#1983)
new 01ca43d fix metrics splitView (#2024)
new 88c820c Split overloaded api 'indexOf' in dictionary into 'indexOf' and (#2022)
new a99afa0 Added object spread operator to eslint (#2026)
new c36cdff [TE] entity filter component - part 1 (#2023)
new f079991 [TE] Decouple Pinot data source and cache loader for future extension (#2018)
new 432232b image in email now also links to the investigate page (#2028)
new 9b95396 [TE] Unify DB Dao Registry in prod and test (#1998)
new ca471db [TE] Self-Serve-Edit - bug fix (#2031)
new 9133bef Fix and commented out 42 failing tests (#2029)
new 4c798a5 [TE] frontend - show contribution table for investigation period only (#2032)
new 0d7592b Add request ID to broker connection logs. (#2027)
new f9805f3 [TE] Added new end points to mirage config for upcoming acceptance test for edit alert (#2035)
new d9b5d4d Change AccessController to be post-match (#2037)
new ea9c4d8 Added an option to allocate Direct Offheap memory for realtime consuming segments (#2038)
new ddb56c1 Fix lease extend protocol handling in the controller (#2040)
new 4e0b842 [TE] Distinguish different anomaly result source (#1993)
new 855ac0e adding link (#2042)
new 5b6e467 Limit the number of segment builds on a server (#2041)
new 55fab91 [TE] Added more acceptance tests for edit alert to check for fields rendering correctly. (#2044)
new c2ea7ec [TE] fix timeseries endpoint and ingraph interoperability (#2043)
new 02aef39 [TE] Resolve most duplicate class issues and remove old version of kafka client (#2047)
new 6ea1fc7 [TE] Bump Guava to version 20 for new methods (#2049)
new 91b0eb1 TE Self Serve Edit Alert Fix: missing cronExpression (#2046)
new 70e272f Fix the problem of directly memory leaking in scatter-gather (#2045)
new 354ad3f [TE] Add force-bump-up section in pom file for LI usage (#2051)
new b16a592 Fix the default config value for direct mem allocation (#2053)
new 484d7cd Handle consumder dir not configured, and cleanup old consuming mmap files if any (#2048)
new c4a5660 Do not put trace info in DataTable when trace is not enabled (#2052)
new a026116 Simplify the anomaly function implementation (#2054)
new 108a1a5 Setting on heap to true in hadoop job (#2055)
new 404d82f [TE] frontend - set up for rca revamp (#2060)
new 4ae951b Refactor Trace for performance concern (#2057)
new 7471cf2 [TE] Move metricAsDimension config from dataset to metric level (#2063)
new cd4897a [TE] frontend - Initial outline of new component (filter-bar) (#2062)
new 4fa55c6 Reduce the temporary files generated by star tree builder (#2025)
new ecba761 [TE] frontend - rootcause poc, basic controller (#2058)
new d5014e2 [TE] frontend - implement ability to toggle filter blocks (#2064)
new 7fb25a5 Remove the on-heap version of the star-tree (#2066)
new 08dd8e5 [PINOT-5455] Added a hook for access control in the broker. (#2065)
new b9064b9 [TE] Self-Serve Alert Page V1 (#2050)
new b220d7f [TE] frontend - added filter by event functionality for filter bar (#2069)
new 9374f9e [TE] rootcause - metric analysis pipeline with quantile predictor (#2036)
new 3f15663 [TE] Rootcause POC - heatmap component, event layouting, metric baselines (#2068)
new 5b58528 [TE] rootcause poc - aggregation endpoint (#2071)
new e6bf94d Adding segment push retries (#2061)
new e616fa7 [TE] yyuen/rca poc summary settings component (#2072)
new 90d2df9 [TE] Decouple Pinot ResultSet from ThirdEye's data storage (#2074)
new fac4f4e [TE] self-serve-replay-wrapper-improvement(#2077)
new 47803c1 [TE] rootcause poc - aggregation loading, heatmap, and frontend rollup (#2076)
new fa376e3 [TE] Enable Performance Evaluation when User Report Anomaly (#2075)
new ef9b79f [TE] dataframe parser utility for ThirdEyeResultSetGroup (#2079)
new f470029 [TE] rootcause entity attributes for holidays, anomalies, metrics (#2078)
new 7824a5f [TE] frontend - added filter bar to rootcause (#2082)
new e8f7413 Fix the consumer directory test in LLCRealtimeClusterIntegrationTest (#2083)
new be4bead [TE] rootcause poc - add legend component (#2084)
new 24f0db2 [TE] rootcause poc - support multiple different display modes for timeseries (#2081)
new e090c7f [TE] Create tables and DAOs for alert email refactoring (#2067)
new 5db0bbe [TE] rootcause poc - add metrics section (#2089)
new 8746388 Remove redundant sout (#2086)
new 42e3481 Metadata Change Notification Framework (#2073)
new 721974c [TE] add holiday mode to email scheduling (#2094)
new 9900ee5 [TE] frontend - populate filter bar dropdown on /rootcause (#2092)
new 474e8fa [TE] use dataframe utils for timeseries resource (#2096)
new 0e359a8 [TE] add high-level summarization to aggregation endpoint (#2098)
new ba9c763 [TE] rootcause poc - caching services, controller refactoring (#2093)
new 5751a31 avoid creating alert subscription group with duplicate name, front-end already enable the duplicate checking, this change will prevent it happening at the database level, the current production database is already reset to these settings (#2087)
new 9ab458b [TE] rootcause poc - legend prototype with categories, sorting, and selection helpers (#2100)
new e62d28b [TE] rootcause poc - cache inconsistency fix on empty result (#2101)
new 30098c8 Remove BlockId class and nextBlock(BlockId) API in Operator (#2099)
new 9342212 [TE] rootcause poc - chart metric split view support (#2103)
new ad203f9 Add support for dimension datatypes other than string (#2097)
new 7cabeaa [TE] Fix issue when parsing empty GroupByResultSets (#2104)
new ebdce57 Cleanup of DashboardConfig and related code (#2107)
new 6d5e406 [TE] front end - Fix filter bar bugs (#2105)
new bf53cfc [TE] Fix the issue that dimension values unexpectedly become null (#2110)
new 7d9d598 [TE] Yyuen/rca poc summary settings logic (#2091)
new a36e67d Add support for hll config in CreateSegmentCommand (#2070)
new c164889 Make EmptyFilterOperator a singleton class (#2113)
new 509acfa [TE] rootcause poc - add chart subchart and zoom (#2109)
new ee9e549 [TE] rootcause poc - basic metric tabs with sorting (#2108)
new 3b1db92 Refactor PredicateEvaluator to include information in Predicate and only pass PredicateEvaluator to the FilterOperators (#2112)
new 27da8b5 [TE] dataframe - remove raw data columns before returning parsed request (#2114)
new 6128759 [TE] rootcause poc - abstract entities support (#2116)
new b78088d stable heatmap (#2117)
new b40aca6 Enable search page to show Notified anomalies (replace previous logic to use alert filter) (#2080)
new c0188a4 [TE] initial styling for the new rootcause page (#2118)
new cf0d3d6 [TE] temp fix for filter bar (#2122)
new aebf303 [TE] Append Pinot tables' custom configs to ThirdEye's dataset config (#2115)
new 92b4e17 [TE] Add try-and-catch block to protect the initialization of cache loader. (#2123)
new 70d10a3 [TE] Implement Anomaly Fetcher (#2090)
new 1b5fd87 [TE] rootcause metrics styling (#2124)
new a40be52 [TE] Rootcause frontend heatmap scrolling (#2125)
new f7e4ab2 [TE] user anomaly reporting enpoint (#2095)
new 17d8744 Enhance StarTreeIndexBasedFilterOperator to support multiple predicates on the same dimension (#2085)
new 82d3e08 [TE] add endpoint to change alert group config, active/deactivate holiday mode by add/rm holiday cron (#2126)
new 8c9c294 [TE] fix loadtest to include current and baseline metrics (#2128)
new 5e4d18a [TE] rootcause poc - heatmap glitch fix (#2127)
new 774f873 [TE] anomaly feedback for time range (#2119)
new 16f0a9c Remove the API open() and close() in Operator (#2131)
new f251e27 [TE] rootcause poc - loading indicator for aggregates and breakdowns (#2132)
new b641e2ac Adding json string serialization/deserialization for startree (#2133)
new 1d9a20b Minor improvements (#2134)
new 6575bf7 [TE] Email endpoint extension for holidays, alert-grouped anomalies (#2120)
new 40ab549 [TE] rootcause poc - support user-selected related metrics (#2130)
new 1cea239 [TE] Rootcause frontend heatmap metric selection (#2129)
new 31f9a2b [TE] rootcause poc - settings time picker refactor (#2136)
new 23aa764 [TE] Holiday Email template
new 9821c0d While building star tree, call flush() explicitly to reduce IO (#2137)
new 516d9a1 [TE] email - holiday event email fix (#2140)
new b6ed5b5 [TE] email event range fix (#2142)
new 56f8920 [TE] typo fix (#2143)
new fbe947c [TE] rootcause poc - related metrics sorting by score (#2141)
new 4553b00 [TE] rootcause - shorten holiday label (#2139)
new a34f417 [TE] rootcause poc - dynamic primary metric (#2144)
new 2ac1ab1 Enable Star Tree index when StarTreeIndexSpec specificed during segment generation. (#2145)
new a60c5e7 fix bug for alert page (#2147)
new 7d0489c [TE] email - fix holiday mode schedule restart code path (#2148)
new 92e47b0 [TE] Add Serializer/Deserializer for TE ResultSet and Create Pinot query proxy (#2146)
new 9ec212b Fix exception when getting group-by column count on empty results (#2106)
new 178dadc Removed rca-poc route and rca-poc-table component (#2150)
new 36ab325 missing import (#2151)
new e49849f [TE] frontend - Fixed filter-bar bug with reloading new entities (#2152)
new 096a70e Clean up Star Tree specific configs in SegmentGenerationConfig. (#2153)
new c68f592 [TE] yyuen/modal component (#2154)
new 579bee0 [TE] hadoop - adapt to star tree index api changes (#2159)
new 8ca64b8 reinstate overridden pods prop (#2157)
new 233503e Override ember-models-table in rca to make it responsive to external changes (#2158)
new b60179a [TE] rootcause poc - entrypoints (#2156)
new cc3f905 [TE] Get Minimum Time to Detection for Function (#2155)
new 646b444 Fixed bug with rca table checkboxes not responsive to interaction with legend or row click (#2160)
new a749e5c [TE] rootcause poc - fix anomaly dropping from context on primary metric change (#2164)
new 7f37944 [TE] rootcause poc - clean up console.log() calls (#2165)
new dfce2d4 [TE] rootcause poc - chart tooltip (#2161)
new d46f77d te-modal wrapper component (#2168)
new 090bc63 [TE] rootcause poc - unified coloring for metrics, events (#2163)
new f4119a5 Change /eval/filter/{functionId} from POST to GET (#2169)
new 15c2d18 [PINOT-5611] Remove FileUploadUtils dependency on hadoop (#2170)
new 9890a7e [TE] rootcause poc - anomaly feedback support (#2166)
new 9b4bff3 [TE] Add HTTPS client for Pinot's auto onboard of metrics (#2172)
new b7015e2 Ttbach/bug fix filter bar (#2175)
new 9a0c373 [TE] rootcause poc - remove chartOnHover console logging (#2174)
new 841515d [TE] rootcause poc - fix loading indicator, double loading of data (#2173)
new 058b3e2 [TE] Use dataframe in thirdeye response (#2176)
new 5058193 TE Self Serve - enabling dimension toggles on metric graph (#2111)
new e461e0f [TE] frontend - filter bar styling (#2180)
new 15d5f9d Change AccessControl API and only call it when necessary (#2177)
new a919ead [PINOT-5458] Fix SegmentFetcherFactory to allow overriding fetcher classes via configuration (#2171)
new a41215a [TE] frontend - rca table fixes (#2181)
new 9ab5312 Added a getter in HelixBrokerStarter to get the access control factory (#2182)
new 203939e [TE] Anomaly Feed Implementation (#2149)
new 5051ecd [TE] rootcause poc - session storage (#2183)
new 73c9be1 [TE] Revert "Use dataframe in thirdeye response" (#2185)
new 55d0ac3 Enable support for splitting predicate values for NOT-IN clause. (#2188)
new e3961e3 [TE] frontend - ttbach/rca header styling (#2189)
new 4943c41 [TE] rootcause poc - get-safe helper to gracefully handle urns with . (dot) (#2191)
new b19a096 Refactor RecordReaders, fix bugs and simplify logic (#2187)
new 95dad2b tooltip template helper (#2186)
new 9bf9e02 [TE] hadoop - adapt to pinot api changes 2017-12-07 (#2196)
new aad0a45 [TE] Alert Page MVP (#2179)
new 2e3442e [TE] rootcause legend styling (#2193)
new afabc52 [TE] rootcause poc - remove rootcause-filter dummy component (#2192)
new f06ec86 [TE] frontend - ttbach/rca anomaly styling (#2195)
new fcf1a26 small tweaks (#2197)
new f2210fc [TE] rootcause poc - chart toolbar (#2190)
new ed94fc7 [TE] Legend styling tweak (#2206)
new 9813b12 Fix SegmentFetcherFactory configuration to be different for controller and server (#2200)
new 5be05f1 Moving uploading new segment logic to make it separate from refresh (#2203)
new a44ceea [TE] Implement Email Formatter (#2184)
new 690d1dc Adding segment push status and custom metadata (#2207)
new bbd12ed [TE] rootcause poc - basic investigation report header (#2194)
new 01a926c [TE] Create Alert Flow Bug Fix (#2199)
new a3ef6de Add DictionaryBasedAggregationOperator to support for max, min, minmaxrange (#2102)
new 8ab0413 Adding unified download method for minion (#2201)
new 148dc60 fix bug (#2209)
new 77e67ba [TE] frontend - ttbach/rootcause ui fixes (#2212)
new 9cc41ed [TE] rootcause poc - message about unavailable data for heatmap (#2211)
new 1092776 [TE] rootcause - shorten anomaly entitiy label (#2205)
new 0faaab3 [TE] rootcause poc - filters help text (#2215)
new 921f1a2 Fix percentile int overflow (#2220)
new df81a20 implemented cards loader and absolute loader (#2218)
new 21f3ab4 [PINOT-5458] Adding Https fetcher (#2210)
new 1e53bf2 [TE] Fix anomaly detection resource return type (#2217)
new 9f71a35 Enhanced raw index readers/writers to support uncompressed data. (#2204)
new afce741 [TE] rootcause poc - collapse anomaly header on existing feedback (#2214)
new 272a173 [TE] rootcause poc - swap anomaly and analaysis headers (#2222)
new 07688a8 Add ability to inject custom code after segment push (#2225)
new 9a96c63 Set start and end time for pinot push in the properties before sending to custom event (#2227)
new e065b2c [TE] rootcause poc - default alignment to days for anomaly analysis range (#2226)
new 0593e87 Clean up dependency issues (#2221)
new 7da7e2f [TE] rootcause poc - fix fetch import, remove cryptojs (#2229)
new 1cf6bcd Empty field value handling on alert creation (#2231)
new a5af8c4 [TE]ProjectedAlertFilterPerformanceEvaluation (#2198)
new 7daa169 replaced string with icons (#2224)
new d232cec [TE] Extend Autotune Endpoint for Performance Tuning (#2232)
new f04a17a [TE] Create detection onboarding execution pipeline (#2230)
new 33c864c [TE] rootcause poc - comparison range layout, heatmap drilldown (#2234)
new 0832173 [TE] UserReport Anomaly: feedback update by dimension (#2233)
new d7d6d0f [PINOT-5458] Remove static initialization of classes in SegmentFetcherFactory (#2238)
new e870e25 Adding minion specific logic to segment upload (#2216)
new 1236e4a [TE] Fix null pointer exception occurs in data report (#2235)
new 0a5cb69 [TE] Enhance the constructor of dimension map (#2237)
new 5365a81 [TE] Preview anomalies by autotune (#2236)
new c47afb2 [TE] frontend - ttbach/edit alert test (#2241)
new 0fb3450 [TE] rootcause poc - disable scrolling in heatmap (#2240)
new 994e789 different height for split mode (#2243)
new 8bb2c69 [TE] frontend - ttbach/event table fixes (#2242)
new d373bf1 fix legend width and collapsing chart (#2245)
new b14c7ff fix heatmap tooltip (#2246)
new 87be4e3 [TE] frontend - Updated test with await/async (#2247)
new eaf844d add missing async keyword (#2249)
new e383b64 [TE] rootcause poc - add user-readable alerts on data error (#2244)
new 7162f42 fix loading spinner (#2250)
new fbb9787 Purge Task (#2252)
new fb99f9d [TE] rootcause - basic support for metric entities with filters (#2239)
new f40099d Adding SegmentFecherFactory to pinot-minion (#2253)
new 7365af5 fix missing props in cookie (#2251)
new 94a8e88 [TE] rootcause poc - support for tracking rca sessions via anomaly id (#2256)
new 8a5be9b [TE] rootcause poc - fix regular event end (#2254)
new ae5956c [MINION] Add BaseSegmentConversionExecutor base class for all minion segment conversion tasks (#2258)
new 819677a [TE] rootcause poc - chart styling (#2263)
new 7cae017 [TE] Enable Anomaly Feed and Email Content Formatter (#2213)
new bb645d1 [MINION] Add PurgeTaskExecutor (#2266)
new 044d0c1 [TE] rootcause poc - input field labels and tooltips (#2264)
new a3cb045 [TE] frontend - ttbach/add radio button to rootcause anomaly (#2269)
new 5520657 style change (#2267)
new 0e475dc Changing metadata field name (#2272)
new 73c1704 [TE] rootcause poc - add metric attributes for inverse, derived, additive (#2265)
new 96863f4 changed header style (#2274)
new 21f14e8 [TE] frontend - ttbach/minor ui fixes (#2275)
new da1b31a tweaked styling (#2277)
new 4556165 [TE] rootcause poc - details in heatmap tootltip (#2278)
new ac6e483 [TE] Implement Onboarding Job and Tasks (#2273)
new 9057f52 [TE] rootcause poc - fix heatmap defaults, tooltip on different granularities, null fill (#2276)
new 842d2d0 [TE] Alert Page: report anomaly + routes for tune/edit (#2228)
new 5822be2 TE modal fix (#2281)
new 8e5befb Adding purge time to metadata (#2257)
new 1cda55c Remove dateTimeType from dateTimeFieldSpec (#2287)
new 808b20a Adding json ser and deser for hll (#2285)
new 84cfcee [TE] frontend - ttbach/padding fix (#2288)
new e200f49 Make RealtimeIndexOffheapMemoryManager re-usable (#2290)
new 9f28a99 [TE] Add prefix 'Thirdeye Alert : ' back to email subject (#2292)
new ee3ffa8 Fixing NPE in Segment Upload (#2294)
new d1f7368 [TE] frontend - ttbach/rca metrics table (#2293)
new a43fc8f [TE] frontend - ttbach/anomaly-overview-fixes (#2297)
new 856f166 [TE] frontend - ttbach/rca text changes (#2298)
new 53239c5 Update the metadata event notifier framework (#2262)
new 1797e4a Change compression type to pass-through for raw indexed realtime metric columns (#2296)
new 5d7df32 Add timezone support in dateTimeConvert udf (#2295)
new fd7367d [TE] Alert performance by application (#2291)
new 4863088 [TE] Self Serve Alert Tuning (#2283)
new 1caf01a Revert changes and fix bugs on detection (#2303)
new 2932ce1 [TE] frontend - quest for unbreakage (#2305)
new f9a143a [TE] frontend - Added number of events to filter bar (#2304)
new e114463 Refactor FileUploadUtils to use HttpClient 4.5.3 (#2301)
new f5c3e11 issue-2270: Adding support for thrift (#2306)
new e5e2fb5 Added a method to get a status description from ServiceStatus (#2315)
new 5350ac9 [TE] rootcause poc - offset data loading and metrics table display (#2313)
new 000dfe6 [TE] frontend - Minor ui fixes (#2317)
new 31d65f0 [TE] rootcause poc - anomaly entity link fix (#2316)
new 0b4a2a0 Implement the segment upload protocol (#2309)
new ed29d21 [TE] rootcause poc - external urls for anomaly entities (#2318)
new 02f6830 Added static method to get the status description string (#2319)
new 420e6d1 [TE] Performance Tuning Bug Fixing (#2312)
new 23d3425 Add support for long queries for Pinot query console (#2322)
new 9f99507 [TE] frontend - ttbach/add rca anomaly links (#2320)
new b97a764 [TE] Setup standalone mode for Summary algorithm (#2308)
new 9c6ae70 Add a controller gauge when short of instance (#2299)
new 73a90a7 [TE] Fill in Function information on backend automatically (#2311)
new 8694766 Fix the flakiness in ConvertToRawIndexMinionClusterIntegrationTest (#2326)
new aca81b6 Support HTTPS for file upload/download (#2323)
new aa8bbdb Minimize window in which two controllers can update metadata for LLC segment commit (#2321)
new b93263e [TE] rootcause poc - support session anomaly urns (#2329)
new 9e83909 [TE] Add graceful error handling for pinot data source (#2328)
new fded7bc Remove having clause from the query generator for integration test (#2327)
new 2883183 [TE] rootcause poc - add more info to heatmap tooltip (#2330)
new 0d56a50 Fixed the flaky test for StringPredicateEvaluator (#2331)
new 27ed2f0 [TE] rootcause - extended metric analysis (#2279)
new 753a005 events now have points (#2333)
new 352a131 Let RetryPolicy throw exception when number of attempts exhausted (#2324)
new 5992d11 [TE] Yyuen/rootcause anomaly fix (#2336)
new 34d02dc Ttbach/anomaly baseline info (#2334)
new ee5ff7d [TE] rootcause poc - hook up rca v2 backend (#2335)
new 77e3fd7 [TE] rootcause poc - support inverse metric color in tooltip (#2337)
new 130758d [TE] rootcause poc - prevent newly loaded session from displaying as modified (#2338)
new 33a9aba [TE] rootcause poc - prevent uncaught error on duplicate removal of timeseries (#2339)
new 5aa249b Use dictionaryBasedOperator only if dictionary is immutable, hence guaranteeing that it is sorted (#2343)
new f784691 [TE] rootcause - external links for metric entities (when possible) (#2332)
new cd91155 [TE] one step towards mock parity (#2342)
new f18077a [TE] frontend - ttbach/add session update (#2344)
new a17bc18 [TE] rootcause poc - support special characters in urns (#2345)
new da2129f [TE] rootcause poc - prevent stale data load (#2347)
new 2c2de72 [TE] rootcause poc - utils jsdoc (#2348)
new cfbebe5 small fix (#2349)
new 046b533 [TE] rootcause poc - support inverse metrics in anomlay header change indicator (#2350)
new 07accd6 allow responsive graphs (#2352)
new 7d7577d [TE] Self-Serve tuning flow 01: associated components/utils (#2351)
new 5c39720 allow for more granular x axes (#2353)
new c1cd79e Simplify the constructor to build FileUploadDownloadClient with SSLContext (#2357)
new c89f66a [TE] rootcause poc - warn on non-additive dataset in heatmap (#2355)
new d276510 Added more logs in https segment fetcher, and removed logging of passwords (#2356)
new cc87801 [TE] rootcause poc - backend partity for frontend urn encoder (#2359)
new 541e61c [TE] Handle NaN, Positive and Negative Infinit in getRoundedValue method (#2363)
new 9de76db Added on-heap dictionary support for int/long/float/double data-types. (#2341)
new c578f90 [TE] rootcause poc - fix urn encoding when adding filter from heatmap (#2360)
new 27f99b0 Read all certificates from certification authority file. (#2362)
new ec8fa3f [TE] rootcause poc - controller cleanup and jsdoc (#2366)
new 15d8c5d [TE] rootcause poc - chart log-transform 0 fix and jsdoc (#2367)
new 397335e [TE] rootcause poc - expand anomaly header by default (unless it has feedback) (#2365)
new 0e01a93 [TE] frontend - Added links to rca metrics (#2368)
new 1d036da Fix duplicate LLC segments possible when lead controller goes through a huge GC (#2361)
new 4daa54f Build shaded jar for pinot-common (#2369)
new eee0075 [TE] rootcause poc - metric card sorting cleanup (#2364)
new ba32393 [TE] rootcause poc - data availability indicator (#2370)
new a037964 [TE] Self-Serve tuning flow 03: preparing create-alert for new flow (#2358)
new 27978b4 [TE] Self-Serve tuning flow 04: Alert page overview (#2371)
new f7d01b4 [TE] frontend - ttbach/Fixed constant anomaly loading when there is none to load (#2374)
new fc97187 [TE] rootcause improvements and bug fixes (#2373)
new 2f8c53d [TE] rootcause poc - time-based session modification check (#2375)
new 8ecd00a [TE] empty state for root cause (#2377)
new 7eae22c [TE] Self-Serve tuning flow 02: refactoring edit alerts + misc (#2354)
new 39f6ce4 [TE] rootcause poc - data indicator copy (#2379)
new cae6f1c Let RetryPolicy.attempt() throw AttemptFailureException instead of general Exception (#2380)
new fdde3bb fix test (#2378)
new 8730456 Shade librarys under org.apache.http in pinot-common to avoid conflicts with hadoop dependencies (#2382)
new 7f6b1c2 Fix the issue where status code 500 is counted as a permanent exception (#2385)
new b33f6e6 [TE] datasource - support timeseries agg functions (#2384)
new c680ddf [TE] Self-Serve tuning flow 05: sensitivity tuning UX (#2372)
new 39a9661 [TE] rootcause poc - save vspace in anomaly header (#2383)
new c0bc1e6 [TE] rootcause poc - fix missing fetch import (#2390)
new aff6efb [TE] rootcause poc - suppress error-messages for timed check in background (#2392)
new 0997cee Fix the bug where version of ZNRecord is not set properly (#2395)
new f36cc83 [TE] anomalies - add result source to anomaly details wrapper (#2394)
new f27c2ae [TE] frontend - ttbach/Fixed styling of error messages on rca (#2396)
new 8c9e07c [TE] Self-Serve Onboarding Flow: handling transition to alert page after creation (#2391)
new 55c0b16 Ttbach/refactored utils file to extract rca-related utils (#2397)
new dc66df1 Re-factor PinotLLCRealtimeSegmentManager methods to reduce the number of arguments. (#2404)
new bed95aa Enhance the exception handling for segment upload (#2401)
new 78afab1 [TE] onboarding - reverse proxy for detection-onboarding (#2405)
new fd61de5 fix rca route (#2410)
new bea7d1a [TE] rootcause - forward to latest session for anomaly id (#2407)
new b207f3a [TE] admin - fetch all anomaly functions instead of active only (#2411)
new 66d0448 [TE] datasource - fix avg function with multiple metrics (#2406)
new f003003 [TE] rootcause - prevent external null links (#2399)
new 55c8fb7 [TE] rootcause - return NaN for aggregate of non-existing data (i.e. future) (#2398)
new 6c9efba [TE] Onboarding jobs (#2386)
new 1edd963 [TE] rootcause - fix metric spinner stuck on non-existing metric (#2415)
new 3198268 Ttbach/created utils folder (#2402)
new 8b755b9 working redirection from legacy to ember app (#2417)
new e65dbca [TE] Self Serve Onboarding: apply new onboard task manager API (#2409)
new 3f86410 Refactor SegmentFetcherFactory to load only defined protocols (#2408)
new eb5b9e7 [TE] Implement Onboard Email Content Formatter and Tasks (#2412)
new 5fe3d2d [TE] fix configuration only returns partial value (#2419)
new 40b5a98 [TE] frontend - Translate granularity dropdown options to human-readable format (#2423)
new f3a059f Add back the setters for star-tree configs in SegmentGeneratorConfig (#2420)
new 4680dc2 Working realtime table config (#2421)
new d95d3c5 Pass memory used by segment to the controller on segment completion, for using it to tune end criteria in future (#2413)
new 6eb7f51 TE concealing link to new self-serve flow (#2433)
new c681bdb [TE] Yyuen/rootcause graph interaction (#2434)
new 7b8ffb5 TE Self-Serve linting error cleanup (#2422)
new 02c4797 Move LLC segment completion to use FileUploadDownloadClient. (#2430)
new b71b68c Removing old jersey jars (1.9) that were pulled by hadoop-common transitively (#2427)
new 7e96f56 [TE] Self-Serve Onboarding 02: enhancements (#2435)
new 628b8fb [TE] frontend - Ttbach/acceptance test rootcause empty state (#2429)
new a3601d1 [TE] Add email recipients when an alert group exists (#2426)
new c2dd6c9 [TE] frontend - ttbach/create translation util (#2424)
new 91fb42a [TE] frontend - Ttbach/set default tabs (#2436)
new 4e78d33 Fix function id comparison exception (#2438)
new 31b1dcb Clean up interface IndexSegment and SegmentMetadata (#2432)
new cd0f018 Fix anomalies number shorter than the actual generated (#2441)
new 07cf990 Modify murmur function to always use UTF-8 bytes (#2431)
new 2ca5b0b Remove QueryPlan related classes (#2440)
new 2bfc5f2 fix metrics card responsiveness (#2442)
new 65cd037 [TE] Onboarding tasks: initial trigger auto-tune with pattern only, anomaly severity score (#2389)
new 7db9074 copy change (#2444)
new 51c203b [TE] Self-Serve Onboarding 03: enhancements (#2443)
new a6c26f7 [TE] frontend - Acceptance test for rootcause metric (#2446)
new 9096ffa Enabling a table also resets errored segments (#2400)
new 6838caf Clean up PR, with only codes related to the walkaround solution (#2445)
new d23ddbe Fix onboarding email exception with no anomaly to be reported (#2447)
new 16fc79f [TE] frontend - Ttbach/acceptance test for rootcause session (#2448)
new 98cbcc1 Clean up RealtimeSegment interface and SegmentMetadataImpl class (#2439)
new b9f3e0f Create a command to check the cluster state (#2425)
new cfc9894 [TE] Bug fix for user report to add function Id (#2449)
new 47c5ead [TE] Self-Serve Onboarding 04: enhancements (#2451)
new 933d193 Create controller gauges for realtime/offline table sizes and quota (#2376)
new fe460bd [TE] Fix funciton id with thousand separator (#2453)
new a3f0cd0 Refactor RealtimeSegmentImpl (#2450)
new d93663c fix test (#2454)
new 2d47c11 [TE] frontend - Ttbach/acceptance test rootcause anomaly (#2456)
new 659a797 [TE] rootcause - re-enable copy, allow save by owner only (#2455)
new 7466094 [TE] Yyuen/fix failing components tests (#2458)
new 940ca14 Fix the bug where rewind() does not work properly in CSV and JSON record reader (#2457)
new 7af6b66 [TE] rootcause - limit rootcause anomaly and analysis ranges (#2459)
new 933c729 [TE] rootcause - adjust display range when investigation out of bounds (#2461)
new d3ef5c2 Fix the API for reload segments (#2464)
new fe2d7f8 Uncomment test which was commented out by mistake (#2462)
new 85d4fa7 [TE] frontend - added test to save session on rootcause page (#2460)
new 8e56a11 [TE] frontend - Ttbach/rootcause tabs test (#2467)
new 84f377f fixing some linting errors (#2466)
new f318abc [TE] Self-Serve Onboarding 05: enhancements (#2468)
new 9a4b6b6 [TE] frontend - ttbach/remove edit button on rootcause page (#2469)
new 28e3594 Re-implement FixedByteSingleColumnSingleValueReaderWriter class to (#2463)
new 4af6d44 Fixed failing acceptance test (#2470)
new 8bbb5fe Clean up equals/hashCode for all configs (#2471)
new 510c2ed Partition assignment support in multi tenant setup (#2428)
new 4e4ab11 Re-factor: Code re-factor to separate out realtime specific references (#2476)
new 02ffe22 [TE] Self-Serve Onboarding 07: enhancements v1 (#2474)
new e7aaac7 When serializing schema to Json string, only preserve necessary fields (#2473)
new 881a099 Using preferred protocol and port for realtime segment upload (#2475)
new fb125d7 [TE] Send out notification upon exceptions (#2477)
new 3cc1e41 Add containsNullCharacter() and removeNullCharacters() methods into StringUtil (#2483)
new 103f829 [TE] rootcause - fine-grained loading (#2484)
new fe6e083 [TE] Self-Serve Onboarding 08: enhancements v1 (#2485)
new 60337be In MetricFieldSpec, only serialize non-default field size (#2489)
new f617059 Revert Schema main method (#2491)
new 2f0e003 Fix the NPE in MetricFieldSpec.setFieldSize() (#2493)
new 96ca68d [TE] Self-Serve Onboarding 09: enhancements v1 (#2492)
new 5e2784f Use Async API to upload segment (#2490)
new 91515c0 Add failure message in entity of response from uploadSegment (#2488)
new f6057c7 Enforce padding character to be '\0' (#2495)
new cc5ef44 [TE] datasource - use dataset maxtime when fetching filters from pinot (#2487)
new 8c1fd0c [TE] rootcause - auto populate selection (#2486)
new 350e986 [TE] detection - abort long-running detection task on interrupt (#2498)
new f4f76bf In SimpleMinionClusterIntegrationTest, manually control the task execution process instead of using sleep (#2496)
new f6621e1 Fix default for ssl server certificate validation (#2500)
new ccaa9f5 [TE] update all thirdeye to the newest module API (#2479)
new 5cb7839 Fix LLC consumer to handle segment build fails gracefully (#2499)
new 4109035 Faster hourly detection (#2504)
new d0e238d Implementations for off-heap FixedIntArray<->Integer map, and single-value multi-column reader/writer. (#2494)
new 4e6f18f Fixed ClusterStateVerifier to consider timeout (#2497)
new d50ea95 [TE] frontend - Ttbach/add baseline scores (#2480)
new 3d941af Disable data completeness check for functions besides Daily (#2510)
new 54963ae [TE] rootcause - dimensions entity for multi-filter support (#2506)
new 1aa6f81 [TE] rootcause - logger settings for RCAFrameworkRunner (#2503)
new 9c5eeba Fix anomalyFunctionDAO typo (#2511)
new adedbe0 [PINOT-5857] Fix off-heap dictionary to return UTF-8 encoded String objects. (#2508)
new 0471e60 Realtime servers missed controller leadership change (#2513)
new a29534b [TE] rootcause launch branch (#2512)
new ad6337d Fixing the overflow issue for FixedBitSingleValueWriter (#2516)
new d4e9310 Utilize latest version of Helix for Pinot (#2452)
new 0ef6d5f [TE] rootcause - clean up metrics view (remove cards) (#2517)
new bab4c45 fix (#2515)
new b54ac44 [TE] Self-Serve Onboard/Tuning flows: perf enhancements + links (#2518)
new 67be7cc Allow user to specify dimension for baseline fetching (#2521)
new 4de0daa [TE] frontend/ ttbach/Refactored card-container in rca (#2519)
new 54f0e49 [TE] Ignore completeness entries that does not need to be updated (#2525)
new 21880ca [TE] fix replay email link (#2524)
new 3c620b3 Set default size 1 to dictionary, so that we handle empty value scenario (#2522)
new b06641c [TE] Self-Serve Onboard/Tuning flows: perf enhancements 02 (#2528)
new 2569cfb Issues 2307 pinot output format (#2388)
new 7d386c8 removed config group name (#2531)
new 84e9cd8 Add 0 value for LLC_PARTITION_CONSUMING gauge whenever we stop consuming (#2523)
new 1a254d7 [TE] Self-Serve Onboard/Tuning flows: copy changes/misc (#2534)
new fb20aea [TE] detection - discard raw anomaly results (#2535)
new f310e47 [TE] admin - fix create dataset endpoint (#2532)
new 4fdbfb0 [PINOT-5728] Backing out changes to SegmentStatusChecker to get table sizes every 5 minutes (#2536)
new d602671 [TE] Self-Serve Onboard/Tuning flows 02: pre-launch adjustments to UX (#2538)
new 008cc2e [TE] frontend - Ttbach/created shared config folder (#2540)
new 1ea26c2 [TE] frontend - ttbach/created new home route (#2541)
new 66dfd67 [TE] Include number of user report anomalies to total response rates (#2530)
new d09dac0 [TE] rootcause - use d3 as default formatter (#2527)
new 4982bf1 Backtrack to helix-0.6.8 since we seem to have some issues with helix-0.8.0 not having a lead controller (#2544)
new b035f89 Support updatable metrics during realtime consumption. (#2537)
new 34e10e8 Remove duplicate starTreeConfig from tableConfig (#2529)
new 228acf2 Rebalance strategy interface and DefaultRebalanceStrategy implementation (#2507)
new ada43b8 Fix the exception response from restlet (#2547)
new 2a4c18b Keep NOT_LEADER meter only in Segment completion related code (#2546)
new 443b186 [TE] frontend - Ttbach/mock data for alerts dashboard (#2548)
new 422ac4e [TE] Add deprecated message to some endpoints (#2550)
new d581e44 Adding response code for segments uploaded during minion task (#2551)
new 4f5f228 Rewrite SegmentDictionaryCreator to use the FixedByteValueReaderWriter (#2501)
new be57fd9 Fix the exception when getting baseline for empty dimension maps (#2554)
new 6632bf9 implemented toggle between legacy and new rootcause (#2555)
new 6b07973 [TE] Add CSV data source (#2533)
new 9a77b01 Fix the bug of data table deserialization failure for empty string (#2556)
new 52d1322 Fix PinotSegmentReatletResource APIs when raw table name is provided and table type is not specified (#2553)
new 77314d0 [TE] Create Alert - adding post payload to create-job (#2558)
new 572194b Add class SimpleHttpResponse to wrap status code and response (#2559)
new 65481bc [TE] frontend - Ttbach/created custom templates (#2557)
new ae2e197 [TE] Update the return format of uploadSegment method (#2562)
new 400abe7 Return no content with empty time line view upon exception (#2561)
new 2c2f015 [TE] Self-Serve Onboard/Tuning flows 03: Alert Edit/Create improvements (#2545)
new c77a3f9 [TE] Allow custom domain name in authentication (#2552)
new 2a8ea42 Fix null dimension drill down in anomaly function spec (#2563)
new bbb73a2 Handle incomplete commit scenarios in realtime segment metadata commit (#2560)
new 2c378d1 [TE] rootcause - humanize heatmap tooltip numbers (#2565)
new 85cee0b [TE] onboarding - use POST payload for create alert call (#2568)
new 53d6ef7 Ttbach/create stats-cards component for alerts dashboard (#2564)
new 14dc5d8 [TE] datasource - filter pre-aggregated dimensions by default (#2566)
new 337e768 fix test + syntax cleanup (#2569)
new 45b3771 TE Self-Serve Onboarding: fix for replay banner (#2571)
new e239255 [TE] fix anomaly-holiday email (#2570)
new 137728f5 mocked auth endpoing for ember serve (#2573)
new d4824dc final component (#2575)
new 62ef764 Added ReplicaGroupPartitionAssignment (#2549)
new de003e2 fix tests warning (#2577)
new 9bfd0c4 [TE] Delete Deprecated Endpoints (#2572)
new 3fd72ff [TE] Add bulk deletion by predicate to GenericPojoDao (#2578)
new daf835a [TE] Yyuen/entity mapping (#2576)
new baca9ae [TE] rootcause - advanced baselines (#2526)
new de95f65 [TE] frontend - Ttbach/redirect link (#2574)
new 0d78615 Removed an empty file that is causing license check failure (#2584)
new d8722bc [TE] Authenticate users with multiple LDAP domain names (#2582)
new 7ccd95e [TE] frontend - aaronucsd/fixed eslint for tests (#2588)
new afd94e3 [TE] datasource - ignore date column dimension by default (#2590)
new 9d2a04b [TE] Set up ThirdEye swagger UI and add API documentations (#2587)
new ec50e5d [TE] Add test for LDAP authentication (#2591)
new fcdc8c6 Corrected documentation on /tables/{tableName} controller API (#2586)
new 5c4937a [TE] rootcause - fix session query by id (#2593)
new e0e2ad8 [TE] frontend - Ttbach/add application dropdown (#2589)
new 08db126 [TE] Self-Serve Api import 1 (#2592)
new d4f55c1 Extend window if the view window is too small for train (#2595)
new 4815586 Change AvroUtils to not infer field type by field name (#2596)
new 41d5b4b [TE] frontend - ttbach/modified current custom template (#2597)
new 95cb2d5 TE Self-Serve Api import 2: create-alert (#2598)
new bd44155 [TE] Add swagger document for self-service APIs (#2599)
new 2af43ea [TE] Duplication removal of raw anomalies in detection runner (#2603)
new 91765a7 [TE] frontend - aaronucsd/added new duration custom template for the dashboard app table (#2600)
new 99aa8b2 [TE] Compress anomaly timelines view when the size is too large (#2601)
new 10a4324 [TE] rootcause - parse ambiguous urn filter strings (#2594)
new 4ec8ccb [TE] Enhance monitor deletion task (#2602)
new 73060ee fix test warnings (#2607)
new da16c67 Issue-2583: Step 1 to support generic streams (#2606)
new fe581d8 Refactor DataBlockCache and DataFetcher for multi-value support and performance (#2610)
new 49955be Realtime Segment Relocation Manager (#2604)
new 771c454 [TE] rootcause - cap change indicator at 10x (#2611)
new b632c48 [TE] Self-Serve Api import 3: Import Metrics (#2605)
new a63c986 [TE] rootcause - anomaly entity with function, metric, weight, and severity attributes
new 8451e42 [TE] frontend - aaronucsd/added new dimensions custom template for dashboard (#2617)
new 22a44df [TE] yyuen/ember upgrade 2.18 (#2615)
new 55cc1a8 fix link (#2619)
new 562473e TE Self Serve Bug fix: typo in helper name (#2621)
new f077f9a [TE] Add transaction update and batch deletion of DB manager (#2612)
new b529eed Reducing timeout to 10 minutes from 1 hour (#2624)
new c118bb6 [TE] Add batch request for getting root cause metric (#2620)
new 334d504 [TE] yyuen/3-14_code_cleanup (#2626)
new 54fc874 Use ONLINE for all minute baseline (#2618)
new 0a5bc80 [TE] Improve monitor's update procedure (#2625)
new b93b6f3 Cleanup: Enable splitting of in-clause strings by default. (#2614)
new 86af1a8 [TE] Bug fix in rootcause batch aggregates (#2628)
new 4142e2a Re-design the TransformExpressionTree, treat expressions with same synmatic as the same TransformExpressionTree (#2623)
new 7604e13 [TE] time range pill selector component and installation (#2613)
new ce2143c mocked new endpoints and fixed failing tests (#2629)
new 33988ca [TE] datasource - remove static event provider manager (#2631)
new 30fae06 Prepare for helix-0.8 upgrade (#2630)
new 7dab387 Make UDF case in-sensitive (#2633)
new 4d573dd [TE] Ingraph metrics loading time improve (#2632)
new 9b209c5 test fix (#2636)
new f356425 baseline line is now dotted (#2635)
new a893483 Fix the controller metric of table size (#2637)
new b0f405f [TE] frontend - aaronucsd/added new custom component for Resolution (#2634)
new 8c64a37 [TE] upgraded all tests syntax
new e9edd09 Larger Mininmum Training Period (#2642)
new 557e6ab Redo the rebalance of ideal state in retry block of DefaultSegmentRebalanceStrategy (#2608)
new 9a1370f Printing headers for segment upload requests (#2640)
new c185d65 Refactor code to avoid duplicate retry on downloading segment to local (#2387)
new 8e9bfe0 Adding https for minion base task (#2622)
new 02ea4a6 [TE] Deprecate raw anomaly manager (#2646)
new 7b318bb [TE] rootcause - hide all selection except metric for blank state (#2644)
new 6e6ab83 Add broker meter to track result's group by size (#2403)
new a0fa59f Return the updated ideal state for api after rebalance (#2643)
new f92f6a8 [TE] Add query for anomaly functions under an application (#2650)
new 462b295 [TE] rootcause - improve concurrency in breakdown endpoint (#2654)
new 4d9c5b7 [TE] frontend - aaronucsd/Added application loading/error start (#2647)
new 9ccfe29 [TE] Self-Serve onboarding acceptance test 1 (#2649)
new 42ea9c3 Use the correct method to emit broker metric (#2656)
new 019ff18 [TE] Add holiday events loader (#2645)
new cff235a [TE] Remove dependency to RawAnomalyDTO (#2657)
new d7e3fcf Fixed the issue with replica group mapping znode in the property store (#2660)
new f6280b0 [TE] dashboard - remove verbose time series logging (#2661)
new d7fef09 [TE] Add create customized event endpoint (#2662)
new f2d2baf [TE] root cause acceptance test fix (#2663)
new 9d1f0c9 [TE] rootcause - use maxtime cache for aggregation loader (#2664)
new 3bb8dd4 [TE] Implement the evaluation endpoint for alert config and application (#2658)
new 94211bb [TE] rootcause - remove deprecated data endpoints (#2641)
new 6c60671 Fix the cron for anomaly function creation (#2665)
new d277c7e [TE] dashboard - anomalies summary endpoint (#2543)
new c5cd3f6 [TE] Readable cron setup (#2666)
new 7768844 Add noDictionaryConfig into IndexingConfig. (#2667)
new 3c084ab Add an interface DataTransformer for general transform on values array (#2659)
new a4b0503 Compile expression when creating GroupByAstNode (#2672)
new adbc712 Add exception message into the failure response of schema upload/validation (#2671)
new a9918c6 Clean up multi part while uploading files (#2673)
new 333d038 [TE] Self-Serve onboarding acceptance test 2 (import) (#2670)
new 7f45055 [TE] datasource - request logger (#2669)
new d184679 [TE] Add tasks' error messages during execution (#2668)
new 06f23d4 Rename class names for some plan nodes, operators, blocks (#2677)
new 1bb018a Add broker metric of maximum response size (#2655)
new 6ee9c69 Add VALUE_IN UDF to support multi-valued column group-by without extra values (#2652)
new e162f2e [TE] dashboard - workaround for anomaly functions not setting metric id (#2678)
new a2a0153 [TE] rootcause - add json path dependency (#2679)
new 50c2782 Realtime Segment Assignment interface (#2676)
new 2ab41a5 [TE] rootcause - generalized baseline granularity (#2683)
new ed9b799 [TE] Self-Serve onboarding acceptance test 2B - transition to Alert Page (#2684)
new 7900f67 Initial code to support derived fields (#2675)
new 642a44e [TE] rootcause - anomaly baseline support (#2648)
new 82fe6ac [TE] rootcause - empty baseline support (#2691)
new 8bbf3a5 [TE] yyuen/entity mapping custom input (#2689)
new 57451e4 Add a class TransformResultMetadata to encapsule the metadata for the result of transform function (#2692)
new 1803ecb [TE] Add CM to RCA events (#2693)
new 2685dbf [TE] rootcause - fix event colors not loading (#2694)
new 09af8d0 Rebalancer for tables with replica group segment assignment (#2481)
new 4afdca2 Cleanup: Removed unused method in DataTableBuilder. (#2681)
new b46c95c [TE] datasource - log metric name rather than id (#2698)
new ee275ce Upgrade antlr to 4.6 (#2688)
new c5bde7c Disable metric aggregation when metrics have dictionary or dimensions (#2696)
new c6d8cbf [TE] rootcause - move predicted baseline to baseline selector (#2699)
new a432cf3 [TE] rootcause - fix chart bouncing between split and absolute mode on session save (#2700)
new 527ad31 [TE] rootcause - dynamically size rollup range to explain fraction of size (#2701)
new d991a90 Add EmptySelectionOperator for cases with limit <= 0 (#2697)
new 11f4e6a [TE] anomalies - gracefully handle pre-aggregated metric in baseline computation (#2704)
new 471d44a [TE] Add customized event modal in ThirdEye frontend (#2702)
new ba75314 [TE] rootcause - trend analysis tab (#2706)
new e61fcd2 Remove DataSourcePlanNode (#2709)
new 1f1c7dc [TE] Refactor HolidayEventsPipeline to accommodate customized events (#2708)
new e7bbcdc integrate ember test in maven test (#2690)
new 7561aa8 Update README.md (#2482)
new f7e94d1 Use Java 8 (#2472)
new ac35e71 CRITICAL: Fix the bug where ExponentialBackoffRetryPolicy is not stateless (#2713)
new 6f71b71 Use Trusty (default Linux dist) for Travis-CI (#2714)
new cdc7432 Resolve maven dependency conflicts (#2682)
new defc554 Remove failure modes from segment status checker (#2712)
new d1f9973 [TE] frontend - aaronucsd/added initial home dashboard page (#2705)
new a9134f7 [TE] frontend - aaronucsd/add service folders and moving it out for now. (#2720)
new 235c521 [TE] Self-Serve onboarding acceptance test 3 - tuning flow (#2716)
new 5ce3d0b [TE] Add swagger documents for ThirdEye public APIs (#2717)
new a61a2f5 [TE] hadoop - check for blank dimension string (#2724)
new d630ebd Re-implement RetentionManager to not cache any RetentionStrategy or SegmentZKMetadata (#2726)
new 723af10 [TE] rootcause - remove legacy toggle (#2725)
new 57a9f18 [TE] rootcause - refresh table columns on change only (#2727)
new 505f8ec [TE] rootcause - determine display offset based on metric granularity for anomaly (#2728)
new 4e55716 Add interface MutableSegment and ImmutableSegment (#2723)
new d88c2a4 Included more parameters in realtime server http requests to the controller. (#2722)
new 893db47 Minion observer framework (#2686)
new 51efd58 Introducing identifier quote character (#2710)
new 8738a2c [TE] cache - replace scheduled reload with on-demand reload (#2730)
new 0e4a1b3 Make all level data manager thread safe, allow loading multiple segments in parallel (#2731)
new d34e4ad Refactor realtime code to eliminate use of znode for storing partition assignment (#2721)
new 99d0385 BYTE_ARRAY Support: Add support to read/write byte[] data using raw (#2732)
new 17d9131 Adding multiple pinot segment record reader (#2707)
new 41d6e21 [TE] rootcause - workaround for function filters not stored in anomaly (#2735)
new e92c09c [TE] onboarding - do not override existing properties when onboarding (#2736)
new 1c1a6a6 [TE] cache - reduce refresh of deprecated caches (#2737)
new 95039bd Remove all array types from FieldSpec.DataType (#2733)
new a65c334 Add BYTES to FieldSpec.DataType (#2741)
new 46b31f0 [TE] dashboard - add metric urn (#2739)
new 0035c83 [TE] Fix zig-zag baseline in ThirdEye (#2740)
new b6b8627 [TE] rootcause - granularity and max time adjustments (#2743)
new 769b55a [TE] dataframe - lazy timestamps (#2744)
new e55fff3 [TE] rootcause - reload filters before adding metric to chart (#2745)
new c2f5dcf [TE] Add rule-based detection in addition to algorithm-based detection (#2747)
new 3bc558d [TE] frontend - aaronucsd/Add ember data to TE on Dashboard (Home) (#2742)
new 7d57001 [TE] Docs: initial setup with placeholders (#2748)
new 5d67699 Better error messages on stream partition fetch errors (#2753)
new 0a87d57 [TE] frontend - aaronucsd/fixed prod bugs on RCA (#2752)
new e7b67df [TE] frontend - aaronucsd/add 'Last 24 Hours' time option. (#2755)
new 7e72193 Make time column name available from table/user config in places where primary time column is needed (#2734)
new 5f36a65 Populating protocol parameters in segment completion (#2757)
new 2d22966 Fix setting of time column name in create segment command (#2761)
new e31b9b5 Add time threshold to flush segment as a realtime segment property so that it can be used for auto tuning (#2759)
new 55d8af4 Cleanup deprecated code from PinotLLCRealtimeSegmentManager and ValidationManager (#2762)
new b40464d Add config annotations for tables and schemas (#2746)
new 270576b Create an abstract class for FlushThresholdUpdate strategies (#2765)
new 4d09391 [TE] moved timerange persistence into services, set up acceptance test (#2763)
new 4785ea9 [TE] Fix bug that user cannot find subscription group for some application name (#2766)
new 85736e3 [TE] frontend - aaronucsd/Add new tab components (#2764)
new 7ecbdf5 [TE] Remove dimensions which also have top k dimensions in heatmap breakdown (#2767)
new e329d15 [TE] Deprecate anomaly page and link to RCA page directly (#2768)
new eb46dc4 Upgrading to helix-0.8.1 (#2770)
new 2ba4159 TDigest Support: Support for TDigest based percentile estimation. (#2754)
new 671b2cd [Minion] Add registry for task executor factory and event observer factory (#2771)
new 96c9cd9 [TE] Holiday refresh endpoint and improve error handling (#2773)
new 909ef7c [TE] onboarding - classic task execution (#2750)
new 1f2f882 Fix status code when got an exception for rebuildBrokerResourceFromHelixTags (#2751)
new b7b6d81 [TE] Fix error message of holiday events loader (#2776)
new 88f48c8 TE self-serve endpoint adjustments 02 (#2775)
new 9710895 Fix PinotHelixTaskResourceManager to allow concurrent jobs (#2778)
new 2e90c64 TE fixing default alert duration caching (#2779)
new a69858c Fix the protected config keys for HttpsSegmentFetcher (#2780)
new 9c74a36 Fix updating realtime table config (#2782)
new 66b8b28 [TE] RCA Advanced Dimension Analysis - tabs for content (#2769)
new 971257e Prevent Realtime table creation if schema is missing (#2777)
new fce6dd5 [TE] Sync comment and feedback between anomaly detail page and RCA page (#2784)
new 754bcbb Add a TaskGeneratorUtils class to provide util methods for task generator (#2785)
new 00d8bba Avoid exception for BYTES size() call when not generating Star Tree. (#2786)
new c540235 Minion segment conversion framework (#2781)
new 4642583 Avoid catching broad 'Exception' and prefer throwing a more specific runtime exception (#2789)
new b4efa3a Purge task optimization (#2787)
new 550112f Add table name to exception logs (#2791)
new c6d982e Add percentileTDigest5 aggregation function for 5th percentile. (#2793)
new 401750b Handle AVRO files with ByteBuffer instead of byte[] in input. (#2792)
new bb27a0f [TE] frontend - aaronucsd/updates and bug fixes for dashboard home (#2790)
new 29516e7 Bug Fix: Fix typo in names of percentile aggregation functions for 70/75 and 20/25. (#2795)
new 44c1deb Segment size based flush threshold update (#2783)
new 49ac850 Fix the time conversion from nanos to millis in TaskFactoryRegistry (#2804)
new efe3d0f Use current time millis in task name for easier debugging (#2803)
new 0f53dd7 [TE] dataframe - string series compression (#2799)
new 7570d00 [TE] frontend - aaronucsd/add Home to nav and update changes (#2802)
new 4f7a970 Fixed NPE bug for PartitionAwareOfflineRoutingTableBuilder (#2806)
new 404be05 [TE] datasource - eliminate potential tracking NPE (#2809)
new ae0a1ff [TE] onboarding - fix incompatible typecast (#2811)
new 7c80f36 [TE] frontend - aaronucsd/Add new investigation period slider (#2810)
new 23495ff Add query options into Broker Request (#2796)
new 0b95798 Add tests for date time conversion with timezone info (#2807)
new 0e2fa4e Remove deprecated class StreamPartitionAssignmentGenerator and all its usages (#2814)
new f9d55f7 Refactor TagConfig to remove helixManager field, as we will be using config only form the tableConfig to decide tags (#2816)
new f60d30f Rename ControllerTenantNameBuilder to TenantNameBuilder (#2813)
new b7a7a3f Rename exception class to remove Error in name (#2801)
new ff2aabd Updating numReplicas correctly for realtime partition aware routing (#2818)
new af947b1 [TE] RCA Advanced Dimension Analysis - Service and Component (#2815)
new 9328c09 Remove references to KAFKA_PARTITION znodes (#2817)
new 605c830 Remove 'queryOptions' from PinotClientRequest (#2821)
new 47c146d Add code to handle missing admin port (#2805)
new a134c2e Add TableQueryQuotaManager to enable rate limiter on broker (#2695)
new 2315a72 [TE] dashboard - disable javascript redirect to investigate view (#2820)
new d836b73 Add support for config profiles (#2808) (#2808)
new e93d3b2 Fix synchronization issue in hit counter (#2823)
new da30818 Change tenant to tag in tenant configuration (#2819)
new 50e2e37 [TE] Improve metric lookup specificity in Alert Page (#2828)
new 3e52885 [TE] frontend - aaronucsd/Fixed dashboard bugs and updates (#2824)
new ac707c4 [TE] frontend - aaronucsd/added the Yesterday time pill to dashboard (#2832)
new 6efb1ba Use smallest available kafka offset when a new partition is detected (#2831)
new 6a1666d license - unbreak local build (#2833)
new 07b0508 Add code to check if the untarred file will be put outside of the target output directory (#2835)
new d01f209 [TE] frontend - aaronucsd/updated the logout dropdown menu (#2834)
new 93468f1 Add unit test (#2837)
new c5b4ce2 Add the query option for preserving types for a broker response (#2830)
new f64283b Adding controller host log info to http methods (#2836)
new 9d7d400 [TE] frontend - aaronucsd/fixed dashboard on default of Last 24 Hours and app (#2843)
new f615f53 Fix RT table creation schema check: the backwards compatibility path was broken due to a sequencing issue. (#2841)
new d3b2ec4 Preserve time info for segment purger (#2846)
new 4d08805 Fix the issue where start/end time defined in SegmentGeneratorConfig is not picked up by IndexCreator (#2849)
new 536a1a5 [TE] Session key and service token for Thirdeye authentication (#2852)
new 5d968c2 Add log to track field name when exception thrown; Hot fix for calculating buffer length (#2850)
new 426ae3d Config to move completed realtime segments (#2851)
new 5ee44c9 Replace return with continue (#2827)
new 16ce744 [TE] frontend - aaronucsd/Added filter By resolution type (#2853)
new 440070e Allow multiple option clauses in queries (#2845)
new bfe75ea Revert " [TE] Session key and service token for Thirdeye authentication (#2852)" (#2854)
new 15c0955 TE Graph dimension fix for Alert Page (#2848)
new 1b0d1b8 Fix the bug where temp bitmap data file is not deleted (#2855)
new 70010ad Use PASS_THROUGH compressor for Metric columns, by default. (#2857)
new eb5a85c [TE] frontend - legacy filter fix (#2856)
new 2d5b932 [TE] frontend - fix edit alert subscription group update on app change (#2859)
new aa417ff Bring back session key and service token for Thirdeye authentication (originally #2852) (#2862)
new 5fd868e [TE] frontend - aaronucsd/added auth to dashboard (#2861)
new 21cb2d3 Remove unused code related to DROPPED state (#2858)
new 80c25a4 [TE] frontend - aaronucsd/fix sticky queary-param/leaking query state (sessionId) (#2864)
new 9e31d0b Added topic name in kafka consumer log messages (#2866)
new 4ebf2e4 [TE] Remove deleted Pinot datasets from the database when autoloading (#2865)
new 21c2d10 [TE] dashboard - add comment to endpoint output (#2868)
new ac0a968 [bug] : deleting temp files before loading the segment (#2867)
new 0d92cc7 [TE] frontend - aaronucsd/add comment, collapsible tables to dashboard (#2870)
new 542d2ab [TE] copy changes for self serve onboarding (#2869)
new 92429b1 [TE] frontend - fix comment truncation on rca page (#2871)
new b4e6e86 [TE] dataframe - fix rank statistics side effects (#2872)
new 2111a49 [TE] dataframe - add long and double comparators (gt, gte, lt, lte, between) (#2873)
new f1bb22b Addressed comments from design review for segment size based row threshold updater (and more). (#2875)
new 5b97faa Limit updates to segment stats history so as to get a better learning curve (#2876)
new fcefc0d [TE] frontend - condensed RCA data error message (#2877)
new 6bc7be9 Addressing review comments from previous commit (#2878)
new bfa3213 [TE] test - fix anomaly manager concurrent test bug (#2879)
new 41a7291 [TE] Fix for Alert Page graph dimension display (#2863)
new 06c5603 Fix toggle segment state description (#2880)
new ac3dcc2 PinotFS, LocalPinotFS, and HadoopPinotFS (#2842)
new ec4c080 [TE] onboarding - fix dataset filtering (#2882)
new f61ba4b [TE] dataset - remove default data source in DatasetConfigBean (#2883)
new 5ffc862 [TE] Advanced Dimension Analysis Table (2) RCA integration (#2881)
new 03852e1 Changing LocalPinotFSTest to fix directory creation (#2885)
new 5cbaec7 Add output granularity support to BaseDateTimeTransformer (#2738)
new bee5e3b [TE] Copy and graph changes self-serve 01 (#2884)
new a501186 Fix bug in realtime segment relocator (#2887)
new 462b795 Add a time buffer in retention manager when picking segments which are missing from ideal state for deletion (#2890)
new 0370753 Fix the percentile aggregation functions (#2892)
new 7278b14 [TE] frontend - aaronucsd/added new comment location to dashboard (#2895)
new e22e1ce Re-implement OffHeapBitmapInvertedIndexCreator to support large number of values (> 500M) (#2886)
new edd1383 Fix PinotInstanceRestletResource APIs (#2899)
new 25db717 Return detailed error responses for Table API errors (#2897)
new 59319dd Refactor PinotResourceManagerResponse to make it immutable and provide a uniform way to construct (#2898)
new 6fff3e8 [TE] frontend - aaronucsd/fixed the display window not syncing with date range (#2900)
new 0b8728a [TE] Advanced Dimension Analysis Table (3): static table improvements (#2893)
new 855f3b6 [TE] rootcause - query endpoint default values (#2901)
new 508081d [TE] rootcause - handle anomalies with multiple values for same dimension gracefully (#2902)
new 04c17e7 [TE] Rule-based detection pipeline (#2903)
new b71b00f [TE] Fix empty email screen shot (#2904)
new c1dfb76 Support arbitrary percentile for percentile aggregation functions (#2894)
new 68f5dee TE Fix for dimension analyis table RCA endpoint in prod (#2907)
new 2f52970 [TE] anomalies - unified anomaly status classification (#2908)
new cae1883 [TE] rootcause - adjust slider margins to line up with chart (#2910)
new ed0ab1e [TE] self-serve edit alert config group editing improvements (#2911)
new b491ed5 [PINOT-6535] Use random port for TableSizeReaderTests (#2912)
new d828e58 Fix the race condition when modifying the same segment file on pinot server (#2906)
new 802d784 [TE] detection - merger default to opportunistic splitting (#2915)
new 78c04b7 [TE] alerting - support email whitelist (#2913)
new d8cc04d [PINOT-6343] Fix table API error responses (#2914)
new 703775d [TE] frontend - aaronucsd/removed the 3 Hours ganularity option from RCA (#2916)
new 10fa9a4 Clean up TarGZCompressionUtils and its test (#2844)
new 2adb415 [TE] Request statistic logger run daily (#2917)
new 7c28538 [TE] datasource - fix daily offset aggregation (#2918)
new 95d010f [TE] RCA advanced dimensions table - enabling interactivity with page context (#2919)
new 4c59879 [TE] dataframe - jackson serializer (#2920)
new e1bfbbd [TE] detection - handle merging of anomalies fully subsumed by parent (#2923)
new 7a75dbc [TE] detection - enable window function change points from anomalies (#2924)
new 0243b97 [TE] detection - tutorial documentation (#2925)
new 8fdd8ec [TE] alerting - support metric filters for scheduled reports (#2928)
new fa6420e Fix the bug where adding default column and inverted index on same MV column throws exception (#2927)
new 2361187 Remove declared exceptions that cannot be thrown (#2860)
new d4013a4 Adding fat jar for pinot-hadoop (#2929)
new 001b076 [TE] alerting - fix metric id link formatting (#2930)
new 733c445 Handle a state created due to race condition between RetentionManager and segment commit, resulting in no consuming segments for the partition (#2891)
new 6940066 [PINOT-6614] Specify ConnectionManagerTimeout in milliseconds (#2934)
new 93de8c5 Add configuration for setting the desired segment size on a per-table basis. (#2931)
new d79d80b Re-implement TimerContext to avoid mixing of millis and nanos (#2926)
new 622b68d Count scheduling time into query timeout, and also add early termination if scheduling takes too long (#2909)
new 84d6166 [PINOT-6628] Add instance ID when reporting stopped consumption (#2940)
new e65d1db TE] frontend - aaronucsd/fixed login routing issue (#2942)
new 0be1ec6 Update copyright year range to 2014-2018. (#2941)
new 4dbfb4f [TE] rootcause - display gaps in time series data (#2945)
new 1a5d52f [TE] rootcause - vanilla dimension table (#2944)
new e8c6d0f Lower the probability of intermittent test failure for LargeClusterRoutingTableBuilderTest (#2943)
new a5baa6b [PINOT-6609] Ignore tables for segment status check if they are disabled in ideal state (#2933)
new b089e64 Update storage checker log to provide clearer stats (#2948)
new 02ac3fa [TE] rootcause - display selected entities in top (#2954)
new ebcac4c [PINOT-6614] Log more messages in the error path for table size estimations and handle estimate of -1 better (#2935)
new 6f11045 Fix NettyCloseChannelTest (#2952)
new f2fd22c Remove unused string (#2956)
new a9c0c31 Increase NUM_LOCKS available for ideal state update during segment commit (#2951)
new 529571f Raise a metric indicating the response from the controller for segment commit (#2950)
new e0dac48 Realtime hosts provisioning tool (#2938)
new 3cb59ff Added dummy idealstate-updater utility (#2955)
new a7c0ca1 Adjust info message to differentiate use case (#2958)
new 9baf2a4 Add logging in segment upload logic to make refresh scenarios traceable in logs (#2957)
new 76a34cc Retrieve hadoop parameters from controller (#2949)
new dd342f5 Reduce the number of threads used in test (#2959)
new ead0cb3 [TE] alerter - refactor internal base class for reuable utilities (#2961)
new 031cacd Add table type filter to retrieve only realtime or offline tables (#2960)
new 0272402 [TE] detection - add config utility and refactor (#2963)
new a3c85c0 [TE] rootcause - fix anomaly feedback sync (#2964)
new d87add7 Fixing maven shade plugin for pinot-hadoop jobs (#2939)
new 80ddaf4 Emitting storage quota utilization (#2967)
new 1bf0f91 Fix NPE when rebalance is called with a table that does not exist (#2969)
new 000ecc1 [PINOT-6677] Clean up any existing segment dir before replacing a segment (#2970)
new 7115324 Added a metric for number of segments for a table (#2973)
new f7d3bb5 Log uncompressed/compressed segment sizes during segment creation time (#2972)
new 82345ea Re-implement PinotDataBuffer to make it thread-safe and support large buffer and native order (#2971)
new 1d61160 Adding table name and segment name headers to segment push (#2974)
new 8e04a12 Fixing setAdditionalProperties bug (#2975)
new 980d768 Fix DISTINCTCOUNTHLL to offer raw value instead of the hashed value (#2976)
new 0c9da75 Revert "Adding table name and segment name headers to segment push" (#2979)
new 140017d [TE] dashboard - expose detection config name and id (#2978)
new 5068ef9 Adding the integration test for Hadoop jobs (#2965)
new 34e5607 Improving segment generation code in hadoop (#2966)
new 059a6e0 Remove leadership check when setting up metadata for new table (#2977)
new 7116662 [TE] Dataset auto onboard (#2984)
new 45c4c62 Fix a bug in OffHeapStarTreeBuilder where file is not properly copied from data file to temp file (#2981)
new f0c1154 Ignore NettyCloseChannelTest (#2980)
new b761292 Replace LBufferAPI in star-tree with PinotDataBuffer (#2983)
new 6be9309 Replace Number.SIZE / Byte.SIZE with Number.BYTES (#2985)
new 0002137 In AggregationFunction, replace method getName() with getType() (#2988)
new 8d39121 Add interfaces for star-tree V2 (#2989)
new 688ba11 [TE] alerts - manage subscription groups modal (#2982)
new cd98557 Change SegmentMetadataImpl.getPropertiesConfiguration() so that we don't need to cache the properties configuration (#2992)
new e54a215 Freeing up the reference to PropertiesConfiguration after loading seg… (#2996)
new 532651f Adding the link for sigmod 2018 presentation (#2993)
new d5a4ad8 Print message in TableSizeReader only when num of errors > 0 (#2990)
new aa140ff [TE] hadoop - upgrade version (#3001)
new e1eb6aa [TE] frontend - unbreak unit tests (#3002)
new ba4e0ef [TE] dashboard - fix anomaly time range queries (#3000)
new b0c20b7 [TE] Set the size of heatmap using other metric (#2999)
new c7953d2 Make DistinctCountHLL, PercentileEst and PercentileTDigest work on both value and serialized bytes (#2997)
new 72e3b7e [TE] Advanced Dimension Analysis: table custom settings (#2932)
new 28b5898 Add stream abstraction for exceptions in realtime consumption code (#3003)
new 3606ace Move kafka connection provider code to a separate class (#3005)
new 4459731 [TE] Fix comment text overflow in dashboard (#3006)
new 76ff835 [TE] groups - fix new group app change (#3007)
new d960c0c Support UDF in aggregation function (#3008)
new 7c141d2 [TE] frontend - fix flakey entity modal test (#3013)
new 5faa4d4 [TE] rootcause - display predicted baseline again (#3015)
new cb62c0b Update star-tree v2 interface (#3017)
new c686fbb [TE] rootcause - shorten anomaly feedback debounce interval (#3016)
new 308a483 [TE] groups - add support for email subject settings (#3019)
new c7da045 [PINOT-6736] Fix min size for hash-set in In/NotIn predicate impl (#3009)
new 32682bb [TE] groups - regression fix for email subject settings (#3021)
new 75446a7 [TE] rootcause - pin PST/PDT time zone (#3020)
new bd06a8e In SingleFileIndexDirectory, fix a bug where we load file with wrong read mode (#3022)
new a7fa16d [TE] frontend - aaronucsd/Added new shared summary email feature (#3018)
new f779b8e Don't allow default column update in V3 format (#3024)
new bcd6f2f Rollup Segment Converter (#3012)
new 5edcec5 Add COUNT_STAR constant into AggregationFunctionColumnPair (#3025)
new 8dd4b1a [TE] dashboard - fix sharing link (#3027)
new 9b325f6 [TE] email - metric header support (#3026)
new 661310f Add some messages to trace zk behaviors (#2946)
new 7fe310f Fix segment reload path error handling (#3030)
new c6df8b5 [TE] email - fix metric link color (#3031)
new 989ffb7 New Star-Tree Execution Part (#3029)
new 05c4050 Adding a base class for multiple segments conversion minion tasks (#3028)
new 6f18049 Segment Metadata Upload (#2953)
new a12cc47 Fix the segment naming issue for segment converter (#3035)
new 17334e8 Bump zookeeper version to 3.4.11 (#3033)
new e3363a8 Adding Azure FS implementation of PinotFS (#2991)
new 84bcbfb Verify hybrid table contains the same table configs in its both subtype tables (#2986)
new c2af44f In GroupBy, always use expressions (remove all usage of getColumns) (#3032)
new 36eaf84 [TE] dashboard - anomaly classification fix (#3038)
new 621f8bd [TE] email - pin link color to white (#3036)
new 8e392e2 Add base test class for StarTreeV2 tests (#3040)
new ed07ddf Add inverted index option for offline data gen (#3039)
new 9d56c0b Revert #2946 extra logs since we already finish debugging ZK (#3043)
new 0915545 Support COUNT in BaseStarTreeV2Test (#3044)
new 071a457 Fix controller starter setting for open source tools (#3046)
new 84029c1 Improve log message on the controller during segment commit (#3045)
new 6a4edf3 Changed server api end points to return string instead of java objects (#3041)
new 205dc66 Reducing garbage when indexing multi-value columns (#3048)
new aa3a4c9 Bump json version from 20080701 to 20170516 (#3042)
new d39dec7 Only log SEVERE message in travis (#3047)
new 584c478 Fix controller query console (#3050)
new d4fb2e0 Remove using http connection pool in table-size api path (#3051)
new aef5f88 Returning default fs on instantiation (#3052)
new 4489b3f Allow 15% quantile error in PercentileTDigestQueriesTest to make tests more stable (#3053)
new 47464cb Expose default sign test window size to fix cases when mttd is 0 (#3056)
new 4d7e004 Cleaning up the headers (#3054)
new 8e2ca1b Testing pinotFS instantiation (#3055)
new 65c9779 Changing pinotFS to use temp file instead of uri for local methods (#3059)
new 5c8ed66 [PINOT-6845] Provide a controller override for storage quota checks (#3060)
new 7b3a746 [TE] Add support for data filter in detection pipeline emulation layer (#3058)
new e470b94 [TE] Endpoint for migrating the anomaly functions (#3062)
new e673a9f Reduce heap allocation during realtime segment creation (#3057)
new 2d63645 Retry hadoop copy to local method in HadoopPinotFS (#3063)
new 522de43 Fix error logs in ServerTableSizeReader (#3065)
new 57d4bf0 Revert "Remove using http connection pool in table-size api path (#3051)" (#3064)
new 3147d2a [TE] Calculate precision/recall stats from anomaly feedback classification (#3066)
new be4c211 [TE] Anomaly function translator fix (#3069)
new b96e2fb [TE] detection - diagnostics support and moving window refactor (#3023)
new ee3b37e [PINOT-6856]: Fix controller storage check logic when it is unable to get reports from servers (#3067)
new 711cc23 [TE] detection - support active flag (#3070)
new 3f15e28 [TE] Detection emulation layer and preview UI integration (#3072)
new 0d7a162 [TE] rootcause - anomaly header adjustment heuristic (#3071)
new 6d7ad06 Reduce heap usage during segment builds (#3073)
new e4ffb86 Bump helix version to 0.8.2 (#3076)
new 25cf812 [TE] rootcause - remove wo3w and wo4w by default (#3074)
new 038e7fe Fix TDigest for cases where only one value in TDigest (#3079)
new 9938cd7 [TE] dependencies - fix zkclient wait() call (#3080)
new d0fd095 [TE] Legacy alert filter support in new pipeline (#3082)
new 99579ac TE] frontend - aaronucsd/fixing the mvvd issues on share dashboard (#3081)
new 6fadce7 [TE] detection - moving window cold start heuristic (#3075)
new 30ba267 [TE] Support legacy alert filter in anomaly function translator (#3083)
new ec41bf9 [TE] alerter - support cc and bcc (#3085)
new 5fa5c6e Adjust code on acquiring rate limit in pinot-broker. (#3049)
new f6a5fa4 [TE] frontend - aaronucsd/replaced with textarea for better markdown support (#3084)
new 76ee958 [TE] detection - fix anomaly window expansion logic (#3087)
new 508dc3e Adding Azure Support for SegmentFetcher (#2995)
new 128b86a TE homepage app search fix (#3092)
new 6d471b1 Fix thread-unsafe issue in HelixExternalViewBasedTimeBoundaryService (#3088)
new 455fbff [TE] Business rule filter wrapper (#3093)
new 199cf88 Adding azure fs improvements (#3094)
new 6f5d9af [TE] auth - fix NPE on redundant logout (#3090)
new b67e5e8 Add tableConfig as input to hosts provisioning tool (#3095)
new 8c5b47c [TE] detection - exp smoothing for window stats (#3097)
new a3859a4 [TE] monitoring - add alert and detection task counters (#3098)
new 2302c7f [TE] ThirdEye admin access control (#3100)
new f3abf28 [TE] dashboard - anomaly source and metric urn (#3101)
new 4a5e8d8 [TE] rootcause - UI fix for rule detection (#3099)
new 0a33cc9 [TE] detection - rename moving window vars (#3103)
new 01da366 Fixing PinotFSFactory Init Bug (#3096)
new 26b0bda TE dimension analysis and report anomaly bug fixes (#3104)
new 655a9c5 [Broker Clean up] Decouple broker from transport package (#3105)
new 72ef41e [TE] report-missing-anomaly from dashboard base functionality (#3109)
new 3f01503 [PINOT-6738] Deprioritize multi-value columns during scan-based filtering (#3111)
new 10f8242 Adding mkdir to PinotFS (#3112)
new a2a9b1f Changing FileUploadPathProvider to use PinotFS (#3113)
new ee83fb0 Adding PinotCrypter interface (#3115)
new 4bdefd2 [TE] groups - more input sanitization for email recipients (#3116)
new 5d3b2a4 Move QueryExecutor and ServerQueryRequest related classes from pinot-common to pinot-core (#3121)
new 8dc3a5d [TE] frontend - aaronucsd/added 401 error handling and login session expired catches (#3117)
new 3e6e649 [TE] detection - legacy emulation properties injection chain and preview fixes (#3122)
new 2a49a65 Merge DataManager interface into InstanceDataManager interface (#3123)
new 1f4fb97 Replace global lock with ConcurentHashMap.compute in CombineGroupByOperator (#3125)
new 0696d87 Refactor ServerQueryRequest to pre-compute and store segment independent information about the request (#3124)
new 0532a20 [TE] groups - endpoint for search and retrieval (#3131)
new f7a4040 Fix the annotation where debug options should be nullable (#3130)
new e813afb [TE] frontend - aaronucsd/fixed edge case on session expired and deeplinking from email. (#3133)
new 66ace21 [TE] RCA Dimension algorithm table: manual order (#3127)
new 5f9b74e [Clean up] Add default implementation of new API getNum(Non)MatchingDictIds in base class (#3129)
new 7fe045e Clean up filter blocks, expose predicate evaluator and data source metadata for scan only (#3128)
new 028ea54 [TE] Bug Fix - Emails not sent out to all the recipients (#3136)
new c6b5807 Enhance service status check (#3137)
new edb214f [TE] dimension analysis settings modal tooltips (#3134)
new ee51a1b [TE] detection - add support for hourly, daily dimension filters (#3139)
new de08c3c Minor log fix (#3138)
new 88f09c0 Support null values in BYTES data type. (#3140)
new 3242457 [TE] frontend - aaronucsd/addressed 500 and other error statuses to display a toast msg (#3141)
new 94035c8 Moved from streamlib's implementation of TDigest to Ted Dunning's, as (#3144)
new c7b5b0b [TE] detection - threshold rule filter (#3142)
new f31dc85 [TE] detection - uniform legacy property injection (#3145)
new fd13fae [TE] detection - support global metric impact filter (#3149)
new 4705c27 [TE] detection - update migration endpoint for unified injection (#3147)
new 5bde32c [TE] frontend - aaronucsd/compressed the share summary view (#3148)
new 0ef084d Invalidate controller leader cached value if exceptions occur during segment completion calls (#3135)
new e36da03 [TE] detection - legacy integration test (#3146)
new 5eabb84 [TE] refactor - move time series and aggregation loaders to datasource package (#3150)
new 15ae028 Async Broker Routing (#3143)
new b582a49 Enhance broker log for queries (#3152)
new 9ea1169 [TE] frontend - fix OOM blowup on exploring 2k+ dimensions (#3154)
new 3ddec34 When deleting task queue, enable force delete (#3155)
new 5a3ad97 Fix the minion config for segment fetcher (#3156)
new 1722a66 [TE] detection - run preview from start of hour (for caching) (#3158)
new 2b2da08 [TE] detection - migrate with user-specified or default last timestamp (#3157)
new 1f301e9 [TE] detection - warn on anomaly store failure (#3161)
new 16ffb20 [TE] YAML configuration translator (#3151)
new fa0b44e [TE] detection - legacy emulation compensate lossy merger (#3162)
new 596502f [TE] project - add license file and headers for .java files (#3164)
new aa2316c [TE] detection - legacy emulation fix for implicit function id override (#3165)
new 8bfca8d Add support for virtual columns (#2000)
new 36908da Fix Swagger urls to use https if enabled (#3166)
new 8c85e7b [TE] datasource - change data source properties type to Map<String, Object> (#3167)
new 2a2315a [TE] Changes to migrate data from comma separated email string to a Set (#3171)
new 61d92e8 [TE] detection - legacy merger global impact fix (#3173)
new 625014e Fix allowable values in swagger (#3091)
new c269cc7 Remove unused classes in realtime (#3172)
new 0950ba4 Put zk client timeout param as constant (#3078)
new 8b314f0 [TE] Yaml endpoint fix (#3174)
new c122396 [TE] rootcause - support agg multiplier workaround for legacy emulation (#3175)
new 7b6101a [TE] email - more address test cases (#3177)
new 6951780 [TE] Add support for TDigest metrics in rca (#3168)
new ff5874f Create StreamConsumer and StreamMetadataProvider interfaces for splitting the consumers created by StreamConsumerFactory (#3169)
new dff2fd4 [TE] report missing anomaly selector in dashboard (#3170)
new aceeade [TE] Add support for cc and bcc recipients in the legacy pipeline (#3176)
new 9795d57 [TE] frontend - aaronucsd/New POC RCA route (#3178)
new d1e741b [PINOT-7011] Log errors caught during query processing (#3180)
new b4a467b Refactoring Pinot segment upload (#3153)
new bab7ef4 [PINOT-7047] Emit metrics for docs scanned per query on server (#3181)
new 614dc9c Optimize TDigest tests (#3184)
new c3e7eb0 TE THIRDEYE-1997: disable dimensions in graph (#3179)
new 4d69933 [TE] Fix thirdeye-frontend build failure on local machine (#3182)
new dcf2752 [TE] detection - more fixes for legacy merge wrapper (#3183)
new a0a1faa In PinotDataBuffer, allow view() and toDirectByteBuffer() with different byte order (#3189)
new c0a63f1 Bug fix: Fix incorrect cast in controller config. (#3190)
new bf9108f Adding tenant to table mapping API (#3186)
new f225a84 Adding content type to uri upload (#3188)
new 0a58013 [TE] callgraph - rca pipeline (#3086)
new e9dfe64 Add table APIs for the new config format (#2937)
new b1fa528 Remove the lower bounded wildcards for value readers since they are not valid (#3193)
new ee9d901 Handle the case where num_scanned metrics might be missing when encountering exceptions (#3195)
new 84964ff Setting file permissions of staging dir to be user-chosen permission during segment creation (#3197)
new 44aac1a1 Convert byte[] defaultNull values into String equivalents during (#3196)
new 90dffb3 [TE] Introduce metric tierLevel and some cleanup (#3198)
new 4940e5d [AnomalyTimelinesView] Fix exceptions when no or shorter baseline or current values in the view (#3194)
new 84ef87d CSV Reader Config when Creating Segments from CSV using Hadoop (#3192)
new c3accf4 [TE] Check for at least one valid email in to-recipients (#3199)
new 5111e65 [TE] dataframe - retainSeries() (#3201)
new e8add66 [TE] pinot - datasource auto-onboard enhancements (#3203)
new 2867d02 [TE] Stage interfaces for detection pipeline (#3191)
new b184998 Azure FS Minor Cleanup (#3206)
new ab0821e [TE] pinot - support topk limit for groupby queries (#3207)
new e5fe003 [TE] pom - add project meta data and license (#3204)
new 7a238f6 Add new configuration format command line tool (#3200)
new e092633 [TE] datasource - csv topk limit support (#3213)
new b745abb [TE] rootcause - faster heatmap via end-to-end topk limit support (#3211)
new a2b655a Changing AzurePinotFS list files to be recursive (#3209)
new 6659de9 [TE] (PINOT-7089) Ability to configure multiple Metadata Sources (#3212)
new 72694e5 [TE] rootcause - heat map ranking fix (#3218)
new 0edd29b Changing permissions parameter to subtract the umask (#3202)
new 980544c Adding local temp dir parameter to distinguish between local and remote fs (#3214)
new aad958a Deleting unused code in segment upload (#3221)
new 05af92f [PINOT-7088] Emit metrics to track when servers are not found segments during query processing (#3220)
new 3f6d161 [PINOT-7088 Formatting changes (#3222)
new 80c7a28 [TE] datasource - mock data generator (#3219)
new 2918060 Fix typo (#3224)
new 7b4ec1b [TE] Adapt threshold & baseline algorithms to stage interface (#3215)
new 12c5f9a [TE] Add owners field to dataset (#3227)
new b80c3bd Log exception on controller for realtime segment commit operations (#3226)
new 9d36680 Improve the way we get instances with a tag from helix (#3223)
new 7a574bb Fix the bug for "select * ... order by" queries (#3229)
new 3817a13 Create abstraction for PartitionOffsetFetcher and PartitionCountFetcher (#3187)
new 831a3d0 Set permissions on staging and output directories for (#3230)
new d58a0ce [TE] Fix for Pinot autoloader (#3232)
new b08c265 [TE] Enable Metric Tagging (#3233)
new e5ec0f9 Simplify the parameter for forward index creators (#3208)
new 9ae1786 In PinotSegmentColumnReader, change methods to public so other class can use it (#3238)
new 50c7ee2 [Star-Tree V2] Extract common star-tree builder methods (#3237)
new d4117b1 [TE] Cleanup and fix exception handler (#3239)
new e016a78 [TE] Create alert bug fix - alert recipients (#3231)
new efd34c5 Add crypter is SegmentZKMetadata, so server can decrypt segments at (#3234)
new 2bc9a2f [Baseline] Fix exceptions in baseline view (#3241)
new 3aba8bd Bug fix: Initialize crypter before using it. (#3242)
new 7049f1a [TE] frontend - aaronucsd/fixed the missing data points on valid analysis range (#3243)
new 877c66b [TE] Fix holiday events time zone information (#3246)
new 85673bb [PINOT-7088] No-downtime table rebalance (#3217)
new 7d56028 [TE] dimensions algorithm table fixes (#3240)
new 2fe2181 TE RCA Trend table fix (#3249)
new 8cf73e2 Passing through a parameter to set permissions for directories in segment creation job (#3250)
new cf868f8 [Clean up] Simplify the logic for RoutingTableInstanceRruner (#3248)
new 0a6df90 [Clean up] Refactor ObjectCustomSerDe into multiple singleton inner classes (#3244)
new 2def3a0 Cleanup PinotFS (#3252)
new e8512bc [TE] frontend - aaronucsd/add new customize email teamplate features (#3251)
new e308e88 [TE] demo - dashboard end-to-end integration with mock data (#3225)
new 6446a9d Fix NPE when getting rebalance strategy (#3102)
new 63a6767 [TE] readme typos and fixes (#3257)
new 398487e Optimize PERCENTILE aggregation function (#3256)
new 6305780 [Star-Tree V2] Enhance AVG, MIN_MAX_RANGE aggregatin function to work on serialized bytes (#3254)
new b323d44 [Star-Tree V2] Value Aggregator (#3236)
new aabcc55 [TE] demo - pinot readme reference to thirdeye (#3258)
new 79eafb3 [TE] Make PqlUtils method convertAggFunction public for use globally (#3228)
new ce29d50 [TE] detection - resurrect filter wrappers (#3265)
new 4e773c5 [TE] Endpoints for bulk on-boarding, subscription and tagging of metrics (#3263)
new d14339e [TE] demo - build core pinot components (#3267)
new 13cfc6d [TE] detection - baseline provider (#3262)
new 4baf2f8 [Star-Tree V2] Index loader for Star-Tree V2 (#3259)
new d6af440 Creating stream and partition level consumer (#3245)
new 5488058 Fix the bug in SegmentGeneratorConfig where columnSortOrder is not copied (#3272)
new bf74587 [TE] frontend - Alert Index bug fixes (#3261)
new 1f62f83 [Star-Tree V2] Index builder for Star-Tree V2 (#3269)
new cefdf4d [TE] tracking - support exception counters (#3266)
new c2b4de5 [TE] frontend - aaronucsd/Added first draft of recommended metric bookmarking (#3273)
new 99bb15f Always build shaded jar for pinot-common (#3275)
new ffc61d6 TE rca metric select editable text (#3277)
new c683ca7 [TE] Update function name to include dataset name for uniqueness (#3282)
new 773340d [TE] rootcause - unbreak rca coldstart page (#3281)
new c9420a7 [TE] frontend - setting limits on custom MTTD in alert tuning (#3279)
new d018597 [TE] rootcause - pin pacific time zone in RCA UI (#3276)
new dbcd462 Rename StreamMetadata to StreamConfig (#3284)
new b6dfde8 Get no-dictionary columns during metrics aggregation for consuming (#3283)
new d04c2f3 [Star-Tree V2] Integration test for star-tree V2 (#3271)
new 5f3ba04 [TE] datasource - csv source support for partially missing dimensions (#3285)
new 39492ac [Star-Tree V2] Added unit tests for all supported aggregations (#3270)
new 2da587d [Async Broker] Add netty connection related metrics (#3268)
new fb3c74d [TE] rootcause - fix unintendent metric selection on tab change (#3286)
new 541aeb8 Fix the missing javadoc (#3287)
new 262330e [TE] detection - merger filling current & baseline value and child keeping merger (#3278)
new 7d50d0a [TE] rootcause - anomaly end time quick fix (#3291)
new 7353160 [TE] rootcause - avoid setup mode sticking around after first load (#3293)
new 400fbe9 [TE] Endpoint for removing a metric tag (#3288)
new 6eb0af9 Have server wait until no incoming queries nor online table resources during shutdown (#3255)
new 0d3be35 Update daily detection default properties when self-serve (#3296)
new 1f1396c [ThirdEye][s] Fix StringUtils on empty property value (#3298)
new 1a7821c Removing extra copy in default crypter during segment upload (#3294)
new be9a2f8 [TE] Deletion endpoint for anomaly functions (#3297)
new 7fd3e8e [TE] rootcause - prevent debounce error on no anomaly or session (#3292)
new a09b739 [TE] Cleanup the legacy check code (#3303)
new a47a6cb [TE] detection - detection stage wrapper metric urn injection (#3306)
new 7358f95 Cleanup segment fetcher instantiation (#3295)
new 7e41a81 Adding logs for segment movement (#3310)
new ff0d35b Adding close to PinotFS to support different fs reqs (#3300)
new 60760ff Fix ValidationManager, reduce logs and ZK accesses (#3302)
new 6d51261 Clean up PinotSegmentUploadRestletResource because methods are moved to SegmentValidator (#3312)
new c946939 [TE] Auto create alert groups with dataset owners as recipients during bulk onboard
new 400a8eb Changing segment fetcher init to use correct constant in controller (#3304)
new 8641978 Add a config to enable/disable segment level validation (#3313)
new 4dac15c [TE] detection - inject metric urn in baseline filling merger (#3309)
new 8080ff1 [TE] frontend - aaronucsd/Format Metrics tab and Metrics to Investigate (#3317)
new 9e9699c [TE] dataframe - prevent metric metadata load amplification in helpers (#3322)
new 3a6eebc [TE] BugFix - Update alert grup with function id (#3321)
new c07bbbf [TE] frontend - Alert page bug fixes (#3320)
new a71b23e TE graph breaks on filter null response (#3307)
new d15ffed [TE] frontend - rootcause chart for daily metrics (#3289)
new 80b66c6 Create StreamDecoderProvider (#3319)
new 12e3bae Refactor checks when shutting down pinot server (#3305)
new cdcd593 [TE] rootcause - exclusion filter support (#3301)
new e321bb8 [TE] dashboard - fetch anomalies by group (#3325)
new b6fa071 Add flags to ensure server loading all segments then start serving query traffic (#3253)
new a09ded1 [TE] frontend - isolate anomalyResponseObject modification to home controller (#3324)
new e9291c7 TE fixing width of custom date range picker input (#3329)
new 133f10e [Clean up] Enhance TableSizeReader to reduce the logs (#3328)
new a186135 TE ommitting failed alerts from list (#3327)
new f0052c1 [TE] YAML pipeline configuration supports multiple rule AND/OR (#3330)
new c878c83 [TE] rootcause - align off-granularity anomaly times to whole minutes (#3331)
new c31392e Move import statements for helix listeners (#3333)
new ce0bb47 [TE] datasource - generate mock data with diurnal patterns (#3334)
new 95143fa Add method nonLeaderCleanUp() in PinotTaskManager and PinotTaskGenerator for necessary clean up after leadership change (#3332)
new de24a02 Removing pinot dash due to security vulnerability (#3338)
new 3ad2fd4 Adding backwards compatibility for segment upload (#3326)
new 8be1836 [TE] rootcause - fix urn generated for anomaly with filters (#3341)
new 1372fdd Adding isDirectory method to pinotFS (#3339)
new 5d59d5c [TE] Customize offset in baseline inclusion/exclusion rule (#3342)
new c3f84c4 Remove ununsed imports from ConnectionFactory (#3323)
new 8d99814 [TE] Store yaml configuration in database (#3335)
new 8d457ac TE bug fix for edit alerts modal (#3346)
new a4e40aa [TE] rootcause - delete rca v1 routes, components, and dependencies (#3345)
new 315ec61 [TE] Find dataset by name rather than metric.getDatasetConfig()
new 8c0a41c In TableConfig, put initial value for TenantConfig and TableCustomConfig to follow the nonnull annotation (#3349)
new f56bfbd [TE] demo - controller support (#3348)
new c66e3f5 Refactor stream config properties related constants (#3352)
new 89a2da3 [TE] Move Thirdeye to Java 8 (#3357)
new 317cd15 [TE] detection - merger handles anomaly properties (#3358)
new b4efa42 [TE] detection - add label for expected anomalies (#3353)
new 7ae43a1 Refactor FileUploadPathProvider to be more robust (#3318)
new ef023d1 Dynamic rate limit for pinot brokers (#2922)
new 51fa4b9 Auto-reformat classes for PR #3356 (#3365)
new c04c9f1 Add the unit test for KafkaHighLevelConsumerRoutingTableBuilder (#3361)
new 239a136 [TE] Iris Alerts - Phase 1 : Introduce configurable alert schemes (#3337)
new 263b305 Fixing server starter endtime when enable segment check (#3367)
new e3c12ba Limit the number of groups stored both inner segment and inter segment (#3362)
new 069d7c6 [TE] Child Anomaly Feedback (#3360)
new b498aed [TE] yaml - fix multiple filters (#3363)
new 78b6210 Stream config refactor (#3364)
new ba03591 [TE] new anomaly resolution labels (#3368)
new 0d92355 Get quickstart-realtime working again (#3374)
new 8f2dba8 Add abstract class PeriodicTask to unify duplicate code (#3264)
new 1eccd34 Backward compatibility for StreamConfig (#3380)
new a3c499c Disable LLC validation when LLC is not enabled (#3381)
new bd80360 Reduce the ZK access for segment metadata fetching (#3356)
new 57014c8 [TE] dashboard - support expected anomaly classification (#3382)
new ee5fa0d [TE] detection - pipeline/preview UI bug fix (#3383)
new 526ba5d Adding way to construct old segment upload endpoint (#3343)
new 86f65e0 Add a flag numGroupsLimitReached to mark responses that hit the number of groups limit (#3378)
new 7d347e9 Cleaning up crypter initialization (#3369)
new 83d61c9 Fix int overflow in FixedByteValueReaderWriter (#3385)
new cd6e32e [TE] rootcause - request scheduler (#3366)
new fcab1ba Supporting delete api for different storage options (#3370)
new b7ea9ee Support .tar.gz upload for pluggable storage (#3377)
new 9c6a0c6 [TE] Iris Alerts - Phase 2 : Refactoring | Cleanup DetectionEmailAlerter (#3375)
new 5071a74 TE rca anomaly label text shorten (#3384)
new f26b818 Update the TimeboundaryInfo in Pinot Broker when offline segments are refreshed. (#3354)
new 0a4712f Reduce FixedByteValueReaderWriter.writeUnpaddedString() to one write to data buffer (#3392)
new 9138751 i`[TE] frontend - aaronucsd/Added new share custom template header (#3391)
new 673af86 [TE] metric onboarding - support custom fabric groups (#3394)
new b763b65 Remove the columnName argument in FunctionExpressionEvaluator constructor (#3397)
new 0c31ba5 Instance level semaphore for realtime parallel segment builds (#3393)
new 390b4f9 [TE] build - cut down CI build time (#3401)
new 309953f [TE] Introduce configurations for alert schemes (#3395)
new 0045436 [TE] Change default alert cron for bulk onboard (#3390)
new b59b03b [TE] detection - moving monitoring window aligned to data set granularity (#3396)
new 07aa3ea Editing Reading of CSVReaderConfig in Segment Creation Job (#3372)
new 4de0b3f [TE] Read alerter configurations as a Map for easy extensibility (#3389)
new 4f34f83 Introduce OffsetCriteria class to handle different offset types and operations (#3402)
new a9550e3 [TE] Move emailWhitelist under alerterConfiguration (#3403)
new cfb97d5 [TE] rootcause - support query params for external setup of investigation (#3388)
new dab4494 [TE] Group together to, cc and bcc under recipients
new 75aa028 Adding overwrite parameter to move in PinotFS (#3386)
new 870c5d7 fix schedulerName (#3421)
new 2d404ac Use maven enforcer plugin to clean up dependencies (#2998)
new e111b8c [TE] frontend - aaronucsd/update Travis to use Chrome (#3400)
new 27ae042 Remove references to kafka from generic classes (#3423)
new f70c9c3 Add stream agnostic metrics for realtime (#3426)
new 4ce685c [TE] frontend - Alert Filter Component (#3419)
new 75e718d Move group id to StreamConfig level (#3425)
new 0a690ad [TE] frontend - aarouncsd/remove rca2 poc route for now (#3428)
new 4f2696d [TE] database - update alias field size (#3429)
new 2023bd1 [TE] frontend - random linter fixes Vol 1 (#3422)
new 18640fe Add segment merge command (#3427)
new f73a365 [TE] yaml - detection annotation (#3399)
new eccdd7d [TE] frontend - aarouncsd/update RCA title (#3430)
new 668aaf4 [TE] alerts - fix null array exception (#3431)
new 5ee6df6 [TE] Fix for zero anomalies email bug and added some additional checks (#3432)
new 5c296de Add RecordTransformer to perform transformation to incoming record (GenericRow) (#3424)
new 0c3f416 PINOT-7296: Reduce lock contention by not sharing DecimalFormat instances (#3433)
new 2e8e20e [TE] user dashboard - hide anomalies if alert is disabled (#3435)
new 0acea8f [TE] Introduce ability to suppress alerts (#3437)
new bbb300c Adding RetentionManager Support for Pluggable Storage (#3371)
new 65ad8a4 Replace ENABLE_SEGMENT_LEVEL_VALIDATION config with SEGMENT_LEVEL_VALIDATION_INTERVAL_IN_SECONDS (#3441)
new e692b89 Add the support for dynamic routing table (#3434)
new ba9f2dd Add stream related javadoc (#3436)
new b3c2e08 [TE] frontend - Anomaly resolution change label fix (#3444)
new 16559dc Limit the ByteArray wrapper logic inside segment creator (#3439)
new a5291e1 Allow generating new star-tree using Hadoop job (#3442)
new e41cd17 [TE] rootcause - inconsistent data warning (#3449)
new 283c70a Add max length limit to string values (#3445)
new b7a8cc1 [TE] anomalies - backend label translation fix (#3452)
new ac2ab14 Fix the bug for transferring more than 2G bytes (#3453)
new b70f2d5 Fixed the naming bug for merge segment admin command (#3446)
new d4104d4 [TE] frontend - aaronucsd/Added basic YAML editor (#3447)
new 6b790e1 [TE] onboarding - support bulk onboarding by dataset (#3455)
new 7341670 [TE] datasource - change epoch time column to aggregation granularity (#3458)
new f9317ec [TE] new detection pipeline - components (#3462)
new b4496b6 [TE] frontend - aaronucsd/fixed some styling issues for share template (THIRDEYE-2432) (#3457)
new 4e73ba3 [TE] detection - Input data fetcher interface (#3468)
new 2f7ddf8 Relocation manager should relocate segments from any servers (irrespective of the tag) to completed servers (#3466)
new a1cd143 Misc Cleanup: Removed unused member variable from (#3472)
new 506799f [TE] Percentage change and absolute change rule detector (#3467)
new eeaf9db [TE] Reset default lookback for merging anomalies (#3450)
new 3bd8fec [TE] datasource - update dataset if underline pinot table time field changed (#3456)
new ec9de38 [TE] rootcause - adjustment anomaly baseline rather than other numbers (#3461)
new 0b4fd51 [TE] Implement Threshold based Time Window Suppressor (#3463)
new a7df756 [TE] Ability to configure reference links like oncall run-book in the email alert (#3473)
new 1ffadef Enhance StarTreeV2ClusterIntegrationTest to cover multi-trees case (#3448)
new d1fb928 [TE] frontend - aaronucsd/home - changes to sort the application list (#3465)
new e7e9954 Fix typo in BaseRoutingTableBuilder (#3479)
new 19286f0 [TE] migrate minute-level and hourly properties (#3480)
new e0b0f3a [TE] detection - site-wide impact rule filter (#3482)
new c9a4e60 [TE] frontend - aarouncsd/Create Alert - Yaml quick fix (#3483)
new e358d7a [TE] frontend - harleyjj/rootcause - integration test for inconsistent data warning in rootcause-anomaly compoment (#3464)
new 3fee99d [TE] frontend harleyjj/manage remove WoW and change buildMetricDataUrl to take user inputs (#3485)
new 2d1bc7a [TE] frontend harleyjj/rootcause patch integration test for inconsistent data warning (#3488)
new e2dface [TE] dataframe - tolerate complex column names (#3491)
new e8516cb [TE] alert filter UX improvements (#3489)
new 6711f67 [TE] site wide impact filter - config global metric from yaml (#3490)
new 1beedd5 Enhance controller periodic task and scheduler (#3475)
new 285649f [PINOT-7328] Reduce lock contention in physical planning phase by reducing the total number of tasks (#3470)
new f6b5dfb [TE] config - enable rca holiday ranking with dimensions by default (#3493)
new fa54c19 [TE] config - fix broken defaults (#3492)
new 402cf3b Change build status to point apache incubator repository (#3498)
new f21519e Add some documentation for readthedocs.io (#3481)
new f64625b [TE] frontend - aaronucsd/share - added % (+/-) for offsets against current like predicted. (#3509)
new 27cbc56 [TE] New detection alerter to support alerting of legacy anomalies (#3495)
new ac915e4 Let PinotDataBuffer.newIndexFor(...) takes long value (#3512)
new 60a72e4 [TE] detection - add two side pattern in rule filters (#3510)
new 0a43bac [TE] Percentage and absolute change rule filter (#3514)
new bcd05ed [TE] alert performnace - resilience to missing alert (#3497)
new 34d0e12 [TE] rootcause - request chunking for aggregates and scores (#3496)
new 8c845ed [TE] rootcause - fix chunking offset cache (#3518)
new 4323070 [TE] rootcause - prevent add-to-chart popup by default (#3517)
new 6581245 [TE] rootcause - prevent line-break of exclude/hide links in filter selector (#3519)
new 7fe991f Deleting extra check for deletedDir during retention (#3516)
new d77982d [TE] Organize the Holiday list in the alert email (#3520)
new d5ba9b0 [TE] Some minor bug fixes (#3521)
new 3774532 [TE] rootcause - fix filter processing for predicted baseline (#3523)
new ce092c8 Fix the bug where time conversion is skipped when incoming and outgoing time column name are the same (#3484)
new fdb9357 Added a log message on pinot-server when controller leadership change is read from zookeeper. (#3522)
new 3fcf8f2 [TE] rootcause - support metric selection from chart labels (#3526)
new bcd91e3 [TE] Attempt to organize and cleanup the email template (#3527)
new 8fb5f28 Ignore generated documentation files while checking for license headers (#3524)
new 77e394d [TE] datasource - aggregate to one minute level for epoch timestamp (#3530)
new 4741882 [TE] rootcause - primary metric selection from legend label (#3529)
new ebe5060 [TE] rootcause - support remove metric from chart label (#3531)
new 5b1e6ec [TE] Some more minor changes to the look of the template (#3534)
new 0389a6c [TE] roocause - fix dimension table metric select/unselect (#3533)
new 2ce0bca [TE] frontend - harleyjj/rootcause - fix data inconsistency warning t… (#3532)
new 4162fb3 [TE] Anomaly function to new pipeline YAML converter (#3537)
new 79f83e9 [TE] rootcause - prevent metric selection change on dimension table tab change (#3536)
new ce6a53f [TE] detection - add more logs for merger for debugging (#3540)
new d48499b [TE] Pipeline migration - fill in metric timezone (#3542)
new 234abc6 [TE] Display Holidays only from Top Countries (#3544)
new f915200 [TE] rootcause - remove tooltipEntities property (#3546)
new 16c5926 [TE] rootcause - redo route hooks (#3547)
new 81589c7 [TE] frontend - harleyjj/rootcause - warning value uses aggregate multiplier and can be turned off if aggregates change without slider being adjusted (#3548)
new 0bea669 First cut documentation. (#3545)
new 9607ca4 Adding execution stats for numSegmentsQueried/Processed/Matched (#3525)
new 37ad801 [TE] rootcause - default millisecond datasets to 15min granularity (#3550)
new 8bf0050 Optimize all filter predicates by adding isAlwaysTrue() (#3535)
new e23f953 [TE] Alerts created by YAML to show up in the alert list (#3551)
new 8e2076f Fix the wrong number reported by numSegmentsQueried (#3552)
new 7d4ab9e Cleaning data dir so URI construction works as expected (#3549)
new ee4784d [TE] rootcause - preselect metric name in metric dropdown (#3553)
new 4f5b2dd Fixing TODOs in new documentation (#3555)
new 61b4814 [TE] Make new alerter tag old anomalies as notified (#3554)
new ac74d13 [TE] frontend - harleyjj/manage - pass correct granularity to buildMetricDataUrl (#3557)
new a834a49 [TE] pom - fix javassist dependency (#3559)
new 232360f Fixes to doc (#3558)
new d9ad5c3 Removing warnings in docs (#3560)
new 1e6fc74 Adding pluggable storage support for realtime upload (#3541)
new 44c2859 [TE] detection - copy nested property in wrapper (#3562)
new 45063b0 Re-org documentation (#3563)
new b62c611 [TE] frontend - harleyjj/create-alert - make minor improvements to YAML alert creator (#3564)
new eceaf76 [TE] Legacy Alert Filter should pick recipients from new alerter configs (#3565)
new 5a610be Add metric gauge to track the number of llc simultaneous segment builds (#3566)
new 9dfb949 [TE] rootcause - rich metric selector (#3567)
new 8b2efec [TE] detection - filter out child anomaly in anomaly fetching (#3570)
new f1559a7 Docs: Fix formatting for UDF docs. (#3569)
new 88ba23a [TE] rootcause - change frontend hack for baseline for new pipeline (#3571)
new 213ff9c [TE] Configure Filters per Alert in Legacy Alerter (#3572)
new f1512e6 [TE] rootcause - hide heatmap tooltip on click (#3574)
new c752a4f Improve the logging for segment merge command (#3573)
new 75a4f85 Adding support for bloom filter (#3528)
new 1d4bf61 [TE] frontend - harleyjj/rootcause - use humanizeFloat to change formatting of y-axis and prevent apparent duplicates (#3575)
new ef42e58 [TE] Minor fix in legacy alert filter (#3576)
new a4505af Applied humanizeFloat to anomaly-graph component (#3577)
new dac899f Changing segmentCommitEnd to support deep storage (#3578)
new acff279 [TE] frontend - default redirect to home dashboard (#3579)
new 5067baf [TE] Support sendOnce in LegacyAlertFilter (#3580)
new 3219189 Segment merge lineage data structure (#3543)
new 15ff8d8 [TE] rootcause - Round end time after adjusting with the maxDateTime (#3581)
new 366bb39 [TE] frontend - harleyjj/manage - align startDate at startOf() instead of endOf() (#3582)
new b21805c Change format of the ApplyConfig computed config (#3500)
new ed1e0c0 Fix NPE while unparsing single key DSL output (#3501)
new d99d8d8 Add new config API Java-properties format output (#3502)
new 5900ff1 Fix NPE on configs with per-profile table types (#3503)
new 5c84702 Change profile separator to triple underscores (#3504)
new ed88fb4 Config loader includes also look in profiles (#3507)
new e2916ed Minor deserialization fixes (#3508)
new 683b22d Handle invalid durations in the duration DSL (#3505)
new 4bed91b Add segment selector for merged segments (#3583)
new 4b4224f [TE] rootcause - baseline monthly, daily, hourly support (#3585)
new 1211b22 Fix the bug in broker logging for execution statistics (#3587)
new dcdf5d4 [TE] detection pipeline - multiple improvements (#3586)
new ea628a9 add kafka simple consumer buffer and timeout to stream config (#3584)
new 555ba8f Add the bloom filter to equals() and hashCode() in IndexingConfig (#3588)
new 99c25a8 [PINOT-7461] Fix segment deletion when folder under Deleted_Segments location does not exist (#3589)
new feaca0a [TE] Update alert schemes and suppressors in the annotation alert registry (#3590)
new 694f6d7 [TE] rootcause - pin predicted baseline time offsets to PT too (#3596)
new 9fc6e02 [TE] Translator - Yaml Alert Config to DetectionAlertConfigDTO (#3595)
new 24c35e8 Allow completing segments to finish before stopping LLC Segment Manager (#3593)
new a7a1745 [TE] detection - yaml validation (#3597)
new 08d26d6 [TE] datasource - correction for maxtime offset in pinot datasource (#3598)
new 5afbebe [TE] frontend - update default yaml (#3599)
new 6a6c175 [TE] Endpoints for create and edit alert yaml along with validators (#3603)
new 732e656 Cleanup: update code to return only physical columns for select * queries (#3602)
new 569022a [PINOT-7476] Add metrics to track cases where segment refresh/reloads fail (#3606)
new fa12efd [TE] detection - convert algo timestamps to dataset timezone (#3608)
new 80be2cb [TE] yaml - create alert endpoint (#3609)
new d48e4fd Fix segment merge command. (#3611)
new a899cf4 Add config to set batchMessageMode on ideal state of new tables (#3613)
new a7a927f [TE] tracking - increase datasource performance output (#3615)
new f079714 [TE] subscription groups endpoint (#3616)
new 65aa0ec Adding a class to get statistics about a request (query) processed by… (#3614)
new 35d61aa Add ControllerLeadershipManager as single place to check controller leadership changes (#3604)
new f26e482 Filter out virtual-columns in realtime segment conversion path (#3607)
new 0146e5f [TE] frontend - harleyjj/create-alert - YAML autocomplete for create … (#3617)
new 7a1fc25 Refactor ControllerPeriodicTask to iterate over tables (#3618)
new 39e3360 [TE] get all subscription groups endpoint (#3619)
new 3b5f0de [TE] frontend - harleyjj/rootcause - add Anomaly to labels for rootcause legend and tooltip (#3621)
new 09eb015 Start and stop ControllerPeriodicTasks based on leadership changes (#3622)
new 5b8042e [TE] pom - add reflections dependency (#3628)
new cd14055 Added more getters to request statistics (#3627)
new b336c5d [TE] task - limit waiting task retrieval size (#3629)
new 9d50d1b [TE] task - relax task retrieval to 5 per polling cycle (#3631)
new d5a9d08 [TE] remove log for the objects (#3630)
new 24897f6 [TE] task - randomize execution order to increase parallel throughput (#3634)
new ef32663 [TE] backend - use task actual start time as start_time instead of create time (#3638)
new f7387e9 [TE] task - add metrics for task execution and gaude for backlog size (#3643)
new d1a46a6 Revert "[TE] task - add metrics for task execution and gaude for backlog size (#3643)" (#3645)
new c25e5ea Use ZkCacheBaseDataAccessor to cache instance configs in PinotHelixResourceManager (#3633)
new d95cb69 [TE] fix database connection close issue (#3647)
new a031e05 remove legacy codes (#3641)
new 0ea2232 [TE] Skip scheduling detection task if one is already in the queue (#3660)
new 6300558 [TE] fix detection config DTO revert in yaml alert creation (#3662)
new 06dc0e9 [TE] rootcause - fix slider and offset quirks (#3663)
new 969e471 [TE] frontend - harleyjj/home - make email share available even when there are no anomalies (#3632)
new 842678d [TE] task - yet another backlog gauge fix (#3666)
new 46ad4f9 Update license header (#3664)
new ef31dca Fix error messages (#3665)
new 493ed64 Split validation manager tasks into separate periodic tasks (#3668)
new a7bf571 Try fixing flaky tests by adding 0.5s sleep (#3667)
new 94cab26 Added a log when access is denied (#3669)
new ed5ec5e Fix minor log bug in BaseSingleTreeBuilder (#3640)
new 36bbdf1 [TE] Endpoint to return the anomalies for a detection config within a range (#3670)
new e7592ef [TE] license - update apache license headers (#3674)
new f6c4d3b Unify all JSON usage to fasterxml.jackson package (#3677)
new b497851 Move all files in com.linkedin to org.apache (#3680)
new e7b20da Rename all package references from com.linkedin to org.apache (#3681)
new 2e7a878 Change javadoc in RealtimeSegmentRelocator (#3682)
new 39130c3 [TE] meta - rename base package to match new apache pinot convention (#3678)
new c42af32 Change the realtime stats file to a new name (#3685)
new 1d6c1b0 [TE] Order the create and update tests of detection alert config (#3688)
new 4279d09 [TE] frontend - aaronucsd/Basic yaml editor with alert and subscription group editors (#3636)
new 059149b Optimize imports for all files (#3683)
new 7d6c4ef Remove redundant imports in pinot-minion pom.xml (#3690)
new ce072d3 Update license-maven-plugin setting to correctly exclude files (#3691)
new 74e2d5f [TE] detection - duration anomaly filter (#3686)
new 9e82f5e [TE] detection - update tunable interface (#3689)
new fd09b02 [TE] Annotate and update the registry with alert filter types (#3692)
new 3755e83 Add touch method in PinotFS; Call touch when moving deleted segments. (#3684)
new e6dd1ec Cleaning up the pom.xml file (#3695)
new 53923bc Fix the TableConfig toJSONConfig() method (#3694)
new 148f992 [TE] - frontend harleyjj/yaml - reset and view documentation buttons (#3699)
new c48a38b [TE] Translate Alert Config to Yaml (#3702)
new 42002f3 Fix the json deserialization issue in ColumnPartitionMetadata (#3703)
new 25d21cc Cleaning up the license-maven-plugin (#3706)
new d0d5434 Fix SegmentStatusChecker exception (#3704)
new 9cc609d [TE] Remove new pipeline prefix and inject yaml in Detection Config (#3708)
new 17c85c5 [TE] aaronucsd/fix for the live alerts not showing bug (#3710)
new 29a8e54 [TE] detection - change last time stamp to handle data incompleteness (#3712)
new c6131cb Fix the issue of realtime data manager calling wrong API to load segment (#3707)
new a658f19 Add Pinot code style (#3705)
new 3c4c1c8 Add pinot.root property into pom file to make file location package independent (#3716)
new f9ced25 [TE] aaronucsd/remove true in alert list (#3719)
new 2dbe2d8 Adding support for MATCHES Predicate
new 0766764 Enhancing PQL to support MATCHES predicate, can be used for searching within text, map, json and other complex objects
new 8f0ed61 Adding support for Object Type
new eacb83c Adding support to index nested fields within a column for e.g. map, json, Text etc. WIP
The 5644 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org