You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by su...@apache.org on 2019/01/30 21:57:24 UTC
[incubator-pinot] tag mp-0.1.89 deleted (was e535dc7)
This is an automated email from the ASF dual-hosted git repository.
sunithabeeram pushed a change to tag mp-0.1.89
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
*** WARNING: tag mp-0.1.89 was deleted! ***
was e535dc7 Fixing the wrong method call for get offline table config
This change permanently discards the following revisions:
discard e535dc7 Fixing the wrong method call for get offline table config
discard c88928d Changes warning text for funnel when only one metric is selected
discard 4347b2d Only render funnel if multiple metrics exist
discard bbd96bf Cleans up funnel code. Moves checker into funnel render function. Makes funnels position relative
discard 22c033f Adds warning alert when only one metric is selected and funnel metric is up
discard e58bac8 adding instance config change listener on helixbrokerstater
discard 200da4e fixing tests
discard 0454681 adding a pre-shutdown hook for graceful shutown of server and pro active routing table updates =
discard f59c58a renaming KafkaTestUtils to KafkaStarterUtils and moving it to src/main from src/test
discard cf33ef9 making sure that we skip retention strategy if time column is null
discard 5990701 Simple quickstart using pinot-admin
discard b5b3691 Quickstart changes
discard 5030a35 Fixed chaos monkey integration test
discard 445b2ee Adding license header
discard 0d18f15 THIRDEYE-402 Removing references to specific data sets
discard 66327ba THIRDEYE-401 Flesh out info button docs on dashboard
discard 3d0041d Fixing the issue that OfflineTableDataManager won't take the index load config from property store.
discard 0d6ac02 Remove the assertion for getOpenFiles calls.
discard a3fdf90 Adding double value comparator.
discard 2cc4315 Adding detailed pql parsing error into query response.
discard 6f634c2 PINOT-1589 Remove dependency on pinot-common from pinot-api
discard 0ae5cca Modifying chaosMonkey test with new quick-start api, the test still fails though
discard 497f39b Modifying chaosMonkey test with new quick-start api, the test still fails though
discard 3670d08 Build fix
discard 5879112 Pinot hadoop job
discard 78b582e Adding pinot-dist module to generate the pinot distribution for users to download
discard 691739d Pinot-Admin cleanup.
discard 5bf0578 adding some more logging to live instance change listener
discard 7ae1206 attaching a liveinstance change listener
discard 3a72331 review changes
discard 9b4c564 deleting un-necessary css files
discard a333f2f quickstart and the new console
discard 8da22a7 Removing pinot-util module
discard 03de878 [PINOT-1597] Removing lock while updating metrics, limit the parallelism per query based on number of segments
discard 55ddf55 Fixing NPE for flitering query. Fixing exception in combinePlanNode.run() will cause query to be halted.
discard 0cc2da2 THIRDEYE-330 : Create DataGeneratorTool which can generate Avro data sets to load into ThirdEye
discard 04cc010 THIRDEYE-330 : Create DataGeneratorTool which can generate Avro data sets to load into ThirdEye
discard 1727475 THIRDEYE-330 : Create DataGeneratorTool which can generate Avro data sets to load into ThirdEye
discard 99afb1a THIRDEYE-356 Remove dashboard related code from thirdeye-server
discard fb2d7e5 Adding filter query schema check in DataSchemaSegmentPruner.
discard 737f995 Updated readme to make it the same as the GitHub wiki's landing page
discard 9a9157c removed schema version
discard d70c340 THIRDEYE-390 : Add Hadoop phase to poll for existence of input.paths
discard c7d1892 THIRDEYE-390 : Add Hadoop phase to poll for existence of input.paths
discard bfb2f29 THIRDEYE-390 : Add Hadoop phase to poll for existence of input.paths
discard 8b3302c NPE fix
discard 54fa0e4 THIRDEYE-400 thirdeye-dashboard breaks with data resulting from XSS attacks
discard 7523942 Changes tooltip text
discard e110901 Adds info tooltip to Aggregate and Moving Average
discard 96874ba Fixing lsof issue.
discard 6f75cbb Fixing the file descriptor leak for mmap mode and heap mode. Adding test for heap/mmap mode reader and make sure no file opens after close() method is called.
discard 2d47114 adding a live instances listener, not wired yet
discard e5f4266 [PINOT-1597] Loading all segment files using DirectMemory to avoid GC issues
discard e35c472 refactored schema and made all the necessary changes
discard 8255732 Quickstart bug fix
discard 166505c Fixed unit test
discard 2e2aa42 PINOT-1610 Make integration test length configurable
discard 9b26577 NPE fix
discard b5017be THIRDEYE-389 : Use proper umask when creating HDFS directories
discard aca2089 THIRDEYE-389 : Use proper umask when creating HDFS directories
discard 1b0d3a2 THIRDEYE-389 : Use proper umask when creating HDFS directories
discard b0df94b THIRDEYE-393 : Purge all segments of lower granularity once one of more coarse granularity is availble on HDFS
discard c49e2a9 THIRDEYE-393 : Purge all segments of lower granularity once one of more coarse granularity is availble on HDFS
discard d9c1230 To fix all the unit test.
discard 1223004 fixed RealtimeFileBasedReaderTest
discard 422aeef fixing ChunkIndexCreationDriverImplTest tests
discard 0b0f26a Fixing multi-value group by test with new data set.
discard bd95e26 Remove all the callables from metrics and addPhasingTime after the call.
discard fe7343d Fixing NPE for getting next row, if the column from avro is not in pinot schema. Now we will just simply skip it.
discard c94e535 PINOT-1608: Quickstart cleanup
discard d5bdfc9 Remove getLock when querying the segments and use ConcurrentHahMap.
discard 84a50bf Added logging
discard 3addf61 PINOT-1308: Fixing tests failures after scrubbing internal data.
discard 7843b8f Fixed another unit test
discard c829505 Fixed another unit test
discard 1b2a80e Fixed another unit test
discard 132b3ec Fixed unit test post-anonymization
discard ef5ebe3 THIRDEYE-388 : Add "thirdeye.skip.missing" config property to skip input.paths that don't exist
discard 92ad61d THIRDEYE-380 Add support to query in-memory tree
discard 86f5316 Clean up netty dependencies
discard a951191 THIRDEYE-377 : Cleanup job to remove folders generated by hadoop workflow
discard 2e2b196 THIRDEYE-377 : Cleanup job to remove folders generated by hadoop workflow
discard 116db33 Minor improvements to the client
discard 42f5440 Set the version of the libraries for which various versions existed through transitive dependencies
discard d2b9639 PINOT-1308: Test fixing after scrubbing internal data, contd.
discard 07f3641 THIRDEYE-380 Add support to query in-memory tree
discard 80dae21 Adding trace logging into NettyServer for pinot-server request latency.
discard 3d8989c Adding dictionary caching during segment creation.
discard ca29736 Added test where we freeze ZK for a couple of seconds to test error recovery
discard 8cd2bcf Merge: 36e96eb c7f7fce
discard f68149f Tenant and Table creation tweaks
discard e2f03d0 PINOT-1308: First round of cleaning regression failures after data scrubbing
discard f21d56f fixing segment view dates
discard bda0e50 Merge branch 'dashboard' of gitli.corp.linkedin.com:pinot/pinot2_0 into dashboard
discard 921816a Adding tenant creation functionality
discard dcff5ae Fixing gitignore for pinot-dashboard specific vim swap files
discard 6a76866 More Pinot 2.0 Dashboard work
discard b86d135 Accommodating changes to controller API endpoints
discard 2d1e600 PINOT-1308: Yet another round of data scrubbing
discard 3074ff7 Given charset to getBytes() method to avoid platform default encoding mismatch.
discard 1df22ec PINOT-1308: Replacing internal data in test.
discard e8e2f40 fixing shema NPE
discard aab8a1b [PINOT-1597]:Adding custom serde's instead of using java's object serialization
discard 3842bbf PINOT-1308: Replacing internal data in test.
discard 2d28bd3 THIRDEYE-388 : Add "thirdeye.skip.missing" config property to skip input.paths that don't exist
discard d8d0610 THIRDEYE-388 : Add "thirdeye.skip.missing" config property to skip input.paths that don't exist
discard 2d69429 Fixing the string write for dictionary with special characters, as for special chars, the string.lenth() != string.getBytes().length.
discard d7a497e Run 10k queries instead of 1k, correct comparison in other branch, use cast to double for average queries
discard 3a4a3d4 Adds else statement to auto set timezone if timezone is set as empty
discard a639b5d Migrates from search to hash for timezone
discard 855cf5c Adds timezone dropdown, Volume/Contribution Modal
discard 30d69b2 Fixing gitignore for pinot-dashboard specific vim swap files
discard 0385ecc More Pinot 2.0 Dashboard work
discard 0dd2d47 Fixed segment exception logging
discard 2118c13 Fixed NPE on broker startup
discard a55fa1a Fix NPE in metrics if the BrokerRequest has a null query source
discard c2a94ab PINOT-1607 Add sleep between requests to fix flaky unit test
discard 1601d9b Fixing the valueOf a dateType for double type.
discard b4d4624 PINOT-1538 Added license headers for thrift and antlr files
discard 88fafe2 PINOT-1599 Javadoc fixes post-refactoring
discard d19c09a Adding more metrics for server phasing
discard c0ef63a Accommodating changes to controller API endpoints
discard fc37e58 THIRDEYE-377 : Cleanup job to remove folders generated by hadoop workflow
discard b65d2e3 THIRDEYE-377 : Cleanup job to remove folders generated by hadoop workflow
discard fcd04a4 Made /tables on the controller list all tables instead of just saying that no table name was specified
discard 1e792d0 Fixing NPE in getting routing table builder
discard c6bca8a THIRDEYE-376 Metric tabular view is off-by-one
discard 02bb44d Fix NPE in integration tests when starting the broker
discard ddddfdc THIRDEYE-379 : Segment descriptors should receive time in millis
discard 2172790 Removed author tags in the client
discard ef43dae Fixed javadoc generation and various javadoc compilation errors
discard 5c5229b Wire balancedRandomRoutingTable to broker.
discard 33a2503 Removed classmexer due to uncertain license terms
discard 2797826 [PINOT-1597] Improving the performance of Planning Phase
discard 752e0e2 Adding single tenant cluster as a default option.
discard 687e1b7 Adding balancedRandomRoutingTableBuilder
discard ab4e0ff Added UI for AddTenant and AddTable commands
discard c6ce903 Added logging for the pinot client, update the integration test to use the pinot client ZK path, added connection setup through properties
discard f8ee33f Pinot client
discard e4871f4 THIRDEYE-372 : Include wall clock times, aggregation unit, bucket size, sxhedule and data time in millis in metadata file
discard 25cfd87 Adding copyright headers to pinot dashboard python files
discard 05f484c THIRDEYE-375 Clicking tabular view off-by-one error
discard d2042bc THIRDEYE: Multiple changes related to unit of time, metadata and selecting trees to query
discard 4b4dcbf Added setters, renamed class, and other cleanups.
discard efe250c Adding instance enable/disable Apis.
discard 15888ce Removing author tags for open source preparation.
discard faf669a [THIRDEYE-371] Select the trees to query such that the number of trees to query is minimized. Bug fixes and adding test case
discard 8e187ba [THIRDEYE-371] Select the trees to query such that the number of trees to query is minimized. Bug fixes and adding test case
discard bf0f278 Fixing NullPointerException during getSegmentInfo. Return Empty response instead of null if routing table contains nothing, this is corresponding to querying a table without any data.
discard 9391f6a Move all the resource related methods, variables, configs to table related stuffs.
discard 4d097cc Clean up controller v1 apis.
discard f1905b5 PINOT-1529: Enhance CSV record reader to take a config file.
discard dbd972c added add table, add schema and add tenant commands, also added sample_schema, sample_tenant, sample_offline_table and sample_realtime_table request.jsons
discard 8343c7b renaming addTenant to createTenant
discard 17cabc7 adding missing segment get APIs
discard 4e0afd1 added table update apis
discard 42ec467 add tenant tools command added
discard 23b62c7 refactoring tenant
discard d557956 THIRDEYE-372 : Include wall clock start and end time and schedule in metadata file
discard 73d0bab [THIRDEYE-371] Select the trees to query such that the number of trees to query is minimized
discard 3ffaeac PINOT-1529: Bug fixes in JSON/CSV Record Readers.
discard 186168c PINOT-1529: Bug fixes in JSON/CSV Record Readers.
discard aac0a56 Removed dead code that accesses Helix, added HelixHelper.updateIdealState which has retry logic to update the Helix state in ZK in the case of concurrent updates
discard 59b8e0e Fixing pinot-broker test. Delete deprecated tests.
discard cd44264 making realtime, offline and hybrid integration test pass
discard 75bb4c6 making realtime, offline and hybrid integration test pass
discard 93eb33e bug fix in the property store helper
discard cf52049 adding a Property Store helper
discard e8d5064 making server and core using the new table Config
discard bcb6c5a renaming Resource to table in pinot-server
discard 8701826 adding sorted columns in the indexing config
discard 953b881 fixing compile issue
discard e6d82c4 adding get calls on table for get brokers, servers and get configs
discard 37e65b0 Fixing broker,controller,transport v2 apis Remove useless test Add deprecated notion to v1 apis
discard d341d43 Fixing ValidationManager with controller v2 apis.
discard 67df11d Fixing RetentionController with v2 APIs
discard 9664f77 Adding ControllerSentinalTest for v2 apis with addTable works.
discard 4853002 removing constants with enum in Tenant name builder
discard 53df8ca removing constants with enum in Tenant name builder
discard 0395fe3 PINOT-1529: JSON record reader.
discard 21d5e62 PINOT-1529: CSV record reader cleanup
discard 58156a1 PINOT-1529: CSV record reader
discard 56e2cd3 adding license headers
discard e5c6765 renaming ResourceType to TableType
discard ff8fad1 finishing up all table APIs update and get
discard 89efedc adding new table APIs ... currently they are not wired
discard 9b6bf95 adding schema apis
discard afc3397 Adding tenant into controller
discard a72ddcc PINOT-1231: Rename all tests with Test prefix (eg. TestFoo) to suffixed form (eg. FooTest)
discard c593a2a Several tickets...
discard d792e9d Remove table concept from codebase
discard fd027b3 Updated log4j.properties
discard 761de38 Added support for multi-value columns to the query generator and H2 insertion
discard c222c37 Put HelixSetupUtils and ZKMetadataUtils into package: com.linkedin.pinot.controller.helix.core.util
discard be7d13a Ignore text and log files from the license header check
discard 9bd174d PINOT-1518: Clean remaining JUnit instances
discard 4074353 Fixing filtering query bugs with *BlockDocIdSet changes.
discard 7764081 Adding file loggers for controller/broker/server in quick-start
discard 955328b postData streams upload instead of keeping all in memory
discard 4fed9d9 1. Fixing String comparison in string dictionary by reversing the way to store a String: first put String then use * to fill the gap. 2. Rewrite the sorted column docIdIterator, the logic before takes multiple pairs but return all the docs between the left value from first pair and right value from last pair. 3. Fixing BitmapDocIdSet advance(int) and next() method. next() should first trim data in priorityQueue, then add new pairs, after all added, we shouldn't remove th [...]
discard 9eb741d PINOT-1518: Removing JUnit dependency
discard 635dea9 remove adding of default value to realtime dictionary on initialization among other things
discard 2d61273 PINOT-1518: Removing JUnit dependency
discard 167c3ee Adding interface for FilterBlockDocIdSet and abstract class for BaseFilterBlock to force some implementation for filtering. Fixing selection unit tests by not hard coded results set. Fixing AndOperator/OrOperator logics to handle all the BaseFilterBlock.
discard 0872bfc fixing testReatlimeSegment
discard 34ecaa9 removed unused blocks
discard 65e2c35 Fixing MProjectionOperator got called many times issue.
discard 77ddc87 THIRDEYE-365 Add support for nested fields in ThirdEyeAvroUtils
discard 483393d Escape HTML from dimension values
discard 62d10c8 Fixing scan based matching docIds find issue by change the binary search to hashset. Adding a test case in QueriesSentinelTest to cover this case.
discard e30b7c0 Fixing the issue of selection will reduce number of rows more than size from query.
discard e58af22 Minor fix - use IOUtils instead because of classpath issue
discard ee22ccf Minor fix - more logging in thirdeye kafka
discard e4ba899 Fixing range function in StringMutableDictionary Fixing PriorityQueue constructor, as raw.length == 0 is not valid. Adding handle ParsePQLException in Broker.
discard 767073a Fixed thirdeye-realtime to use new data directory structure
discard 957057c Extracting sorted columns list from controller command, key is metadata.sorted.columns
discard a3777a8 Skip check of selection queries for now
discard a5e4617 Re-added generated queries to the integration tests, added logging of mismatched queries with debugging information, added log replay, fixed number of generated aggregation functions to 1 for easier debugging, changed query comparison to check that Pinot results are contained within 10k rows of H2 results (skipping comparison if more than 10k rows are returned by H2), added dataset with no null values, changed query generation to not quote numbers sent to H2
discard 5755257 Fixed file handle leaks
discard e02f124 Added comments and renamed a variable to make OrBlockDocIdSet more readable
discard b30d1f8 Removed Sonar warning about passing an int to longBitsToDouble
discard 5815df5 Fixed broken unit test, added unit test for findNthBitSetAfter, corrected implementation of findNthBitSetAfter
discard 593c75c changes to the *Stats collector
discard 5ad0c21 using FixedBitSkipList reader/writer for MultiValue columns
discard c9e30b6 Added log4j.properties for pinot-admin
discard ed22fec Adding sorted column list config into DataResourceZKMetadata and link it with RealtimeSegmentDataManager.
discard 06ca045 PINOT-1507: Adding skipList based storage for Multi Valued columns
discard a512c52 Using BlockingQueue for server merging phase. Fixing NPE in related combine functions. Remove Exception handlers in MSelectionOnlyOperator. Handling metadata only DataTable Ser/DeSer.
discard 93f8e8d Fixing broken version pin
discard 0ed864f Adding provisions to hook into the wrapper's auth mechanisms
discard 536a27a renaming dicIds to dictionaryIds in the evaluators
discard 0a3919d removing fastutil sets from segment creation
discard f5fe25a adding license headers
discard e6f173b adding default values in the query generator for nulls, also using bitmap operator when inverted index is present in the filter plan node
discard 9004dd9 using predicate evaluators in all the filter operators
discard 55763d3 added predicate evaluators
discard 7824cc3 removing the hasDictionary check from the filter plan node
discard fff1309 adding dictionary for metrics in realtime segment
discard f3c7249 now sorting the data on the sort column while converting segment from realtime to offline
discard 5571dfd Use separate path to static content to avoid conflicting with mppy tools wrapper
discard 2bdbcc6 Minor fix, offset bug in dashboard overlay timeseries
discard 29618d7 Minor fix, do not select default metric if some selected
discard c2d0f75 Minor fix, dashboard metadata in table
discard 67952e8 Display stats in dashboard tooltip
discard 64c4a82 Added missing license header
discard 14b0e1e Added multiple node offline cluster integration test
discard 2a86a5b Code cleanups: Added exception logging, adding missing hashCode method, made fields final, changed ControllerRequestURLBuilder to be immutable
discard 81696f8 Invalidate the segment metadata if refresh fails, log if for some reason the ZK write to the ideal state fails
discard b1cc77f Revert "Adding mutable dimension store"
discard 83555b0 adding license headers
discard 429f5f8 Adding more logs for dealing with segment deletion.
discard 6d1e42d fixing realtime dictionary get checks, removing default checks for == nulls, all tests pass
discard 401badd adding dictionary check into the bitmap operator for realtime vs offline range computation
discard c995134 merged with kishore's changes, also made modifications to Range Delimiter and MutableDictionary(min,max) values
discard b246bf1 merged with kishore's changes, QueriesSentinelTest passing
discard a24a507 changing perf drive to use the new apis
discard aa19bbd IndexSegmentImpl now uses the new *ColumnIndexContainer, also cleaned up segment loading
discard 5907459 finished implementation of the *ColumnIndexContainers, also checking in a new IndexSegmentImpl and DataSourceImpl, the plan is to get the new impls to replace the new impls eventually
discard 67915ff foundation for Segment Loading refactoring
discard 586bcff more index creator refactoring
discard 7079c5a added license headers
discard 1371352 refactored forward index creation during segment creation
discard 843a9de packages refactoring for index creation
discard 6b99544 Added another exclude for thirdeye-realtime
discard 2e46bf8 Downgrading thirdeye Kafka dependency to 0.8.1 for container compatibility
discard 34087a9 Fixed loggers in if statements for thirdeye
discard 98264bd Logger cleanup part 2: All loggers renamed to LOGGER, all loggers made static and final (except where we can't)
discard 2ac53dd Added api to set port for broker.
discard 8022388 Minor UI fix
discard 91bf2a6 THIRDEYE-262 (Several dashboard bug fixes)
discard 85b0fcc PINOT-1457 Converted code to use slf4j instead of log4j or JUL
discard a17fe7c Adding mutable dimension store
discard cae7411 THIRDEYE-351 : Bootstrap should continue gracefully if no records below threshold
discard a292ca9 THIRDEYE-351 : Bootstrap should continue gracefully if no records below threshold
discard 8ecc6df Adding another port for brokerTest
discard 050a3b9 Use available number of threads.
discard 171bcad PINOT-1449: Refactoring RangePredicate Evaluation logic into one class
discard ecc2627 THIRDEYE-350 : Output metadata file with min and max time after StartreeBootstrapPhase2
discard 8f2d28c THIRDEYE-350 : Output metadata file with min and max time after StartreeBootstrapPhase2
discard 6364345 THIRDEYE-350 : Output metadata file with min and max time after StartreeBootstrapPhase2
discard 288a6dd THIRDEYE-350 : Output metadata file with min and max time after StartreeBootstrapPhase2
discard b71b0d5 Bug fix: File size must be in bytes not bits.
discard 26cd285 PINOT-1448: Rewriting the Filter Operator Plan and execution phase to incorporate efficiency
discard 45ef241 Made segment creation multi-threaded.
discard d413df1 Chunk POST request in server_push
discard bcf9223 Use jackson 2.4.3 in thirdeye-server
discard e6608e2 Reorder the data schema segment pruner checking sequence. Swallow zk shutdown exceptions for OfflineClusterIntegrationTest.
discard 34d8bdb Reverting (Upgrade thirdeye-server dropwizard dependency to 0.8.1)
discard a4d643d Upgrade thirdeye-server dropwizard dependency to 0.8.1
discard 6ab1787 Fixed unit test coverage not appearing in sonar
discard 6733144 Copying DropWizardApplicationRunner to thirdeye-dashboard
discard acc3c18 Separate SelectionOperator to SelectionOnlyOperator and SelectionOrderByOperator. Abstract static functions to SelectionOperationUtils.
discard 2b1457f Refactorings changes for Pinot1 vs Pinot2 comparisons
discard 16b47fc Refactorings changes for Pinot1 vs Pinot2 comparisons
discard b09db97 THIRDEYE-336 : RollupPhaseThreeJob during bootstrap should not fail if it can't find dimension combination
discard 4f207fb Removed Praveen from the product owners, to cut down on email spam
discard 494515d Removed some unused variables, extracted some hardcoded constants
discard 24f7844 Removed some redundant column type code, added time type for time columns
discard 9fb7674 Change jvm setting for mvn test
discard c8ecda3 Adding IndexLoadingConfigMetadata and use it to switch on loading inverted index.
discard cabb7bb THIRDEYE-348 : LeaseExpirationException on hdfs as multiple mappers writing to same file in StarTreeGeneration
discard 4ab0437 feedback changes on Realtime Segment Changes
discard 64e5e27 Realtime Segment now using *ReaderWriter for indexing and scanning
discard 05706a9 minor cleanup
discard 3986517 PINOT-1478: Adding implementation for ReaderWriter for SingleColumnSingleValue and SingleColumnMultiValue
discard 6e5b11a Remove test annotation for TestPerfBenchMarkDriver
discard bc5c01d THIRDEYE-335 : DELETE of collection resulted in NPE
discard 6f3827e THIRDEYE-346 : Throw exception from hadoop jobs for Azkaban to identify failed jobs
discard 00fe751 Fixing HybridClusterIntegrationTest by changing some member variables from static to non-static in HelixServerStarter and SegmentOnlineOfflineStateModelFactory. Fixing the override issue from error msgs to response Fixing EqualityUtils for map fields.
discard 75fa6cd avoiding recomuting filteredIds twice
discard 3bea287 THIRDEYE-344 Add ratio support to dashboard (fix bug)
discard af46643 [PINOT-1410]: Quick Start Pinot Admin Console contd. Cleanup after RB feedback.
discard c5505e9 Fixed Kafka consumer path for hybrid
discard 63e86ac Initialize the metadata of the data table with some defaults, fixes NPEs in DefaultReduceService and BrokerRequestHandler
discard 3424709 THIRDEYE-345: Separating Hadoop workflow into DIMENSION_INDEX and METRIC_INDEX
discard 9240cf4 Clean up OfflineClusterIntegrationTest/RealtimeClusterIntegrationTest. Move the common test cases to BaseClusterIntegrationTest
discard 956595d Fixed failing unit test
discard a2170fc Added equals/hashCode to KafkaStreamMetadata and Schema, removed star imports
discard 9905b72 Adding missing license header
discard d2daec3 Adding missing file, cleaned up some equals and hashCode methods
discard 94d094e5 Added unit tests for ZK metadata equals and hashCode methods, fixed issues with equals and hashCode
discard 2173103 Added missing hashCode methods to classes having equals methods
discard 52b05e6 Fixing {Offline/Realtime}ClusterIntegrationTest. Change retention to be 3000 days to avoid segments being deleted during test.
discard 7c962cf Fixing NPE from segment pruner by adding schema to realtime segment metadata. Fixing NPE from not in predicate. Adding hard coded query set to RealtimeClusterIntegrationTest.
discard 6718033 modified constructor for *ReaderWriter based on feedback
discard 271c4eb license headers and constructor for *ReaderWriter
discard 1d32e53 PINOT-1478: Adding base reader writer interface for realtime
discard 69a4d66 THIRDEYE-344 Add ratio support to dashboard (fix bug)
discard 6a0d8f5 THIRDEYE-344 Add ratio support to dashboard
discard 568d66f Refactor HelixHelper from pinot-controller to pinot-common. Change the module dependency for pinot-controller to test scope in pinot-broker. Fix failed pinot-broker unit test.
discard 7111d46 Fixed inverted constructor, removed unnecessary boxing, removed unused variable assignment
discard 529dc38 Added two minute time limit for realtime consumption to complete on integration tests
discard 9120e58 Added property to override the segment creation time, so that we can keep it on segment conversion
discard 7ecff86 Added logging for exceptions occurring in the state model transitions, rethrow exceptions instead of swallowing them, so that Helix can get them
discard e2c441b Fixing NPE in ControllerSentinelTest. Fixing TestPinotResourceMananger by using empty server instances.
discard 1a5df5d Added thirty minute per test time limit, to avoid stalled tests
discard f370cf6 Fixed data not being written properly in the realtime integration test, added kafka directory creation/cleanup, fixed single valued fields being treated as multivalued, added logging for kafka production
discard 3e884cc Fixing RetentionTest by change the delay time to start.
discard ad0944b license headers
discard df0988a Enable segment pruners for time interval and data schema. Adding hardcoded query set of OfflineClusterIntegrationTest
discard 728b240 changed the integration test query set since the data set has been changed
discard 9834fa9 fixing some edge case bugs in data block iterators, also added a blocks test to compare BlockDocIds with filtering across fwd index and inverted index filtering
discard 9cfecbb (ThirdEye dashboard tickets)
discard 6586615 Added missing integration test data file update, removed table creation in hybrid and realtime modes
discard 70d01ef THIRDEYE-340 Add funnel view to thirdeye-dashboard
discard 539ec9b single and multi value blocks without inverted index changes
discard dcf2313 single and multi value blocks without inverted index changes
discard d3b5ded single and multi value blocks without inverted index
discard a719b6c Adding logic to filter plan node to take care of filtering with/without inverted index.
discard 4dba380 adding license headers
discard 1039e2d Adding Scan based AND/OR block/iterator, which compute the filtering on the fly.
discard b54c08f logging query , expected and actual values on Assertion failures
discard b2d9a4a Added a DataSourceMetadata inteface and relavant impls, some more refactoring, also added a backup filter2 method
discard 34ddd6e added a sorted inverted index block, currently does nothing different
discard ada02b8 more refactoring to DataSource, Blocks and Predicate evals
discard 7ccf334 now using BitmapUtils for or'ing at the predicate level as well
discard 7ca12ea refactoring And and Or operation performed on bitmaps inside BAnd and BOr operators in to a util class BitmapUtils
discard 072b1ea split the actual data source predicate filtering and bitmap or'in .. moving the dictionaryId filtering into a separate util class so that it can be used elsewhere too
discard 839992a using InvertedIndexReader instead of Bitmap*Reader
discard e6c2012 adding sorted column inverted index reader of type inverted index reader
discard 88e4f7a adding sorted column inverted index creator as an inverted index creator
discard a1b6f53 made changes to make realtime test schema creation correctly and also handling default values for null in relatime ingestion as well
discard df78578 converted unnessecary log.info to log.debug and also generated the static query set again
discard 2d7971d added a static queries file and also now segment creator assigns default null values for all data types, no more null checks
discard 4872eb4 Making aggregation functions to take care of null value. Changing query set for integration tests.
discard 507eb75 Change method name for Jackson
discard 2faf15e Adding configurable broker timeout
discard f8ea3c4 Fixing pinot-server unit test
discard 2c1edb7 PINOT-1448: Removing unneccesary conversion of bitmap index from immutable to mutable
discard 89c36de Adding inverted index for realtime segment metric column.
discard bf2e746 Generate a random number in the getRandomFileName instead of just using System.currentTimeMillis, as it causes a race condition if multiple threads are converting segments and start conversion at the same millisecond.
discard e2e5a28 fixing realtime block iterators hasNext logic
discard f8c5cfc remove auto generated comment
discard ae7bba1 making size consistent across types of realtime blocks
discard 9fb798e finxing inconsistensies between realtime segment and the converted offline segment, now data matches across the realtime and the converted offline segments
discard 8ec2a5a PINOT-1448: RB=467880 Optimizing the bitmap intersection logic by ordering the evaluation according to the posting list length
discard ca4844a Renamed variable that referred to offline servers, removed code that is not needed/fixed compilation since RB466683 is in master (test refactorings)
discard c899af2 Added hybrid integration test
discard 71537aa THIRDEYE-338 Refactor dashboard into separate thirdeye-dashboard module
discard a3a826d THIRDEYE-337 Collection restore with no data dirs requires server bounce
discard 808526b Adding simple support for filtering metric column. This is a temp workable solution. Need to improve this later.
discard ab4c5d8 Adding default null value to FieldSpec. Adding default null value to realtime dictionary. Fixing the issue that time field only persisted dictId to disk. Simpliying rowComparator logic in SelectionOperatorService. Removing dummy timeFieldBlockIterator.
discard 33ffa9e Remove _segmentId and _docId from selection. Fixing time value iterator. Fixing multi-value block reader of null value selection. Remove unused log from TermsFilterQueryTreeConstructor. Unit test fixing for: TestPlanMaker,TestSelectionQueries,TestSelectionQueriesForMultiValueColumn
discard 50fb9b0 Rebuilding the Thrift-generated code now requires -DbuildThrift=true, as it breaks subsequent builds later due to there being no license headers on the autogenerated code
discard af81dbb Refactored interegration tests to share the common tests, added hardcoded query set
discard 9a8321d PINOT-1448: First set of Performance improvements targetted towards selection queries
discard 9387478 Replaced System.exit in a test by a volatile boolean check
discard 487be96 Fixing issue of block iterator that last doc get EOF.
discard f8684e0 Fixing RealtimeDictionary issue. Now RealtimeQuerySentinelTest passed.
discard 7bb371d Added logging where missing, replaced throw new RuntimeException(e) calls by Utils.rethrowException
discard 1345c13 incrementing the success count as well in the realtime segment impl
discard 0d9cfd4 making read faster in realtime segment
discard 5ba2a70 Fixing NPE during filtering unexisted value in realtime segment. Fixing no docs matched issue for aggregation functions.
discard af2bb58 Added missing license header
discard 713dd16 Added realtime integration test
discard 2f15091 PINOT-1465: Fixing complex filter tree constructor use duplicated hashcode as id.
discard 8bcf3fe 1. Bug fixing for string dictionary collect handling null value for multi-value column. 2. Bug fixing for LongDictionary Reader.
discard fcdc9a5 PINOT-1434 Consolidate Helix access into one class
discard fbca3cf THIRDEYE-334: Default date on dashboard should be max date loaded into collection
discard 00f4bc3 Fixing dictionary interfaces and implementation
discard 415c492 THIRDEYE-325 : CollectionsResource should record lastPostDataMillis on a per-collection basis
discard 3956b40 More code cleanups: Added logging, added missing @Overrides annotations, etc.
discard 0a24345 Added projection queries to the query generator
discard 7cf5eb8 Exception handling cleanup and added exception logging for pinot-core
discard 1b3eee6 PINOT-1456 Setting default value for selection limit and offset in request.thrift file and regenerated the source files
discard fbcbd46 THIRDEYE-266 A user should be able to delete individual time segments
discard 9066abc THIRDEYE-104 Add SQL support for star tree queries
discard 29a49df Refactored integration test, made functions in the query generator lowercase
discard 9f343f9 PINOT-1456 Setting default value for limit and offset
discard 523ff72 More code cleanups: Fixed equals methods that did not take Object, added missing @Override annotations
discard e59da35 More code cleanups: removed unused imports, merged ifs, renamed variables and removed unused fields
discard 5766d8c Removed hashCode method that was unused, as none of the FilterQueryTreeConstructors are on the key side of a hashmap
discard 6806421 Code cleanup: Added logging instead of printStackTrace statements, used IOUtils.closeQuietly instead of try/catch blocks
discard 6136c91 Adding missing hashCode method (for parity with equals)
discard 1dd93fc Generate different queries for H2 and PQL dialects
discard c763792 Run generated queries instead of fixed queries
discard 5d80ae6 Run queries against H2 instead of using hardcoded result values
discard 10a2d66 Adding comment for usage of .unTar versus .unTarOneFile
discard 202ef37 PINOT-1434: Adding performance benchmark driver, set up local cluster, uploads segments and runs perf queries.
discard c86d3ad Converting pinot-dashboard to be deployable egg
discard 7c0d03e Converting pinot-dashboard readme to markdown
discard d6dc4c0 [PINOT-1410]: Quick Start Initial Version on local host
discard 14b8e71 Incorporating more of Dhaval's suggestions
discard c084490 Incorporating Dhaval's suggestions
discard bd5f31c Adding code for python-based pinot dashboard
discard 5a90294 THIRDEYE-324 HDFS data directory should use wall clock time
discard 2bb1a10 Adding Mayank to the ACL
discard 5fa191e Fixed unit test that depended on zk running on localhost:2181
discard 111e59f Fixed TestDimensionsAndMetricsSerDe, which was expecting null, not the string "null"
discard df533b0 Clean up temporary files when an exception occurs during download or unpacking of a segment file
discard 5f9506a Add segment load retry logic with exponential backoff
discard 2605607 Rethrow load exceptions so that Helix can properly mark the partition as ERROR instead of ONLINE
discard 28748d0 Properly handle Exceptions for timeBoundaryService.
discard 126f5c5 Lazily load bitmap inverted indices
discard 39cb75c PINOT-1395 Make Predicate to be abstract and make EQ, NEQ, IN, NOT_IN, RANGE predicates.
discard 8c79125 Adding if-check for skipTo(docId) method. For realtime segment query, we may skip to a docId larger than the searchable index.
discard 0616a52 THIRDEYE-318 : Output counters for the total sum of each metric in startree_bootstrap_phase1
discard ecacd6b THIRDEYE-318 : Output counters for the total sum of each metric in startree_bootstrap_phase1
discard b0a9dbf Fixed unit tests (DefaultReduceService, QueryExecutor) that broke after the time column fixes
discard 4db92ca Changed hardcoded path in unit test to one that's looked up at runtime
discard 08bad55 Fixed unit test that broke after the time column fixes
discard 0091d86 Adding dynamic config to initial RealtimeSegmentDataManager
discard df48394 Move realtime segment persistence dir to "$resourceDataDir/_tmp" to prevent potenial filling up /tmp during large segment creation.
discard af2e649 THIRDEYE-323 : GET "/" should redirect to "/dashboard"
discard ce4e840 THIRDEYE-323 GET "/" should redirect to "/dashboard"
discard 4519f94 THIRDEYE-319 : Add health check to identify possible holes in the data
discard 18d27e9 THIRDEYE-319 : Add health check to identify possible holes in the data
discard 13511a6 THIRDEYE-319 : Add health check to identify possible holes in the data
discard ca27df0 Fixing issue when delete resource, controller won't clean up PropertyStore clearly.
discard 9caafdb Fixing NPE from uploading large segment.
discard e4f8a0d Fixing the issue of fetch kafka schema everytime when indexing event.
discard 161d8ba Adding Kafka metrics
discard 6329338 fixing for set/get time column name in SegmentGeneratorConfig
discard 4d4254f Remove unused config in segment creation; Adding more logs.
discard 1f75386 THIRDEYE-322 Downgrade thirdeye-server avro to 1.4.0
discard dc4e770 THIRDEYE-300 : Add StarTreeHealthCheck to server
discard 4e2e1fc THIRDEYE-320 Publish shaded jar for thirdeye-server
discard c1438d9 Fixing for getting data type inside avro from union data type.
discard c6816fb THIRDEYE-316 DropWizardApplicationRunner should expose MetricRegistry in return value
discard ee14202 Removing references to AutoCloseable, since it's a JDK 7 feature
discard 108587d THIRDEYE-321 Multi-time series accept: */* rendering error
discard 54f64c0 Compilation fixes
discard 9a84847 Adding missing license header
discard 743b5ad Close all file handles after creating a dictionary, added a utility to unload direct byte buffers from memory
discard ea55296 Adding enum to string converter for Object[].
discard 6e378d4 THIRDEYE-316 DropWizardApplicationRunner should expose MetricRegistry in return value
discard 094ee52 THIRDEYE-320 Publish shaded jar for thirdeye-server
discard 87eedd4 THIRDEYE-299 : Add TimeRangeHealthCheck to server
discard e936b21 THIRDEYE - 296 : Complete implementation of snapshots heatmap
discard 040ebda THIRDEYE-299 : Add TimeRangeHealthCheck to server
discard 5676541 Bug fixing for handling null value in multi-value column.
discard 88e4128 THIRDEYE-316 DropWizardApplicationRunner should expose MetricRegistry in return value
discard 2b2076d converting enum and boolean fields in incoming realtime record toString
discard 5c1a7fe Fixing issue with multi-value column equals null.
discard f03b44e - {THIRDEYE-314} : Catching exceptions in StarTreeRecordStoreFactoryDefaultImpl.MetricStoreRefreshWatcher to make sure that the file system watcher thread never dies.
discard b641679 [THIRDEYE-293] Fixing broken test case
discard dfcd58c THIRDEYE-291 Add day as unit for baseline on input
discard 3c2504c THIRDEYE-308 High-loss of dimensional data (most records go to other) from Kafka consumer
discard dbcbbc9 [THIRDEYE-293] Fixing broken test case
discard c0284a8 - {THIRDEYE-302} : Dropping record with timestamp field set to 0
discard 5112f25 In controller, make eval(..) function synchnized to make sure one time only one thread is creating realtime segments.
discard 1c8bc7a Removed wait for partitions to be online in the external view, made the online to offline timeout configurable
discard 16d24cd adding Joe as one of the committers
discard a0647c1 Workaround for HELIX-590, where multiple concurrent writes to the same znode would cause some of the writes to fail silently, causing some partitions to not be brought online
discard 5b21e38 [THIRDEYE-293] Adding Generic Join Job
discard 5907640 Adding dynamic groupId creation to avoid duplication. If config: stream.kafka.hlc.group.id config is setting, we will pick up the string from it and as groupId. Otherwise, we will create it dynamically by given: [resourceName + timestampInMills].
discard be4d145 THIRDEYE-307 Add startTimeMillis to kafka.yml configuration
discard 0ce9921 THIRDEYE-304 : Make representation of divide-by-zero metric values on the dashboard uniform
discard 1e3aff0 - THIRDEYE-221 : Adding MRUnit test cases for startree bootstrap phases 1 and 2
discard 0cf839e THIRDEYE-296 Complete implementation of snapshot heatmaps
discard 8a45290 Bug fixing for federated broker test. 1. fix logic for RealtimeColumnDataSource set range predicate as for realtime dictionary, dictId is from 1 to dictionary size. 2. in MutableDictionaryReader, getString is also used for other data type dictionary, so change it to toString(). 3. fixing offline ColumnDataSourceImpl would set filteredBitMap to null in Range query, which actually should be an empty bitMap. This is the root cause of DummyBlock get called. 4. fixing NPE in [...]
discard c234ab0 [PINOT-1347] Fixing bugs for realtime indexing. RealtimeSentinelTest for aggregation and aggregationGroupBy
discard 95ccd7c Adding RealtimeQuerySentinelTest
discard 1290605 Fixing realtime indexing issue with dimBuff hash computation. Changing data type from IntBuffer to ByteBuffer for dimBuff.
discard 3603f07 THIRDEYE-229 Initial implementation for high-level Kafka consumer
discard 9cb3489 THIRDEYE-290 : Server should return 400 Bad Request if provided invalid dimension
discard f2bd893 THIRDETE-290 : Server should return 400 Bad Request if provided invalid dimension
discard eeee638 THIRDEYE-290 : Server should return 400 Bad Request if provided invalid dimension
discard 798c947 THIRDEYE-290 : Server should return 400 Bad Request if provided invalid dimension
discard 354dbe6 Revert "THIRDEYE-290 : Server should return 400 Bad Request if provided invalid dimension"
discard 50cfddf THIRDEYE-290 : Server should return 400 Bad Request if provided invalid dimension
discard 19b4b4f 1. Adding realtime retention. 2. For RealtimeSegmentMetadata, still put hybrid resource name in SegmentZKMetadata and converted offline SegmentMetadata. 3. Fixing issue when we drop a RealtimeSegment, we may delete it twice.
discard 6d1a704 Added logging whenever we are not refreshing a segment due to it already existing, return success whenever we are not refreshing the segment due to it being identical
discard 2108a63 Add instance ID and current timestamp to the temporary untar directory
discard 451e4b1 Fixed issues when the segment temp directory is shared over NFS between multiple controllers
discard d8101c6 Fixed NPE
discard fd455f8 - THIRDEYE-298 : Using input time unit size while computing source time window
discard cbae61d - THIRDEYE-219 : Adding MRUnit tests for rollup phases and startree generation.
discard 6a7962c - THIRDEYE-296 Taking aggregation bucket size into consideration while doing aggregation.
discard 9950f7e Fixing time value object inconsistency bug. Fixing groupBy query to take no dictionary column as group key. Fixing BlockValueSetIterator return type issue
discard 90160c1 - THIRDEYE-254 Exposing number of reducers to be configured in startree bootstrap phase2
discard 2f46a36 THIRDEYE-244 Dimension-by-value time series should display dimension value
discard d64f19e THIRDEYE-246 Add test for non-beta HeatMap implementations
discard d332fa8 Fixed unit test after the FieldType case changes
discard 2a866ce Fixed PINOT-1376 (mismatched queries between Pinot 1 and 2) due to an exception occurring in Roaring Bitmap
discard c9ac6e3 returning specific data type based RealtimeBlockValueSets for time and metric columns to aid agggreation function computations
discard 64a95cd not explicitly converting time column type to long anymore
discard 5a68e8b Fixing broken query for realtime selection order by time/metric columns. Fixing select * query.
discard 0cb0706 making sure that we updated the maxmultivalues map
discard 2b6363e Make FieldType all upper case.
discard 507bda0 Fixing concurrency issue of adding new segment from bootstrap.
discard 9a3753d Realtime query processing. 1. add getTotalDocs into IndexSegment Interface. 2. move ResourceDataManager to get SegmentDataManager not OfflineSegmentDataManager 3. make different block implementations for selection query getRawEvent() method.
discard b4e65a3 Added exception logging for the segment REST endpoint
discard f5a13ff Fixed property store accesses failing due to the property store having an incorrect property store ZK path
discard 64a526f THIRDEYE-214 POST /collections/* should fail if resource already exists
discard 3ea4d06 THIRDEYE-214 POST /collections/* should fail if resource already exists
discard 2abdb77 THIRDEYE-214 POST /collections/* should fail if resource already exists
discard 20c28d7 THIRDEYE-214 POST /collections/* should fail if resource already exists
discard f46c670 THIRDEYE-256 StarTreeManagerImpl#remove should remove collection from openTrees
discard 19413e2 Fixed missing metadata when querying the dataresources REST endpoint with a resource name
discard 7a3d674 Bug fixing: infiniti adding realtime segment to Helix.
discard 50f31a6 Bug fixing for Realtime Indexing persistence.
discard ca81f4b THIRDEYE-268 Remove composite record store implementation
discard c792080 Fixed assertion occurring during unit test
discard 6a1cf30 THIRDEYE-258 Make ThirdEyeApplication deployable in-process
discard 13b9441 Realtime E2E test bug fixing.
discard 4a903df Make pinot-controller /dataresources/ endpoint return application/json header mimetype
discard b6837cf Fixing getEnum exception for TimeUnit field
discard 7b20d4f Fixed broken unit test
discard 1a58871 Start Zk as part of unit tests instead of relying on an instance running on localhost port 2181
discard bdc8b62 THIRDEYE-216 Server returns 500 when delete collection that does not exist
discard f5318b7 Moved temporary files used in tests to /tmp
discard 4e80cf1 Fixed helix broker starter test
discard 9f83467 Fixed integration test which broke after a move refactoring
discard 779fa16 Replaced hardcoded class names in strings with references to the actual classes, so that future refactorings don't break the tests
discard a02ffa5 THIRDEYE-252 Funnel breaks page when metric values are zero
discard 6bdea25 THIRDEYE-265 Drill down on "" value followed by any other drill down breaks UI
discard c4b57ea PINOT-1362: Fixed table names appearing as "null" in the validation metrics
discard 5a94766 Renamed integration test to OfflineClusterIntegrationTest
discard 8514e94 Adding Kafka starter
discard 50322c1 Adding missing license headers
discard f29cd77 Extracted code to start the broker, controller and server into utility classes to allow tests to use them without inheriting from ControllerTest or ClusterTest. Added code to start Zookeeper during a unit test, as not to require a local Zk instance already started up.
discard 8926e59 Reverting rebase merge commit to be at xiafu's version
discard ec21dd0 Removed an unnecessary offline suffix in the helix instance data manager after a rebase
discard 132f690 Changed test dependencies to be properly scoped to test and to depend as necessary on the other test jars
discard 49a4fab Adding missing license headers
discard 664a4fc Adding data file containing 100k rows to be reused between integration tests
discard b73c535 Updated integration test with RB feedback, fixed min, max and sum aggregation functions on filtered dictionary columns with null values
discard 319a335 Fixed integration test with latest changes
discard 1c72a06 Added integration test that creates several segments from Avro data, uploads them to the controller and queries them through the broker endpoint
discard aeea166 - THIRDEYE-236 Adding checks while reading time column in the analysis phase to make sure it is a valid timestamp
discard 6805807 Fixing mismatched querysource issue. Migrate TimeBoundaryService to use ZKMetadata.
discard 15574c7 Fixing TableNamePruner and SentinalQueryTest.
discard 8ae4b03 THIRDEYE-250 StarTreeRecordStoreFactoryFixedImpl should be more robust to file-system changes
discard 611378e Realtime/Offline Resource Bug fixing
discard e448ff2 - THIRDEYE-241 Adding a cleanup method to delete the files created due to MRUnit tests.
discard ff56915 throwing unsupported access exception, in case of getRaw access in blockDockIds that are not backed by bitmaps
discard f0ae7ca THIRDEYE-225 .funnel-quotient color should render visible on projectors
discard aaf7510 THIRDEYE-249 Add /aggregate resource to compute aggregate for metrics in time range
discard cbda62d THIRDEYE-242 Using aggregate smoothing option and funnel view breaks UI
discard 0b5c950 THIRDEYE-239 Heat map cells do not reflect aggregate smoothing option
discard 6e176a7 THIRDEYE-233 Tool-tip for volume heat map ratios should be percentage, not float
discard 68ec463 THIRDEYE-233 Tool-tip for volume heat map ratios should be percentage, not float
discard c590eb1 - THIRDEYE-243 Fixing MetricTimeSeries test
discard b087814 THIRDEYE-239 Heat map cells do not reflect aggregate smoothing option
discard cbfb2fd fixing test realtime segment data manager
discard 1b49fbf THIRDEYE-218 Call StarTreeManager#remove instead of #close in CollectionsResource#deleteCollection
discard 5c24934 Use helix cluster name, not the resource name, when getting the property store
discard cc4e2fe Adding missing license header
discard 47d0a6b Extracted common code to get the property store
discard 1b577ce Fixed NPE on server due to path changes, causing all segments to error out instead of coming online
discard cc91ea3 - THIRDEYE-106 MRUnit testcases for pre processing stages of bootstrap workflow.
discard 8766614 Refactored realtime and offline name checks into the same method, fixed setOfflineResourceZkMetadata to use the offline name instead of the undecorated name
discard f9dcd6a Fixed names that had uppercase characters in the routing table builder factory class names map
discard f0a7f3f Kafka high level consumer integration
discard 6c06e9c Adding missing license headers
discard a33a4e8 THIRDEYE-12 A user should view heat maps in order specified in StarTree by default
discard daf9077 Added a better toString for the ZK metadata
discard 822c867 Removed useless object creation
discard 475350a THIRDEYE-238 Funnel view does not reflect moving average / aggregate
discard c113f60 Make ZKMetadataUtls only depends on PropertyStore.
discard 6677628 Adding addSegment(SegmentZKMetadata); Interface to InstanceDataManager and ResourceDataManager.
discard da07b9a Move ZKMetadata related utls from HelixHelper to ZKMetadataProvider in pinot-common.
discard 5ccb295 Adding KafkaHighLevelCOnsumerBasedRoutingTableBuidler and surrondings.
discard 6d04856 THIRDEYE-226 Time mismatch on the UI
discard 096bd82 THIRDEYE-205 Baseline / current time series points have "undefined" as metricName in label
discard da1b6e9 THIRDEYE-223 Drill-down with "" dimension value breaks UI
discard d8607f1 adding licenses
discard 10013b3 adding a realtime state mode, realtime resource data manager, realtime segment data manager, realtime segments listener, nothing is tied up as of now checking this in since xiang and I need to work on same parts together at the same time and this is the foundation
discard 820cba9 Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
discard adfd8e5 pinot helix resource manager conflicts
discard 0d95085 Added logging, made most restlets return HTTP 500 on errors, improved messages sent to clients on errors.
discard 1322c6b Adding missing license headers
discard 255bbe0 Whitespace fixes: removed spaces/tabs on empty lines, removed spaces/tabs at the end of lines, replaced all tabs by two spaces
discard 3e4fc7d THIRDEYE-212 Add funnel view to dashboard
discard 1f365e1 Fix nullpo exception when timeunit is null, replace type coercion through string concatenation by the appropriate call to Long.toString
discard e3667dc THIRDEYE-213 Aggregation on dashboard double counts end point
discard 6afa8c2 Improve upload error logging by adding a message, so that the full stack trace goes into the logs
discard 6dfadb1 pinot helix resource manager conflicts
discard 81a1694 Move KafkaProperties to common
discard 83c763a Adding ZKMetadata into pinot-common as an interaction with helix. Replace many places currently using ZNRecord by *ZKMetadata.
discard 3e19617 THIRDEYE-209 Metric selection resets when heat map changes
discard fc23a2d - Introducing getRollupAggregationGranularity functionality to account of time dimension based aggregation in the RollupThresholdFunction API.
discard 7e22999 Adding license headers
discard a3a2c65 removing conflicts and fixing helix property store apis
discard 81b2a42 Fixing dependency change for BrokerRequestUtil changes.
discard adf04d6 Adding metadata utils skeleton
discard 9a372e7 Code cleanup
discard 09000ea Fixed bug where the matched resources would be a realtime resource where there is only an offline resource.
discard 2347aac Fixed a bug where columns with a cardinality of zero (all nulls) cause the number of bits needed to encode values to be Integer.MIN_VALUE, as log(0) = -Infty
discard a2fa9b2 Made validation metrics return Double.MIN_VALUE instead of (Double)Long.MIN_VALUE when the value is invalid, fixed inverted unit conversion
discard 831d4f2 Added Apache 2 license in the POM, removed checkstyle, cobertura and findbugs plugins (since we use sonar anyway) and added the maven license plugin
discard ad894ed Merge: e5bd6ca 8a4358f
discard 1925e4a Made the kafka version overridable through the command line
discard 2640143 - Throwing a relevant error message when no dimension combination can be rolled up, currently we are getting NPE
discard 4ede860 removing duplicated implimentations of BlockDocIdSet across several block implementations into a Single BlockUtils class, also removed bitmap.toArray and using bitmap.iterator instead
discard a95c195 - Making sure that the incremental updates are run against the most recent tree.bin THIRDEYE-200
discard 8380996 changes to pom files to remove slf4j errors
discard 5ed121c Merge: c4e91fe 3a6fcf4
discard 7593f2d adding npawar to acls
discard 42dd440 Move this to pinot-common, so that the code actually builds
discard 79ce523 Added metric for the offline delay in hours instead of milliseconds
discard f7edd48 Adding unit test for SearchableByteBufferUtil, added assertion in SearchableByteBufferUtil for Float.NaN and Double.NaN
discard f6333be review feedback and also making KafkaAvroDecoder extract md5 schema String from incoming events and use that to fetch schema from schema registry
discard efa636d merge conflicts
discard 04c6665 added a high level kafka consumer wrapped around the StreamProvider interface, also added a kafka decoder interface to decode incoming messages and added a schema based decoder implementatin
discard de7d4ed THIRDEYE-196 Refactor UI components, render server-side
discard 86d5c28 Adding ResourceType Offline/Realtime suffix
discard c1617df PINOT-1277 phase 1 of realtime kafka comsumption.Includes the skeleton flow
discard 4ad7eac started writing tests for comparing realtime and offline segment on the same data
discard b9e6cc3 using realtime dimensions and metrics SerDe in the RealtimeIndexSegmentImpl
discard cd98730 adding realtime dimensions and metrics SerDe
discard e9ae3fb changes based on review feedback on RB 432628 .. realtime segment iml
discard 052371f Fixing contains check for HelixExternalViewBasedTimeBoundaryService
discard 06e44f1 Adding another workflow to process federated request. After fully tested, we can merge the logic of single request to it.
discard eeb3932 Adding test for TimeBoundaryService
discard 49b5b6c Adding TimeBoundaryService interface and its simple externalViewBased implementation.
discard 853fdde THIRDEYE-203 Publish shaded jar for thirdeye-bootstrap and accept thirdeye.phase config
discard c856745 THIRDEYE-196 Refactor UI components, render server-side
discard 234fc2f THIRDEYE-196 Refactor UI components, render server-side
discard c9e27aa Made the file upload restlet return HTTP errors when an exception occurs, added unit test
discard d4e5cbb Refactored ControllerSentinelTest to extract code that could be reused to ControllerTest
discard 1e87cc0 Fixing other tests: pinot-core TestSelectionQueries.testSelectionIteration TestSelectionQueriesForMultiValueColumn.testSelectionIteration
discard e587152 Remove unused code. Fixing TestHelixBrokerStarter Remove resource version map in HelixExternalViewBasedRouting
discard 1164594 Fixing selection with no dictionary.
discard 9dbd8e0 Merge: 20772f3 3a2f7b4
discard fb991cb fixing tests in pinot-server
discard af97a59 fixing all the tests
discard 8a78081 Fixed Javadoc, added comments, removed some duplicate code, used static imports to shorten some code
discard 28d3819 Merge: 09897d3 0afcdd6
discard 883e37e renaming Builder to SchemaBuilder
discard c569e91 Adding aggregation functions for no dictionary columns
discard b8a9c3c Adding hasDictionary() method into SegmentMetadata. Default is always true.
discard cd308b5 THIRDEYE-194 Add back time series by dimension value view
discard 7cb3085 - THIRDEYE-167 : Extend AnalysisPhaseJob to compute dimension value distribution
discard bfae748 - THIRDEYE-163 : New MultipleAggregateBasedRollupFunction that handles multiple metrics
discard db5f234 THIRDEYE-190 Move logic in ThirdEyeJob#main to ThirdEyeJob#run
discard f847ea0 Merge: 77b7665 6a43682
discard e8e42c1 adding a SchemaBuilde to make schema construction easy
discard 5525428 Adding gbrandt,ajaspal to acl
discard 7f3e551 changes based on RB feedback
discard 1cc65cf adding an implementation of RealtimeSegment and changes that go along with it
discard a83da1b Remove local directory bootstrap process. Return segment reference after query processing
discard e214b67 Fixing dynamically created ResourceDataManager doesn't start.
discard 18ea2da refactor segment upload and refresh logic and adding more logs
discard 0d837e4 Added server metrics, refactored common code between broker and server metrics into AbstractMetrics
discard 905ed6f Replaced equality check with NaN (which is always false) with Double.isNaN
discard 149d1fa Replaced condition that is always true by the intended one
discard 1e2f849 Refactor some code of segment loading and query layer. Adding more logs.
discard f61ecbc Simplified IndexType code, added unit test
discard c6d9933 Renamed method that looks like a constructor to init
discard c36dde1 Fixing InstanceDataManager with QueryEngine
discard 4effd1d Make HelixInstanceDataManager to implement DataManager. Adding more log to SegmentOnlineOffline.
discard 9841600 Using modification time to replace data version when externalview updated.
discard da87241 Using modification time to replace data version when externalview updated.
discard 8b1ce27 Adding log for HelixResourceManager refresh logic.
discard 34789ef won't return segments for now.
discard db29b81 Moving classes that should not be in the main source tree to tests, removed two unused classes.
discard 8c6b051 Fixed unit tests with columns that contained null values with the recent FixedBitWidth API changes
discard 5ce7462 Added missing hashCode method for DataResource, removed a field that was compared for equality twice
discard 835f2e6 Fixing segment refresh logic to make sure idealStates changes relected on externalView.
discard 8439031 Adding returnSegmentReaders method, so we only deleted segment when no segment read happens.
discard 0ee1dcd Fixed unit tests that were using local file resources to unpack their resource from the JAR if needed
discard bf7e48a Fixed the 4 of the 5 server integration tests, stopped exception swallowing in unit tests, added utility to extract files from JARs
discard bda98ce PINOT:1271 Support negative numbers in FixedBitEncoding
discard af00c37 PINOT:1271 Support negative numbers in FixedBitEncoding
discard 250f494 PINOT:1271 Support negative numbers in FixedBitEncoding
discard b02a8be Replaced some hardcoded string constants by actual constants
discard a7bb382 Configure surefire to only use one fork, without reusing processes
discard 2745f85 Fixing broken JSON unit tests, set up Maven projects to run properly with Sonar
discard 906ec68 Fixing segment refresh won't get notified to server.
discard 1eb0189 Adding version map to routing table builder to prevent unnecessary routing table build process.
discard 3b15e83 Fixing refresh logic to destroy segment correctly.
discard e179851 fixing typo and empty space
discard 07fb38a Adding destroy method to IndexSegment interface to handle destory a segment in heap and mmap mode when we delete a segment from resourceDataManager
discard df79b03 Fixing issue with empty/null time interval/duration causing failed to load segment.
discard 2854abf Fixed unit test with incorrect maven dependencies
discard dfa5408 Fix failed tests.
discard 893c456 Add hasNullValue into column metadata
discard 6add04d getRealtimeSegment in RealtimeSegmentDataManager will send back a MutableIndexSegment
discard 92e0711 removing seal method from the RealtimeIndexSegment
discard 4b4300f third draft of realtime interfaces
discard c30dd1e second draft of realtime interfaces
discard 275c86f first draft of realtime interfaces
discard a5e8574 Adding small simple data source to fix failed unit test. Remove IndexSegmentUtils.
discard 936c3ad Bug Fix for PINOT-1223: Got exception when trying to render groupby results for empty broker response.
discard 50b1838 1. Make Server and Broker StateModel constants. 2. Remove dropServer and dropBroker functions. 3. drop instance before checking live instance.
discard 423757c Review feedback for RB-408183.
discard 31f70f9 Fixed the pinot-integration-tests maven project so that it builds and is part of the main build
discard 54d50d1 Move HelixInstanceDataManager to server module Move timeout to ServerQueryExecutorV1Imple Fix some tests. Adding Metrics constants to CommonConstants.
discard 91f1b38 Review feedback for RB-407626.
discard 3b95430 Downgrade the commons-compress version back to 1.4.1, as the other one is also broken in Artifactory.
discard e032e3f fixed commented tests in pinot-core
discard 536f881 For some reason, the commons-compress dependency is broken in our artifactory because ligradle tries to pull commons-compress-1.4.1-sources.jar which doesn't exist there. The 1.8.1 version seems to contain this.
discard efeea67 Fixed PINOT-1233, PINOT-1235
discard e8529c2 Refactored checks for brokerResource
discard 1b63801 Minor code cleanups: Removed some System.out.println calls, fixed a variable name, removed a hardcoded class name, logged an error properly and rethrew the exception instead of NPE'ing on the next line
discard d401826 Made constants final
discard 0ddb04b Bugfix for the ControllerSentinelTest, caused by serialization and deserialization of DataResource not being isomorphic (when metadata is null, it is deserialized as a NullNode rather than an ObjectNode, causing an exception when the constructor is invoked by Jackson, as NullNode isn't an ObjectNode
discard 0f492e0 Bugfix for unit tests that were not updated to create a table for their segments, causing the PinotResourceManager to reject segments for tables that did not exist
discard 6893e81 Added meters for exceptions that occur during broker processing along with the appropriate logging
discard cfb4387 Removing exception that was being thrown when we set a predicate on the DataSource
discard d329404 Changes made based on previous RB Comments, across segment creator, DataSource etc
discard 3fb786e Feedback for RB-408636. 1. Remove unused methods. 2. Use AtomicReference for state
discard 5c011be Review feedback based on RB-408710 1. Change System.out.println to log.debug 2. Delete unused method 3. Move constants and default configs to CommonConstants
discard 353216a removing unused tests and adding jfim to the owners list in the acls
discard e2b268a Made avro version configurable, can be overridden through the commandline using -Davro.version=1.4.0
discard 4552fab Skip the findbugs report for now, as it times out on while running on Hudson
discard 7cb5020 PINOT-1208 phase 1 of refactoring/code cleanup - removing dead files
discard 576d64f Ignore segments that have no time interval, avoid emitting metrics when there is no push time metadata.
discard 1661b94 Swallow and log exceptions that happen during validation, as not to prevent future executions of the validation from happening.
discard 4acbba7 Removed references to validation at file upload time
discard 43b888d Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
discard 2f8d459 Change addProperty to setProperty
discard 0a38391 Bugfix for issue introduced during refactoring
discard a5cb716 Refactored repetitive code
discard afedfdb Fixing the bug of missing time column during segment metadata persistance.
discard ec0c87a Store the segment push time and refresh time in Helix properties
discard 55e4617 Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
discard 0ec4e8b Adding custom segment name setting for segment creation.
discard 6001154 Removing useless local variable
discard 7044e92 Adding segment validation
discard 6b2e5e1 Adding static exceptions and error code to Pinot 2.0
discard 9a48e0a Added test and correction for the silly Commons Configuration API
discard 93b6a3d Added hooks to listen for new MetricsRegistries being used, so that users can add custom actions to monitor those metrics in their production systems
discard 629d436 Refactor SegmentTimeUnit to TimeUnit.
discard 8f93d08 Adding SegmentDeletionManager to handle deletes segment from helix property store and local disk. Adding more test into TestRetentionManager
discard 41f8f3e 1. Adding segmentMetadata start/end time into segmentMetadata creation, segmentMetadata loader. 2. Adding RetentionManager to scan segments and delete them based on configs.
discard 42355aa Updated metrics code to inject the BrokerMetrics dependency
discard 29bcb60 Adding broker-level metrics
discard 253ecc3 adding some rb realted helper in the README file
discard 51add84 adding branch in .reviewboard rc file
discard d38dd13 removing a duplicate copy of DataSource interface and removing all references
discard 009c6d1 adding acls
discard 5d5dd72 dummy commit
discard 19cb3b8 now acls only has tester as a whitelist user
discard 3a42107 modified .gitignore to exlcude test*.xml files and test-output, also changed .reviewboardrc to fix git review create bug, also added all the missing files
discard d7ce168 Revert "missing files between github repo added and to this repo, also everything has been copied over beyond this point and this repo should be open for business"
discard 63e97b4 missing files between github repo added and to this repo, also everything has been copied over beyond this point and this repo should be open for business
discard 0dbff10 Scatter/Gather transport layer along with Helix based routing
discard 19949d0 PINOT-1011 Core data readers and writers, all high level datasources and operators will be composed using these readers/writers
discard f1e34fa adding groups in the .reviewboardrc
discard 46b0e12 Helix server starter
discard b33e82c Helix Broker
discard d2a5117 Merge branch 'master' of gitli.corp.linkedin.com:pinot/pinot2_0
discard 6dceef6 utils, common, api, hadoop, integration tests, pom files and missing tests from core and controller
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org