You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by xi...@apache.org on 2019/08/22 07:27:55 UTC

[incubator-pinot] branch bugfixing_orderby updated (ceee22f -> 384cc70)

This is an automated email from the ASF dual-hosted git repository.

xiangfu pushed a change to branch bugfixing_orderby
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.


 discard ceee22f  Update selection columns logic
 discard 71b10d6  Fixing the issue of selection order by won't give correct ordering
     add 9f276fc  Fix the record reader for multi-value with null element (#4541)
     add f3893c9  [Instance Assignment] Rename instance level replica to replica-group (#4536)
     add 3d9bd65  Fixing metrics aggregation logic for Float and Double (#4537)
     add 83cc88d  Expose a method to determine if a QueryExceptionErrorCode represents a client-side error (#4532)
     add d261a76  Expose table-name via StreamConfig (#4543)
     add d62f467  [TE] Make HTTPS redirection configurable (#4539)
     add c0dbbfc  Set the consumption end time of a segment based on its creation time (#4546)
     add d3c2695  [TE] Skip creating notification task when there is no anomaly to notify (#4544)
     add cf980cb  [Instance Assignment] Plug in segment assignment for offline segment assignment and table rebalance (#4549)
     add 66d31d6  Fixing bytes data type merge logic (#4534)
     add 87a58ab  Fixing the issue of selection order by won't give correct ordering
     add 1153ed2  Update selection columns logic
     add 384cc70  Make comparison for selection order by query when order by columns are also in selection columns

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (ceee22f)
            \
             N -- N -- N   refs/heads/bugfixing_orderby (384cc70)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

No new revisions were added by this update.

Summary of changes:
 .../apache/pinot/broker/api/RequestStatistics.java |   9 +-
 .../broker/routing/RoutingTableBuilderFactory.java |   3 +-
 .../common}/assignment/InstancePartitions.java     |  68 +-
 .../InstancePartitionsType.java                    |  12 +-
 .../assignment/InstancePartitionsUtils.java        |  59 +-
 .../apache/pinot/common/config/TableConfig.java    |   2 +-
 .../apache/pinot/common/config/TagNameUtils.java   |   2 +-
 .../config/instance/InstanceAssignmentConfig.java  |  14 +-
 .../instance/InstanceAssignmentConfigUtils.java    |  25 +-
 ...va => InstanceReplicaGroupPartitionConfig.java} |  32 +-
 .../pinot/common/exception/QueryException.java     |  24 +
 .../common/restlet/resources/RebalanceResult.java  |  79 --
 .../pinot/common/utils/helix/HelixHelper.java      |  47 +-
 .../pinot/common/config/TableConfigTest.java       |  29 +-
 .../impl/kafka/KafkaLowLevelStreamConfigTest.java  |   2 +-
 .../kafka/KafkaPartitionLevelConsumerTest.java     |  14 +-
 .../kafka2/KafkaPartitionLevelConsumerTest.java    |  19 +-
 .../KafkaPartitionLevelStreamConfigTest.java       |   3 +-
 .../apache/pinot/controller/ControllerStarter.java |   9 +-
 .../PinotInstanceAssignmentRestletResource.java    |  12 +-
 .../PinotSegmentUploadRestletResource.java         |  88 ++-
 .../api/resources/PinotTableRestletResource.java   |  84 +-
 .../controller/api/upload/SegmentValidator.java    |  25 +-
 .../api/upload/SegmentValidatorResponse.java       |  55 --
 .../pinot/controller/api/upload/ZKOperator.java    |  17 +-
 .../helix/ControllerRequestURLBuilder.java         |  31 +-
 .../helix/core/PinotHelixResourceManager.java      | 171 ++--
 .../helix/core/PinotTableIdealStateBuilder.java    |   2 +-
 .../controller/helix/core/TableRebalancer.java     | 735 -----------------
 .../instance/InstanceAssignmentDriver.java         |  16 +-
 ... => InstanceReplicaGroupPartitionSelector.java} | 129 +--
 .../segment/OfflineSegmentAssignment.java          |  68 +-
 .../segment/RealtimeSegmentAssignment.java         |  55 +-
 .../core/assignment/segment/SegmentAssignment.java |   4 +-
 .../assignment/segment/SegmentAssignmentUtils.java |  35 +-
 .../realtime/PinotLLCRealtimeSegmentManager.java   |  11 +-
 .../core/realtime/PinotRealtimeSegmentManager.java |   5 +-
 .../segment/FlushThresholdUpdateManager.java       |   2 +-
 .../rebalance/DefaultRebalanceSegmentStrategy.java | 357 ---------
 ...onstants.java => RebalanceConfigConstants.java} |  26 +-
 .../helix/core/rebalance/RebalanceResult.java      |  71 ++
 .../core/rebalance/RebalanceSegmentStrategy.java   |  55 --
 .../rebalance/RebalanceSegmentStrategyFactory.java |  50 --
 .../ReplicaGroupRebalanceSegmentStrategy.java      | 532 -------------
 .../helix/core/rebalance/TableRebalancer.java      | 405 ++++++++++
 .../core/relocation/RealtimeSegmentRelocator.java  | 249 ++----
 .../BalanceNumSegmentAssignmentStrategy.java       |  98 ---
 .../core/sharding/BucketizedSegmentStrategy.java   |  66 --
 .../core/sharding/RandomAssignmentStrategy.java    |  64 --
 .../ReplicaGroupSegmentAssignmentStrategy.java     | 105 ---
 .../core/sharding/SegmentAssignmentStrategy.java   |  41 -
 .../sharding/SegmentAssignmentStrategyEnum.java    |  31 -
 .../sharding/SegmentAssignmentStrategyFactory.java |  46 --
 .../helix/core/util/ZKMetadataUtils.java           |   4 +-
 .../RealtimeSegmentValidationManager.java          |   4 +-
 ...PinotInstanceAssignmentRestletResourceTest.java |  68 +-
 .../api/PinotTableRestletResourceTest.java         |  59 +-
 .../instance/InstanceAssignmentTest.java           | 110 +--
 ...fflineNonReplicaGroupSegmentAssignmentTest.java |   4 +-
 .../OfflineReplicaGroupSegmentAssignmentTest.java  |  89 ++-
 ...altimeNonReplicaGroupSegmentAssignmentTest.java |  27 +-
 .../RealtimeReplicaGroupSegmentAssignmentTest.java |  58 +-
 .../segment/SegmentAssignmentUtilsTest.java        | 104 +--
 .../rebalance/DefaultRebalanceStrategyTest.java    | 639 ---------------
 .../ReplicaGroupRebalanceStrategyTest.java         | 318 --------
 .../core/rebalance/TableRebalancerClusterTest.java | 322 ++++++++
 .../helix/core/rebalance/TableRebalancerTest.java  | 187 +++++
 .../relocation/RealtimeSegmentRelocatorTest.java   | 292 -------
 .../sharding/SegmentAssignmentStrategyTest.java    | 330 --------
 .../helix/core/util/TableRebalancerTest.java       | 427 ----------
 .../controller/utils/ReplicaGroupTestUtils.java    | 140 ----
 .../realtime/HLRealtimeSegmentDataManager.java     |   2 +-
 .../realtime/LLRealtimeSegmentDataManager.java     |  21 +-
 .../pinot/core/data/readers/RecordReaderUtils.java |  30 +-
 .../recordtransformer/NullValueTransformer.java    |  16 +-
 .../indexsegment/mutable/MutableSegmentImpl.java   |   4 +-
 .../transform/TransformBlockDataFetcher.java       |   5 +-
 .../core/query/selection/SelectionFetcher.java     |   5 +-
 .../query/selection/SelectionOperatorUtils.java    |   9 +-
 ...ator.java => BytesSelectionColumnIterator.java} |  20 +-
 ...SelectionSingleValueColumnWithDictIterator.java |   5 -
 .../iterator/StringSelectionColumnIterator.java    |  10 +-
 .../stream/PartitionLevelStreamConfig.java         |   5 +-
 .../pinot/core/realtime/stream/StreamConfig.java   |  17 +-
 .../apache/pinot/core/util/ReplicationUtils.java   |   7 +-
 .../realtime/LLRealtimeSegmentDataManagerTest.java |   1 +
 .../core/data/readers/RecordReaderUtilsTest.java   |  36 +-
 .../MutableSegmentImplAggregateMetricsTest.java    |  18 +-
 .../impl/fakestream/FakeStreamConfigUtils.java     |   5 +-
 .../core/realtime/stream/StreamConfigTest.java     |  67 +-
 .../selection/SelectionOperatorServiceTest.java    |  46 +-
 ...umSegmentAssignmentStrategyIntegrationTest.java | 136 ----
 .../tests/ClusterIntegrationTestUtils.java         |  42 +-
 .../pinot/integration/tests/ClusterTest.java       |   2 +-
 ...eRebalancerAdminToolClusterIntegrationTest.java | 871 ---------------------
 .../apache/pinot/tools/PinotSegmentRebalancer.java | 242 ------
 .../apache/pinot/tools/PinotTableRebalancer.java   | 121 +--
 .../pinot/tools/admin/PinotAdministrator.java      |  42 +-
 .../tools/admin/command/RebalanceCommand.java      | 114 ---
 .../tools/admin/command/RebalanceTableCommand.java | 101 ++-
 .../pinot/tools/perf/PerfBenchmarkDriver.java      |   4 -
 .../app/pods/application/route.js                  |  11 +-
 thirdeye/thirdeye-frontend/config/environment.js   |   2 +
 .../detection/alert/DetectionAlertJob.java         |  30 +-
 .../pinot/thirdeye/datalayer/DaoTestUtils.java     |  40 +
 .../integration/NotificationTaskSchedulerTest.java | 183 +++++
 .../src/test/resources/sample-alert-config.yml     |  38 +
 .../src/test/resources/sample-detection-config.yml |  19 +
 108 files changed, 2573 insertions(+), 7129 deletions(-)
 rename {pinot-controller/src/main/java/org/apache/pinot/controller/helix/core => pinot-common/src/main/java/org/apache/pinot/common}/assignment/InstancePartitions.java (54%)
 rename pinot-common/src/main/java/org/apache/pinot/common/{utils => assignment}/InstancePartitionsType.java (75%)
 rename {pinot-controller/src/main/java/org/apache/pinot/controller/helix/core => pinot-common/src/main/java/org/apache/pinot/common}/assignment/InstancePartitionsUtils.java (71%)
 rename pinot-common/src/main/java/org/apache/pinot/common/config/instance/{InstanceReplicaPartitionConfig.java => InstanceReplicaGroupPartitionConfig.java} (75%)
 delete mode 100644 pinot-common/src/main/java/org/apache/pinot/common/restlet/resources/RebalanceResult.java
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/api/upload/SegmentValidatorResponse.java
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/TableRebalancer.java
 rename pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/{InstanceReplicaPartitionSelector.java => InstanceReplicaGroupPartitionSelector.java} (51%)
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/DefaultRebalanceSegmentStrategy.java
 rename pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/{RebalanceUserConfigConstants.java => RebalanceConfigConstants.java} (59%)
 create mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceResult.java
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSegmentStrategy.java
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceSegmentStrategyFactory.java
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/ReplicaGroupRebalanceSegmentStrategy.java
 create mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/sharding/BalanceNumSegmentAssignmentStrategy.java
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/sharding/BucketizedSegmentStrategy.java
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/sharding/RandomAssignmentStrategy.java
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/sharding/ReplicaGroupSegmentAssignmentStrategy.java
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/sharding/SegmentAssignmentStrategy.java
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/sharding/SegmentAssignmentStrategyEnum.java
 delete mode 100644 pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/sharding/SegmentAssignmentStrategyFactory.java
 delete mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/rebalance/DefaultRebalanceStrategyTest.java
 delete mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/rebalance/ReplicaGroupRebalanceStrategyTest.java
 create mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancerClusterTest.java
 create mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancerTest.java
 delete mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/relocation/RealtimeSegmentRelocatorTest.java
 delete mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/sharding/SegmentAssignmentStrategyTest.java
 delete mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/util/TableRebalancerTest.java
 delete mode 100644 pinot-controller/src/test/java/org/apache/pinot/controller/utils/ReplicaGroupTestUtils.java
 copy pinot-core/src/main/java/org/apache/pinot/core/query/selection/iterator/{StringSelectionColumnIterator.java => BytesSelectionColumnIterator.java} (69%)
 delete mode 100644 pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/BalanceNumSegmentAssignmentStrategyIntegrationTest.java
 delete mode 100644 pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TableRebalancerAdminToolClusterIntegrationTest.java
 delete mode 100644 pinot-tools/src/main/java/org/apache/pinot/tools/PinotSegmentRebalancer.java
 delete mode 100644 pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/RebalanceCommand.java
 create mode 100644 thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/integration/NotificationTaskSchedulerTest.java
 create mode 100644 thirdeye/thirdeye-pinot/src/test/resources/sample-alert-config.yml
 create mode 100644 thirdeye/thirdeye-pinot/src/test/resources/sample-detection-config.yml


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org