You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by xi...@apache.org on 2019/11/27 23:37:00 UTC
[incubator-pinot] branch pinot-ingestion-refactor updated (5bc3ebb
-> 3ffb018)
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a change to branch pinot-ingestion-refactor
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard 5bc3ebb Update Readme to refer to docs
discard 3601d26 Adding docs for pinot spark usage
discard 6fe1c77 Add linking
discard bd69212 upgrade version
discard 99e7bc9 Adding support for retry and back-off
discard 8a6d36b update with new API
discard 0e031f4 Adding config to control spark job push parallism
discard bb900bf Fixing parent directory creation issue
discard 37c1bfd don't use default conf
discard 74cc45c fixing class: com.databricks.backend.daemon.data.client.DBFS not serializable issue
discard 2362a07 Ensure segment push job is Serializable
discard 02f1f6a Make spark job an option to parallel push segments
discard 8a0a506 Address comments
discard 10823e1 Move segment tars recursively and override existed files
discard 3fe3ee4 Initial refactor
discard e6639c4 Make temp directory with uuid appended
discard 3866151 Make PushLocation Serializable
discard baf3e38 Use SparkContext.getOrCreate() to use shared SparkContext if possible.
discard 69f68b0 Initial commit for pinot-spark
add 9881764 Introduce the base JSON config class and clean up the config classes (#4858)
add 8ba44af Fix the usage of removed guava API (#4864)
add 05c89fc Enhance TagNameUtils to handle the TagOverrideConfig (#4859)
add 03ae23c [TE] Check if snapshot exists before attaching file (#4867)
add fdfe8a2 Misc clean up for PinotHelixResourceManager (#4865)
add 8c72408 Initial commit for pinot-spark
add 2dadcd6 Use SparkContext.getOrCreate() to use shared SparkContext if possible.
add 1715be3 Make PushLocation Serializable
add 511f1a9 Make temp directory with uuid appended
add 5281119 Initial refactor
add 0152259 Move segment tars recursively and override existed files
add 78d0b5b Address comments
add fcbbd47 Make spark job an option to parallel push segments
add 32f1fa3 Ensure segment push job is Serializable
add 89c4522 fixing class: com.databricks.backend.daemon.data.client.DBFS not serializable issue
add 24d9549 don't use default conf
add 171532c Fixing parent directory creation issue
add 4fe4904 Adding config to control spark job push parallism
add 45abb1b update with new API
add 146d54a Adding support for retry and back-off
add 12f1e2c upgrade version
add e325428 Add linking
add c975433 Adding docs for pinot spark usage
add 3ffb018 Update Readme to refer to docs
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (5bc3ebb)
\
N -- N -- N refs/heads/pinot-ingestion-refactor (3ffb018)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../broker/broker/helix/HelixBrokerStarter.java | 21 +-
.../routing/builder/BaseRoutingTableBuilder.java | 5 +-
...elixExternalViewBasedQueryQuotaManagerTest.java | 31 +--
.../BalancedRandomRoutingTableBuilderTest.java | 8 +-
...rtitionAwareOfflineRoutingTableBuilderTest.java | 7 +-
...titionAwareRealtimeRoutingTableBuilderTest.java | 3 +-
.../builder/RoutingTableBuilderTestUtil.java | 44 ----
.../common/assignment/InstancePartitionsUtils.java | 11 +-
.../Segment.java => config/BaseJsonConfig.java} | 39 +--
.../pinot/common/config/ColumnPartitionConfig.java | 40 +---
.../pinot/common/config/CompletionConfig.java | 40 +---
.../apache/pinot/common/config/IndexingConfig.java | 94 +-------
.../org/apache/pinot/common/config/Instance.java | 22 +-
.../pinot/common/config/OfflineTagConfig.java | 39 ---
.../apache/pinot/common/config/QuotaConfig.java | 49 +---
.../pinot/common/config/RealtimeTagConfig.java | 64 -----
.../common/config/ReplicaGroupStrategyConfig.java | 80 ++-----
.../apache/pinot/common/config/RoutingConfig.java | 53 ++---
.../common/config/SegmentPartitionConfig.java | 69 +-----
.../SegmentsValidationAndRetentionConfig.java | 157 +++----------
.../pinot/common/config/StarTreeIndexConfig.java | 48 ++--
.../apache/pinot/common/config/TableConfig.java | 60 +----
.../pinot/common/config/TableCustomConfig.java | 76 +-----
.../pinot/common/config/TableNameBuilder.java | 22 +-
.../pinot/common/config/TableTaskConfig.java | 41 +---
.../org/apache/pinot/common/config/TagConfig.java | 43 ----
.../apache/pinot/common/config/TagNameUtils.java | 144 ++++++++----
.../pinot/common/config/TagOverrideConfig.java | 56 +----
.../org/apache/pinot/common/config/Tenant.java | 111 ++-------
.../apache/pinot/common/config/TenantConfig.java | 95 ++------
.../config/instance/InstanceAssignmentConfig.java | 46 ++--
.../instance/InstanceAssignmentConfigUtils.java | 22 +-
.../config/instance/InstanceConstraintConfig.java | 21 +-
.../InstanceReplicaGroupPartitionConfig.java | 68 ++----
.../config/instance/InstanceTagPoolConfig.java | 52 ++--
.../pinot/common/data/StarTreeIndexSpec.java | 53 +----
.../org/apache/pinot/startree/hll/HllConfig.java | 45 +---
.../pinot/common/config/TableConfigTest.java | 44 +---
.../pinot/common/config/TagOverrideConfigTest.java | 92 ++------
.../org/apache/pinot/common/config/TenantTest.java | 9 +-
.../api/resources/PinotSegmentRestletResource.java | 2 +-
.../PinotSegmentUploadDownloadRestletResource.java | 4 +-
.../helix/core/PinotHelixResourceManager.java | 261 ++++++++++-----------
.../helix/core/PinotTableIdealStateBuilder.java | 7 +-
.../instance/InstanceAssignmentDriver.java | 12 +-
.../instance/InstanceTagPoolSelector.java | 1 -
.../helix/core/minion/ClusterInfoProvider.java | 34 +--
.../core/relocation/RealtimeSegmentRelocator.java | 8 +-
.../helix/core/retention/RetentionManager.java | 2 +-
...PinotInstanceAssignmentRestletResourceTest.java | 22 +-
.../api/PinotTableRestletResourceTest.java | 71 ++----
.../pinot/controller/helix/ControllerTest.java | 18 +-
.../helix/core/PinotHelixResourceManagerTest.java | 43 ++--
.../instance/InstanceAssignmentTest.java | 121 ++++------
.../OfflineReplicaGroupSegmentAssignmentTest.java | 6 +-
.../core/rebalance/TableRebalancerClusterTest.java | 16 +-
.../converter/RealtimeSegmentConverter.java | 8 +-
.../core/transport/InstanceRequestHandler.java | 4 +-
.../ControllerPeriodicTasksIntegrationTests.java | 9 +-
...vertToRawIndexMinionClusterIntegrationTest.java | 21 +-
.../tests/SimpleMinionClusterIntegrationTest.java | 6 +-
.../tools/admin/command/AddTenantCommand.java | 7 +-
.../pinot/tools/perf/PerfBenchmarkDriver.java | 15 +-
.../notification/commons/ThirdEyeJiraClient.java | 4 +-
64 files changed, 815 insertions(+), 1911 deletions(-)
delete mode 100644 pinot-broker/src/test/java/org/apache/pinot/broker/routing/builder/RoutingTableBuilderTestUtil.java
copy pinot-common/src/main/java/org/apache/pinot/common/{data/Segment.java => config/BaseJsonConfig.java} (58%)
delete mode 100644 pinot-common/src/main/java/org/apache/pinot/common/config/OfflineTagConfig.java
delete mode 100644 pinot-common/src/main/java/org/apache/pinot/common/config/RealtimeTagConfig.java
delete mode 100644 pinot-common/src/main/java/org/apache/pinot/common/config/TagConfig.java
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org