You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by sn...@apache.org on 2022/09/07 17:39:41 UTC
[pinot] branch master updated (06b76e6dd1 -> 334c978367)
This is an automated email from the ASF dual-hosted git repository.
snlee pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pinot.git
from 06b76e6dd1 Add Spark Job Launcher tool (#9288)
add 334c978367 Enable Consistent Data Push for Standalone Segment Push Job Runners (#9295)
No new revisions were added by this update.
Summary of changes:
.../common/utils/FileUploadDownloadClient.java | 73 ++++++
.../connector/flink/sink/FlinkSegmentWriter.java | 3 +-
.../flink/sink/PinotSinkIntegrationTest.java | 4 +-
.../tests/HybridClusterIntegrationTest.java | 4 +-
.../tests/SegmentUploadIntegrationTest.java | 155 +++++++++++-
.../SegmentWriterUploaderIntegrationTest.java | 4 +-
.../batch/common/BaseSegmentPushJobRunner.java | 160 +++++++++++++
.../standalone/SegmentGenerationJobRunner.java | 8 +
.../standalone/SegmentMetadataPushJobRunner.java | 60 +----
.../batch/standalone/SegmentTarPushJobRunner.java | 63 +----
.../batch/standalone/SegmentUriPushJobRunner.java | 70 +-----
.../standalone/SegmentGenerationJobRunnerTest.java | 52 +++-
.../filebased/FileBasedSegmentWriter.java | 3 +-
.../local/utils/ConsistentDataPushUtils.java | 266 +++++++++++++++++++++
.../table/ingestion/BatchIngestionConfig.java | 21 +-
.../batch/spec/SegmentNameGeneratorSpec.java | 6 +
.../pinot/spi/utils/IngestionConfigUtils.java | 14 ++
.../utils/builder/ControllerRequestURLBuilder.java | 23 +-
18 files changed, 792 insertions(+), 197 deletions(-)
create mode 100644 pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-common/src/main/java/org/apache/pinot/plugin/ingestion/batch/common/BaseSegmentPushJobRunner.java
create mode 100644 pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/ConsistentDataPushUtils.java
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org