[incubator-pinot] branch invalid-partition-configs updated (f1995fb -> a658143)

[je...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

jenniferdai pushed a change to branch invalid-partition-configs
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.


 discard f1995fb  Ensuring partition config is valid during preprocess
     new a658143  Ensuring partition config is valid during preprocess

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (f1995fb)
            \
             N -- N -- N   refs/heads/invalid-partition-configs (a658143)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../main/java/org/apache/pinot/hadoop/job/SegmentPreprocessingJob.java   | 1 +
 1 file changed, 1 insertion(+)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[incubator-pinot] 01/01: Ensuring partition config is valid during preprocess

[je...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

jenniferdai pushed a commit to branch invalid-partition-configs
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git

commit a65814389a0c0b894dc80e5ed6d08c7f9f396619
Author: Jennifer Dai <jd...@linkedin.com>
AuthorDate: Tue Jul 23 13:29:49 2019 -0700

    Ensuring partition config is valid during preprocess
---
 .../java/org/apache/pinot/hadoop/job/SegmentPreprocessingJob.java  | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/SegmentPreprocessingJob.java b/pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/SegmentPreprocessingJob.java
index a3f3901..4a13721 100644
--- a/pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/SegmentPreprocessingJob.java
+++ b/pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/SegmentPreprocessingJob.java
@@ -41,6 +41,7 @@ import org.apache.pinot.common.config.IndexingConfig;
 import org.apache.pinot.common.config.SegmentPartitionConfig;
 import org.apache.pinot.common.config.TableConfig;
 import org.apache.pinot.common.config.TableCustomConfig;
+import org.apache.pinot.core.data.partition.PartitionFunctionFactory;
 import org.apache.pinot.hadoop.utils.PushLocation;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -286,6 +287,12 @@ public class SegmentPreprocessingJob extends BaseSegmentJob {
           .format("Partition column: %s is not found from the schema of input files.", _partitionColumn));
       Preconditions.checkArgument(_numberOfPartitions > 0, String.format("Number of partitions should be positive. Current value: %s", _numberOfPartitions));
       Preconditions.checkArgument(_partitionFunction != null, "Partition function should not be null!");
+      try {
+        PartitionFunctionFactory.PartitionFunctionType.fromString(_partitionFunction);
+      } catch (IllegalArgumentException e) {
+        _logger.info("Partition function needs to be one of Modulo, Murmur, ByteArray, HashCode");
+        throw new IllegalArgumentException(e);
+      }
     }
     if (_enableSorting) {
       Preconditions.checkArgument(_sortedColumn != null, "Sorted column should not be null!");


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org