You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by je...@apache.org on 2019/01/08 00:30:37 UTC
[incubator-pinot] branch inputformat created (now 1d39ddc)
This is an automated email from the ASF dual-hosted git repository.
jenniferdai pushed a change to branch inputformat
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
at 1d39ddc Making input format configurable
This branch includes the following new commits:
new 1d39ddc Making input format configurable
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[incubator-pinot] 01/01: Making input format configurable
Posted by je...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jenniferdai pushed a commit to branch inputformat
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit 1d39ddc710beaec358e264fe76cb84938b59eaad
Author: Jennifer Dai <jd...@linkedin.com>
AuthorDate: Mon Jan 7 16:30:22 2019 -0800
Making input format configurable
---
.../main/java/com/linkedin/pinot/hadoop/job/SegmentCreationJob.java | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/pinot-hadoop/src/main/java/com/linkedin/pinot/hadoop/job/SegmentCreationJob.java b/pinot-hadoop/src/main/java/com/linkedin/pinot/hadoop/job/SegmentCreationJob.java
index 8217a96..4224b74 100644
--- a/pinot-hadoop/src/main/java/com/linkedin/pinot/hadoop/job/SegmentCreationJob.java
+++ b/pinot-hadoop/src/main/java/com/linkedin/pinot/hadoop/job/SegmentCreationJob.java
@@ -126,6 +126,10 @@ public class SegmentCreationJob extends Configured {
}
+ public void setInputFormat(Job job) {
+ job.setInputFormatClass(TextInputFormat.class);
+ }
+
public void run() throws Exception {
LOGGER.info("Starting {}", getClass().getSimpleName());
@@ -194,7 +198,7 @@ public class SegmentCreationJob extends Configured {
job.getConfiguration().set("mapreduce.job.credentials.binary", System.getenv("HADOOP_TOKEN_FILE_LOCATION"));
}
- job.setInputFormatClass(TextInputFormat.class);
+ setInputFormat(job);
job.setOutputFormatClass(TextOutputFormat.class);
job.setMapOutputKeyClass(LongWritable.class);
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org