[nutch] branch master updated: NUTCH-2810 FreeGenerator to actually apply configured number of fetch lists

[sn...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

snagel pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/nutch.git


The following commit(s) were added to refs/heads/master by this push:
     new 46f7dc2  NUTCH-2810 FreeGenerator to actually apply configured number of fetch lists
     new cac79d3  Merge pull request #542 from sebastian-nagel/NUTCH-2810
46f7dc2 is described below

commit 46f7dc2bb2a767199be2d25355cf48e4025833a5
Author: Sebastian Nagel <sn...@apache.org>
AuthorDate: Mon Jul 27 12:05:23 2020 +0200

    NUTCH-2810 FreeGenerator to actually apply configured number of fetch lists
---
 src/java/org/apache/nutch/tools/FreeGenerator.java | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/java/org/apache/nutch/tools/FreeGenerator.java b/src/java/org/apache/nutch/tools/FreeGenerator.java
index b51be74..57344bb 100644
--- a/src/java/org/apache/nutch/tools/FreeGenerator.java
+++ b/src/java/org/apache/nutch/tools/FreeGenerator.java
@@ -208,6 +208,7 @@ public class FreeGenerator extends Configured implements Tool {
           "FreeGenerator: running in local mode, generating exactly one partition.");
       numFetchers = 1;
     }
+    job.setNumReduceTasks(numFetchers);
     job.setOutputFormatClass(SequenceFileOutputFormat.class);
     job.setOutputKeyClass(Text.class);
     job.setOutputValueClass(CrawlDatum.class);