You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nutch.apache.org by sn...@apache.org on 2025/07/15 18:47:22 UTC
svn commit: r78195 [2/3] - /dev/nutch/1.21/
Added: dev/nutch/1.21/CHANGES.md
==============================================================================
--- dev/nutch/1.21/CHANGES.md (added)
+++ dev/nutch/1.21/CHANGES.md Tue Jul 15 18:47:22 2025
@@ -0,0 +1,3404 @@
+# Nutch Change Log
+
+## Nutch 1.21 Release 15/07/2025 (dd/mm/yyyy)
+Release Report: https://s.apache.org/bs58y
+
+### Breaking Changes
+
+- LinkDB records now can hold metadata, see [NUTCH-3101](https://issues.apache.org/jira/browse/NUTCH-3101).
+ This requires that existing LinkDBs are created anew starting from the segments. Nutch 1.21 cannot process LinkDBs written with prior Nutch versions.
+
+
+### New Feature
+
+- [NUTCH-2856](https://issues.apache.org/jira/browse/NUTCH-2856) - Implement a protocol-smb plugin based on hierynomus/smbj
+- [NUTCH-3063](https://issues.apache.org/jira/browse/NUTCH-3063) - Support for "addBinaryContent" from REST API
+
+### Sub-task
+
+- [NUTCH-2812](https://issues.apache.org/jira/browse/NUTCH-2812) - Methods returning array may expose internal representation
+
+### Bug
+
+- [NUTCH-3039](https://issues.apache.org/jira/browse/NUTCH-3039) - Failure to handle ftp:// URLs
+- [NUTCH-3044](https://issues.apache.org/jira/browse/NUTCH-3044) - Generator: NPE when extracting the host part of a URL fails
+- [NUTCH-3055](https://issues.apache.org/jira/browse/NUTCH-3055) - README: fix Github "hub" commands
+- [NUTCH-3057](https://issues.apache.org/jira/browse/NUTCH-3057) - Arbitrary indexer "leaks" previous value into a field processed after an exception
+- [NUTCH-3060](https://issues.apache.org/jira/browse/NUTCH-3060) - Javadoc link broken on website
+- [NUTCH-3066](https://issues.apache.org/jira/browse/NUTCH-3066) - Protocol plugin unit tests fail randomly
+- [NUTCH-3067](https://issues.apache.org/jira/browse/NUTCH-3067) - Improve performance of FetchItemQueues if error state is preserved
+- [NUTCH-3072](https://issues.apache.org/jira/browse/NUTCH-3072) - Fetcher to stop QueueFeeder if aborting with "hung threads"
+- [NUTCH-3075](https://issues.apache.org/jira/browse/NUTCH-3075) - tld plugin makes injector crash
+- [NUTCH-3078](https://issues.apache.org/jira/browse/NUTCH-3078) - Database is not unlocked when injector fails
+- [NUTCH-3079](https://issues.apache.org/jira/browse/NUTCH-3079) - Dumping a segment fails unless it has been fetched and parsed
+- [NUTCH-3087](https://issues.apache.org/jira/browse/NUTCH-3087) - Nutch crawling inconsistent on URLs with userinfo
+- [NUTCH-3092](https://issues.apache.org/jira/browse/NUTCH-3092) - Replace all imports of commons-lang by commons-lang3
+- [NUTCH-3093](https://issues.apache.org/jira/browse/NUTCH-3093) - Ant target test-plugins to depend on compile-core-test
+- [NUTCH-3094](https://issues.apache.org/jira/browse/NUTCH-3094) - Github tests to run if build configuration changes
+- [NUTCH-3096](https://issues.apache.org/jira/browse/NUTCH-3096) - HostDB ResolverThread can create too many job counters
+- [NUTCH-3097](https://issues.apache.org/jira/browse/NUTCH-3097) - Plugin indexer-elastic throws ClassNotFoundException due to invalid dependencies
+- [NUTCH-3103](https://issues.apache.org/jira/browse/NUTCH-3103) - Improper fetch interval given as example
+- [NUTCH-3106](https://issues.apache.org/jira/browse/NUTCH-3106) - Issue with SSLHandshakeException in v1.20 using protocol-http plugin and proxy
+- [NUTCH-3108](https://issues.apache.org/jira/browse/NUTCH-3108) - Fix SLF4J Class Loader Conflict in language-identifier
+- [NUTCH-3114](https://issues.apache.org/jira/browse/NUTCH-3114) - Avoid stale fetching when only URLs from queues blocked by the exponential backoff remain
+
+### Improvement
+
+- [NUTCH-1806](https://issues.apache.org/jira/browse/NUTCH-1806) - Delegate processing of URL domains to crawler commons
+- [NUTCH-2157](https://issues.apache.org/jira/browse/NUTCH-2157) - Parent Issue for Addressing Miredot REST API Warnings
+- [NUTCH-2771](https://issues.apache.org/jira/browse/NUTCH-2771) - Tests in nightly builds: speed up long runners
+- [NUTCH-2976](https://issues.apache.org/jira/browse/NUTCH-2976) - SitemapProcessor: verify sitemap values added from sitemap to CrawlDB (priority, modification time and change frequency)
+- [NUTCH-3043](https://issues.apache.org/jira/browse/NUTCH-3043) - Generator: count URLs rejected by URL filters
+- [NUTCH-3058](https://issues.apache.org/jira/browse/NUTCH-3058) - Fetcher: counter for hung threads
+- [NUTCH-3061](https://issues.apache.org/jira/browse/NUTCH-3061) - URL filters to log name of the rule file rules are read from
+- [NUTCH-3062](https://issues.apache.org/jira/browse/NUTCH-3062) - protocol-okhttp: optionally record HTTP and SSL/TLS versions
+- [NUTCH-3065](https://issues.apache.org/jira/browse/NUTCH-3065) - Format changelog as Markdown
+- [NUTCH-3073](https://issues.apache.org/jira/browse/NUTCH-3073) - Address Java compiler warnings
+- [NUTCH-3083](https://issues.apache.org/jira/browse/NUTCH-3083) - Add RobotRulesParser to bin/nutch
+- [NUTCH-3086](https://issues.apache.org/jira/browse/NUTCH-3086) - Consolidate plugin extension names and IDs
+- [NUTCH-3095](https://issues.apache.org/jira/browse/NUTCH-3095) - Update .gitignore to ignore Hadoop native libraries
+- [NUTCH-3100](https://issues.apache.org/jira/browse/NUTCH-3100) - HostDB to support minimum records per host
+- [NUTCH-3101](https://issues.apache.org/jira/browse/NUTCH-3101) - LinkDb's Inlink class to support metadata
+- [NUTCH-3112](https://issues.apache.org/jira/browse/NUTCH-3112) - Utilize parameterized logging
+- [NUTCH-3113](https://issues.apache.org/jira/browse/NUTCH-3113) - Group commands in bin/nutch command-line help
+- [NUTCH-3115](https://issues.apache.org/jira/browse/NUTCH-3115) - Allow POJO in Arbitrary Indexer to access indexing objects in filter constrctor
+- [NUTCH-3116](https://issues.apache.org/jira/browse/NUTCH-3116) - Minor dependency upgrades and update of license list and notice file
+
+### Task
+
+- [NUTCH-1942](https://issues.apache.org/jira/browse/NUTCH-1942) - Remove TopLevelDomain
+- [NUTCH-3041](https://issues.apache.org/jira/browse/NUTCH-3041) - Address confusing logging in o.a.n.net.URLExemptionFilters
+- [NUTCH-3054](https://issues.apache.org/jira/browse/NUTCH-3054) - Address deprecation of Node16 for all GitHub Actions
+- [NUTCH-3084](https://issues.apache.org/jira/browse/NUTCH-3084) - Improve CI by filtering and separating plugin and core test execution
+
+
+## Nutch 1.20 Release 09/04/2024 (dd/mm/yyyy)
+Release Report: https://s.apache.org/ovjf3
+
+### Sub-task
+
+- [NUTCH-2596](https://issues.apache.org/jira/browse/NUTCH-2596) - Upgrade from org.mortbay.jetty to org.eclipse.jetty
+- [NUTCH-2852](https://issues.apache.org/jira/browse/NUTCH-2852) - Method invokes System.exit(...) 9 bugs
+- [NUTCH-2972](https://issues.apache.org/jira/browse/NUTCH-2972) - Javadoc build fails using JDK 17
+- [NUTCH-3007](https://issues.apache.org/jira/browse/NUTCH-3007) - Fix impossible casts
+
+### Bug
+
+- [NUTCH-2634](https://issues.apache.org/jira/browse/NUTCH-2634) - Some links marked as "nofollow" are followed anyway.
+- [NUTCH-2820](https://issues.apache.org/jira/browse/NUTCH-2820) - Review sample files used in any23 unit tests
+- [NUTCH-2924](https://issues.apache.org/jira/browse/NUTCH-2924) - Generate maxCount expr evaluated only once
+- [NUTCH-2937](https://issues.apache.org/jira/browse/NUTCH-2937) - parse-tika: review dependency exclusions and avoid dependency conflicts in distributed mode
+- [NUTCH-2973](https://issues.apache.org/jira/browse/NUTCH-2973) - Single domain names (eg https://localnet) can't be crawled - filtering fails
+- [NUTCH-2974](https://issues.apache.org/jira/browse/NUTCH-2974) - Ant build fails with "Unparseable date" on certain platforms
+- [NUTCH-2979](https://issues.apache.org/jira/browse/NUTCH-2979) - Upgrade Commons Text to 1.10.0
+- [NUTCH-2982](https://issues.apache.org/jira/browse/NUTCH-2982) - Generator: parameter for URL normalization not passed forward
+- [NUTCH-2985](https://issues.apache.org/jira/browse/NUTCH-2985) - Disable plugin urlfilter-validator by default
+- [NUTCH-2992](https://issues.apache.org/jira/browse/NUTCH-2992) - Fetcher: always block fetch queues when exceptions threshold is reached
+- [NUTCH-3000](https://issues.apache.org/jira/browse/NUTCH-3000) - protocol-selenium returns only the body, strips off the <head/> element
+- [NUTCH-3001](https://issues.apache.org/jira/browse/NUTCH-3001) - protocol-selenium requires Content-Type header
+- [NUTCH-3002](https://issues.apache.org/jira/browse/NUTCH-3002) - Protocol-okhttp HttpResponse: HTTP header metadata lookup should be case-insensitive
+- [NUTCH-3008](https://issues.apache.org/jira/browse/NUTCH-3008) - indexer-elastic: downgrade to ES 7.10.2 to address licensing issues
+- [NUTCH-3012](https://issues.apache.org/jira/browse/NUTCH-3012) - SegmentReader when dumping with option -recode: NPE on unparsed documents
+- [NUTCH-3027](https://issues.apache.org/jira/browse/NUTCH-3027) - Trivial resource leak patch in DomainSuffixes.java
+- [NUTCH-3035](https://issues.apache.org/jira/browse/NUTCH-3035) - Update license and notice file for release of 1.20
+
+### New Feature
+
+- [NUTCH-2832](https://issues.apache.org/jira/browse/NUTCH-2832) - Create tutorial on sending Nutch logs to Elasticsearch
+- [NUTCH-2888](https://issues.apache.org/jira/browse/NUTCH-2888) - Selenium Protocol: Support for Selenium 4
+- [NUTCH-2920](https://issues.apache.org/jira/browse/NUTCH-2920) - Implement a indexer-opensearch plugin
+- [NUTCH-2991](https://issues.apache.org/jira/browse/NUTCH-2991) - Support HTTP/S Header Authorization for Solr connections
+- [NUTCH-3029](https://issues.apache.org/jira/browse/NUTCH-3029) - Host specific max. and min. intervals in adaptive scheduler
+
+### Improvement
+
+- [NUTCH-2853](https://issues.apache.org/jira/browse/NUTCH-2853) - bin/nutch: remove deprecated commands solrindex, solrdedup, solrclean
+- [NUTCH-2883](https://issues.apache.org/jira/browse/NUTCH-2883) - Provide means to run server as a persistent service in Docker container
+- [NUTCH-2897](https://issues.apache.org/jira/browse/NUTCH-2897) - Do not supress deprecated API warnings
+- [NUTCH-2961](https://issues.apache.org/jira/browse/NUTCH-2961) - Upgrade dependencies of parsefilter-naivebayes
+- [NUTCH-2980](https://issues.apache.org/jira/browse/NUTCH-2980) - Upgrade Selenium Java to 4.7.2
+- [NUTCH-2983](https://issues.apache.org/jira/browse/NUTCH-2983) - nutch-default.xml improvements
+- [NUTCH-2990](https://issues.apache.org/jira/browse/NUTCH-2990) - HttpRobotRulesParser to follow 5 redirects as specified by RFC 9309
+- [NUTCH-2993](https://issues.apache.org/jira/browse/NUTCH-2993) - ScoringDepth plugin to skip depth check based on URL Pattern
+- [NUTCH-2995](https://issues.apache.org/jira/browse/NUTCH-2995) - Upgrade to crawler-commons 1.4
+- [NUTCH-2996](https://issues.apache.org/jira/browse/NUTCH-2996) - Use new SimpleRobotRulesParser API entry point (crawler-commons 1.4)
+- [NUTCH-2997](https://issues.apache.org/jira/browse/NUTCH-2997) - Add Override annotations where applicable
+- [NUTCH-3004](https://issues.apache.org/jira/browse/NUTCH-3004) - Avoid NPE in HttpResponse
+- [NUTCH-3005](https://issues.apache.org/jira/browse/NUTCH-3005) - Upgrade selenium as needed
+- [NUTCH-3009](https://issues.apache.org/jira/browse/NUTCH-3009) - Upgrade to Hadoop 3.3.6
+- [NUTCH-3010](https://issues.apache.org/jira/browse/NUTCH-3010) - Injector: count unique number of injected URLs
+- [NUTCH-3011](https://issues.apache.org/jira/browse/NUTCH-3011) - HttpRobotRulesParser: handle HTTP 429 Too Many Requests same as server errors (HTTP 5xx)
+- [NUTCH-3013](https://issues.apache.org/jira/browse/NUTCH-3013) - Employ commons-lang3's StopWatch to simplify timing logic
+- [NUTCH-3014](https://issues.apache.org/jira/browse/NUTCH-3014) - Standardize Job names
+- [NUTCH-3015](https://issues.apache.org/jira/browse/NUTCH-3015) - Add more CI steps to GitHub master-build.yml
+- [NUTCH-3017](https://issues.apache.org/jira/browse/NUTCH-3017) - Allow fast-urlfilter to load from HDFS/S3 and support gzipped input
+- [NUTCH-3025](https://issues.apache.org/jira/browse/NUTCH-3025) - urlfilter-fast to filter based on the length of the URL
+- [NUTCH-3031](https://issues.apache.org/jira/browse/NUTCH-3031) - ProtocolFactory host mapper to support domains
+- [NUTCH-3032](https://issues.apache.org/jira/browse/NUTCH-3032) - Indexing plugin as an adapter for end user's own POJO instances
+- [NUTCH-3036](https://issues.apache.org/jira/browse/NUTCH-3036) - Upgrade org.seleniumhq.selenium:selenium-java dependency in lib-selenium
+
+### Task
+
+- [NUTCH-2959](https://issues.apache.org/jira/browse/NUTCH-2959) - Upgrade to Apache Tika 2.9.0
+- [NUTCH-2977](https://issues.apache.org/jira/browse/NUTCH-2977) - Support for showing dependency tree
+- [NUTCH-2978](https://issues.apache.org/jira/browse/NUTCH-2978) - Move to slf4j2 and remove log4j1 and reload4j
+- [NUTCH-2984](https://issues.apache.org/jira/browse/NUTCH-2984) - Drop test proxy server and benchmark tool
+- [NUTCH-2989](https://issues.apache.org/jira/browse/NUTCH-2989) - Can't have username/pw AND https on elastic-indexer?!
+- [NUTCH-2998](https://issues.apache.org/jira/browse/NUTCH-2998) - Remove the Any23 plugin
+- [NUTCH-2999](https://issues.apache.org/jira/browse/NUTCH-2999) - Update Lucene version to latest 8.x
+- [NUTCH-3016](https://issues.apache.org/jira/browse/NUTCH-3016) - Upgrade Apache Ivy to 2.5.2
+- [NUTCH-3019](https://issues.apache.org/jira/browse/NUTCH-3019) - Upgrade to Apache Tika 2.9.1
+- [NUTCH-3020](https://issues.apache.org/jira/browse/NUTCH-3020) - ParseSegment should check for protocol's flags for truncation
+- [NUTCH-3024](https://issues.apache.org/jira/browse/NUTCH-3024) - Remove flaky 'dependency check' target
+- [NUTCH-3033](https://issues.apache.org/jira/browse/NUTCH-3033) - Upgrade Ivy to v2.5.2
+- [NUTCH-3037](https://issues.apache.org/jira/browse/NUTCH-3037) - Upgrade org.apache.kafka:kafka_2.12: to v3.7.0
+- [NUTCH-3038](https://issues.apache.org/jira/browse/NUTCH-3038) - Address issues discovered during 1.20 release management dryrun
+
+
+## Nutch 1.19 Release 22/08/2022 (dd/mm/yyyy)
+Release Report: https://s.apache.org/lf6li
+
+### Breaking Changes
+
+- Nutch is built on JDK 11 (NUTCH-2857)
+- the Nutch WebApp was moved to a separate repository (NUTCH-2886)
+ - see https://github.com/apache/nutch-webapp
+ - or https://gitbox.apache.org/repos/asf?p=nutch-webapp.git
+- the plugin parse-swf for parsing Shockwave/Adobe Flash content was removed (NUTCH-2861)
+
+### Sub-task
+
+- [NUTCH-2819](https://issues.apache.org/jira/browse/NUTCH-2819) - Move spotbugs "installation" directory to avoid that spotbugs is shipped in Nutch runtime
+- [NUTCH-2846](https://issues.apache.org/jira/browse/NUTCH-2846) - Fix various bugs spotted by NUTCH-2815
+- [NUTCH-2850](https://issues.apache.org/jira/browse/NUTCH-2850) - Method ignores exceptional return value
+- [NUTCH-2851](https://issues.apache.org/jira/browse/NUTCH-2851) - Random object created and used only once
+- [NUTCH-2855](https://issues.apache.org/jira/browse/NUTCH-2855) - Update org.elasticsearch.client
+
+### Bug
+
+- [NUTCH-2290](https://issues.apache.org/jira/browse/NUTCH-2290) - Update licenses of bundled libraries
+- [NUTCH-2512](https://issues.apache.org/jira/browse/NUTCH-2512) - Nutch does not build under JDK9
+- [NUTCH-2821](https://issues.apache.org/jira/browse/NUTCH-2821) - Deduplicate licenses in LICENSE.txt file
+- [NUTCH-2822](https://issues.apache.org/jira/browse/NUTCH-2822) - Split the LICENSE.txt file into two files for source resp. binary releases
+- [NUTCH-2831](https://issues.apache.org/jira/browse/NUTCH-2831) - Elastic indexer does not support SSL
+- [NUTCH-2843](https://issues.apache.org/jira/browse/NUTCH-2843) - Duplicate declaration of dependencies in ivy.xml
+- [NUTCH-2858](https://issues.apache.org/jira/browse/NUTCH-2858) - urlnormalizer-protocol: URL port is lost during normalization
+- [NUTCH-2862](https://issues.apache.org/jira/browse/NUTCH-2862) - Do not include Ivy jar in source release package
+- [NUTCH-2863](https://issues.apache.org/jira/browse/NUTCH-2863) - Injector to parse command-line flags case-insensitive
+- [NUTCH-2866](https://issues.apache.org/jira/browse/NUTCH-2866) - MetaData.toString() should return "key=value ..."
+- [NUTCH-2868](https://issues.apache.org/jira/browse/NUTCH-2868) - urlnormalizer-protocol fails with StringIndexOutOfBoundsException when reading invalid line in configuration file
+- [NUTCH-2881](https://issues.apache.org/jira/browse/NUTCH-2881) - bug in 'nutch' symlink in docker container
+- [NUTCH-2889](https://issues.apache.org/jira/browse/NUTCH-2889) - nutch indexer-elasticsearch plugin, doesn't work with https protocol
+- [NUTCH-2890](https://issues.apache.org/jira/browse/NUTCH-2890) - Protocol-okhttp: upgrade okhttp to 4.9.1 to address infinite connection retries
+- [NUTCH-2894](https://issues.apache.org/jira/browse/NUTCH-2894) - Java plugin compilation classpath: priorize plugin dependencies
+- [NUTCH-2899](https://issues.apache.org/jira/browse/NUTCH-2899) - Remove needless warning about missing o/a/rat/anttasks/antlib.xml
+- [NUTCH-2902](https://issues.apache.org/jira/browse/NUTCH-2902) - Jexl parsing error on statements
+- [NUTCH-2905](https://issues.apache.org/jira/browse/NUTCH-2905) - Mask sensitive strings in log output of index writers
+- [NUTCH-2910](https://issues.apache.org/jira/browse/NUTCH-2910) - FetchItemQueues overloaded constructor also interprets fetcher timeout as -1 e.g. no-timeout.
+- [NUTCH-2915](https://issues.apache.org/jira/browse/NUTCH-2915) - Upgrade to log4j 2.15.0
+- [NUTCH-2916](https://issues.apache.org/jira/browse/NUTCH-2916) - Fix log file rotation / rename default log file
+- [NUTCH-2917](https://issues.apache.org/jira/browse/NUTCH-2917) - Remove transitive dependency to log4j 1.x
+- [NUTCH-2922](https://issues.apache.org/jira/browse/NUTCH-2922) - Upgrade to log4j 2.17.0
+- [NUTCH-2935](https://issues.apache.org/jira/browse/NUTCH-2935) - DeduplicationJob: failure on URLs with invalid percent encoding
+- [NUTCH-2936](https://issues.apache.org/jira/browse/NUTCH-2936) - Early registration of URL stream handlers provided by plugins may fail Hadoop jobs running in distributed mode if protocol-okhttp is used
+- [NUTCH-2945](https://issues.apache.org/jira/browse/NUTCH-2945) - Solr Index Writer pluging schema.xml missing a copyToField
+- [NUTCH-2947](https://issues.apache.org/jira/browse/NUTCH-2947) - Fetcher: keep state of empty fetch queues unless queue feeder is finished
+- [NUTCH-2949](https://issues.apache.org/jira/browse/NUTCH-2949) - Tasks of a multi-threaded map runner may fail because of slow creation of URL stream handlers
+- [NUTCH-2951](https://issues.apache.org/jira/browse/NUTCH-2951) - Crawl datum with metadata WRITABLE_GENERATE_TIME_KEY awaits fetching forever
+- [NUTCH-2955](https://issues.apache.org/jira/browse/NUTCH-2955) - indexer-solr: replace deprecated/removed field type solr.LatLonType
+- [NUTCH-2969](https://issues.apache.org/jira/browse/NUTCH-2969) - Javadoc: Javascript search is not working when built on JDK 11
+
+### New Feature
+
+- [NUTCH-2901](https://issues.apache.org/jira/browse/NUTCH-2901) - migrate to maven or gradle
+
+### Improvement
+
+- [NUTCH-1403](https://issues.apache.org/jira/browse/NUTCH-1403) - Add default ScoringFilter for manipulating metadata
+- [NUTCH-2429](https://issues.apache.org/jira/browse/NUTCH-2429) - Fix Plugin System to allow protocol plugins to bundle their URLStreamHandlers
+- [NUTCH-2449](https://issues.apache.org/jira/browse/NUTCH-2449) - Usage of Tika LanguageIdentifier in language-identifier plugin
+- [NUTCH-2573](https://issues.apache.org/jira/browse/NUTCH-2573) - Suspend crawling if robots.txt fails to fetch with 5xx status
+- [NUTCH-2795](https://issues.apache.org/jira/browse/NUTCH-2795) - CrawlDbReader: compress CrawlDb dumps if configured
+- [NUTCH-2807](https://issues.apache.org/jira/browse/NUTCH-2807) - SitemapProcessor to warn that ignoring robots.txt affects detection of sitemaps
+- [NUTCH-2808](https://issues.apache.org/jira/browse/NUTCH-2808) - Document side effects of ignoring robots.txt
+- [NUTCH-2840](https://issues.apache.org/jira/browse/NUTCH-2840) - Fix 'report-vulnerabilities' ant target in build.xml
+- [NUTCH-2842](https://issues.apache.org/jira/browse/NUTCH-2842) - Fix Javadoc warnings, errors and add Javadoc check to Github Action and Jenkins
+- [NUTCH-2845](https://issues.apache.org/jira/browse/NUTCH-2845) - Update urlfilter-suffix rules
+- [NUTCH-2847](https://issues.apache.org/jira/browse/NUTCH-2847) - HttpDateFormat: Simplify based on new Java 8 DateTime API
+- [NUTCH-2849](https://issues.apache.org/jira/browse/NUTCH-2849) - Replace remaining package.html files with package-info.java
+- [NUTCH-2857](https://issues.apache.org/jira/browse/NUTCH-2857) - Upgrade from JDK1.8 --> JDK11
+- [NUTCH-2859](https://issues.apache.org/jira/browse/NUTCH-2859) - urlnormalizer-protocol: allow to normalize domains
+- [NUTCH-2861](https://issues.apache.org/jira/browse/NUTCH-2861) - Remove parse-swf
+- [NUTCH-2864](https://issues.apache.org/jira/browse/NUTCH-2864) - Upgrade Dockerfile to use JDK 11
+- [NUTCH-2865](https://issues.apache.org/jira/browse/NUTCH-2865) - WARC exporter support for metadata and dropping empty responses
+- [NUTCH-2867](https://issues.apache.org/jira/browse/NUTCH-2867) - Support for custom HostDb aggregators
+- [NUTCH-2869](https://issues.apache.org/jira/browse/NUTCH-2869) - Add @Override annotations to Nutch plugins
+- [NUTCH-2879](https://issues.apache.org/jira/browse/NUTCH-2879) - fireant upgrade dependency hadoop-hdfs in ivy/ivy.xml from 3.1.3 to 3.3.1
+- [NUTCH-2882](https://issues.apache.org/jira/browse/NUTCH-2882) - Configure NutchUiServer for DEPLOYMENT and improve logging
+- [NUTCH-2885](https://issues.apache.org/jira/browse/NUTCH-2885) - Upgrade to Log4j2
+- [NUTCH-2886](https://issues.apache.org/jira/browse/NUTCH-2886) - Move Nutch WebApp to separate repository
+- [NUTCH-2891](https://issues.apache.org/jira/browse/NUTCH-2891) - Upgrade to Tika 2.1
+- [NUTCH-2892](https://issues.apache.org/jira/browse/NUTCH-2892) - Upgrade to Any23 2.5
+- [NUTCH-2893](https://issues.apache.org/jira/browse/NUTCH-2893) - fireant upgrade dependency elasticsearch-rest-high-level-client in src/plugin/indexer-elastic/ivy.xml from 7.11.1 to 7.13.2
+- [NUTCH-2896](https://issues.apache.org/jira/browse/NUTCH-2896) - Protocol-okhttp: make connection pool configurable
+- [NUTCH-2898](https://issues.apache.org/jira/browse/NUTCH-2898) - IDE Setup for nutch with Intellij IDEA is not well documented
+- [NUTCH-2903](https://issues.apache.org/jira/browse/NUTCH-2903) - Unable to Connect to Elasticsearch over HTTPS
+- [NUTCH-2904](https://issues.apache.org/jira/browse/NUTCH-2904) - Upgrade to crawler-commons 1.2
+- [NUTCH-2908](https://issues.apache.org/jira/browse/NUTCH-2908) - Log mapreduce job messages and counters in local mode
+- [NUTCH-2911](https://issues.apache.org/jira/browse/NUTCH-2911) - Add cleanup call in Fetcher.java
+- [NUTCH-2914](https://issues.apache.org/jira/browse/NUTCH-2914) - nutch-default.xml: remove obsolete and unused properties
+- [NUTCH-2918](https://issues.apache.org/jira/browse/NUTCH-2918) - Upgrade to log4j 2.16.0
+- [NUTCH-2919](https://issues.apache.org/jira/browse/NUTCH-2919) - Upgrade to Tika 2.2.1 and Any23 2.6
+- [NUTCH-2923](https://issues.apache.org/jira/browse/NUTCH-2923) - Add Job Id in Job Failure messages
+- [NUTCH-2929](https://issues.apache.org/jira/browse/NUTCH-2929) - Fetcher: start threads slowly to avoid that resources are temporarily exhausted
+- [NUTCH-2930](https://issues.apache.org/jira/browse/NUTCH-2930) - Protocol-okhttp: implement IP filter
+- [NUTCH-2946](https://issues.apache.org/jira/browse/NUTCH-2946) - Fetcher: optionally slow down fetching from hosts with repeated exceptions
+- [NUTCH-2948](https://issues.apache.org/jira/browse/NUTCH-2948) - Upgrade dependencies to Any23 2.7 and Tika 2.3.0
+- [NUTCH-2950](https://issues.apache.org/jira/browse/NUTCH-2950) - UpdateHostDb: performance improvements
+- [NUTCH-2952](https://issues.apache.org/jira/browse/NUTCH-2952) - Upgrade core dependencies (Hadoop 3.3.3, log4j 2.17.2)
+- [NUTCH-2953](https://issues.apache.org/jira/browse/NUTCH-2953) - Indexer Elastic to ignore SSL issues
+- [NUTCH-2956](https://issues.apache.org/jira/browse/NUTCH-2956) - index-geoip: dependency upgrades and improvements
+- [NUTCH-2957](https://issues.apache.org/jira/browse/NUTCH-2957) - indexer-solr / Solr schema: add fall-back field definitions for unknown index fields
+- [NUTCH-2958](https://issues.apache.org/jira/browse/NUTCH-2958) - Upgrade to crawler-commons 1.3
+- [NUTCH-2962](https://issues.apache.org/jira/browse/NUTCH-2962) - Update and complete package info of protocol plugins
+- [NUTCH-2963](https://issues.apache.org/jira/browse/NUTCH-2963) - Upgrade dependencies before release of 1.19
+
+### Task
+
+- [NUTCH-2826](https://issues.apache.org/jira/browse/NUTCH-2826) - Migrate Nutch Site from Apache CMS to Hugo
+- [NUTCH-2870](https://issues.apache.org/jira/browse/NUTCH-2870) - fireant upgrade dependency junit in ivy/ivy.xml from 4.13.1 to 4.13.2
+
+
+## Nutch 1.18 Release 14/01/2021 (dd/mm/yyyy)
+Release Report: https://s.apache.org/lqara
+
+### Breaking Changes
+
+- As part of NUTCH-2805, the plugin urlfilter-domainblacklist has been renamed to urlfilter-domaindenylist. And the fields required for the plugin urlfilter.domainblacklist.rules and urlfilter.domainblacklist.file has been replaced with urlfilter.domaindenylist.rules and urlfilter.domaindenylist.file respectively. See [NUTCH-2802](https://issues.apache.org/jira/browse/NUTCH-2802) for more details.
+
+### Sub-task
+
+- [NUTCH-2671](https://issues.apache.org/jira/browse/NUTCH-2671) - Upgrade ant ivy library
+- [NUTCH-2672](https://issues.apache.org/jira/browse/NUTCH-2672) - Ant build erronously installs *-test.jar instead *.jar for target "nightly"
+- [NUTCH-2805](https://issues.apache.org/jira/browse/NUTCH-2805) - Rename plugin urlfilter-domainblacklist
+- [NUTCH-2809](https://issues.apache.org/jira/browse/NUTCH-2809) - Upgrade any23 plugin dependency to 2.4
+- [NUTCH-2816](https://issues.apache.org/jira/browse/NUTCH-2816) - Add Spotbugs target to ant build
+- [NUTCH-2817](https://issues.apache.org/jira/browse/NUTCH-2817) - Avoid check for equality of URL path and file part using ==/!=
+- [NUTCH-2829](https://issues.apache.org/jira/browse/NUTCH-2829) - Fix ant target "clean-cache"
+
+### Bug
+
+- [NUTCH-2669](https://issues.apache.org/jira/browse/NUTCH-2669) - Reliable solution for javax.ws packaging.type
+- [NUTCH-2697](https://issues.apache.org/jira/browse/NUTCH-2697) - Upgrade Ivy to fix the issue of an unset packaging.type property
+- [NUTCH-2801](https://issues.apache.org/jira/browse/NUTCH-2801) - RobotsRulesParser command-line checker to use http.robots.agents as fall-back
+- [NUTCH-2810](https://issues.apache.org/jira/browse/NUTCH-2810) - FreeGenerator to actually apply configured number of fetch lists
+- [NUTCH-2813](https://issues.apache.org/jira/browse/NUTCH-2813) - MoreIndexingFilter - can't parse erroneous date - 2019-07-03T10:28:14
+- [NUTCH-2814](https://issues.apache.org/jira/browse/NUTCH-2814) - HttpDateFormat's internal time zone may change after parsing a date
+- [NUTCH-2818](https://issues.apache.org/jira/browse/NUTCH-2818) - Ant build: upgrade Apache Rat report task
+- [NUTCH-2823](https://issues.apache.org/jira/browse/NUTCH-2823) - IllegalStateException in IndexWriters.describe() when validating url param for SolrIndexer
+- [NUTCH-2824](https://issues.apache.org/jira/browse/NUTCH-2824) - urlnormalizer-basic to unescape percent-encoded host names
+
+### Improvement
+
+- [NUTCH-1190](https://issues.apache.org/jira/browse/NUTCH-1190) - MoreIndexingFilter refactor: move data formats used to parse "lastModified" to a config file.
+- [NUTCH-2582](https://issues.apache.org/jira/browse/NUTCH-2582) - Set pool size of XML SAX parsers used for MIME detection in Tika 1.19
+- [NUTCH-2730](https://issues.apache.org/jira/browse/NUTCH-2730) - SitemapProcessor to treat sitemap URLs as Set instead of List
+- [NUTCH-2782](https://issues.apache.org/jira/browse/NUTCH-2782) - protocol-http / lib-http: support TLSv1.3
+- [NUTCH-2796](https://issues.apache.org/jira/browse/NUTCH-2796) - Upgrade to crawler-commons 1.1
+- [NUTCH-2799](https://issues.apache.org/jira/browse/NUTCH-2799) - Add .asf.yaml file
+- [NUTCH-2833](https://issues.apache.org/jira/browse/NUTCH-2833) - Upgrade to Tika 1.25
+- [NUTCH-2835](https://issues.apache.org/jira/browse/NUTCH-2835) - Upgrade commons-jexl from 2 --> 3
+- [NUTCH-2836](https://issues.apache.org/jira/browse/NUTCH-2836) - Upgrade various commons dependencies
+- [NUTCH-2837](https://issues.apache.org/jira/browse/NUTCH-2837) - Update multiple dependencies
+- [NUTCH-2841](https://issues.apache.org/jira/browse/NUTCH-2841) - Upgrade xercesImpl dependency
+
+### Wish
+
+- [NUTCH-2834](https://issues.apache.org/jira/browse/NUTCH-2834) - Deduplication mode via command line in crawl script
+
+### Task
+
+- [NUTCH-2830](https://issues.apache.org/jira/browse/NUTCH-2830) - Upgrade any23 to v2.4
+
+
+## Nutch 1.17 Release 18/06/2020 (dd/mm/yyyy)
+Release Report: https://s.apache.org/ovhry
+
+### Bug
+
+- [NUTCH-1559](https://issues.apache.org/jira/browse/NUTCH-1559) - parse-metatags duplicates extracted metatags
+- [NUTCH-2379](https://issues.apache.org/jira/browse/NUTCH-2379) - crawl script dedup's crawldb update is slow
+- [NUTCH-2419](https://issues.apache.org/jira/browse/NUTCH-2419) - Some URL filters and normalizers do not respect command-line override for rule file
+- [NUTCH-2507](https://issues.apache.org/jira/browse/NUTCH-2507) - NutchTutorial wiki pages as a lot of outdated command line calls when it starts with the solr interaction
+- [NUTCH-2511](https://issues.apache.org/jira/browse/NUTCH-2511) - SitemapProcessor limited by http.content.limit
+- [NUTCH-2525](https://issues.apache.org/jira/browse/NUTCH-2525) - Metadata indexer cannot handle uppercase parse metadata
+- [NUTCH-2567](https://issues.apache.org/jira/browse/NUTCH-2567) - parse-metatags writes all meta tags twice
+- [NUTCH-2720](https://issues.apache.org/jira/browse/NUTCH-2720) - ROBOTS metatag ignored when capitalized
+- [NUTCH-2745](https://issues.apache.org/jira/browse/NUTCH-2745) - Solr schema.xml not shipped in binary release
+- [NUTCH-2748](https://issues.apache.org/jira/browse/NUTCH-2748) - Fetch status gone (redirect exceeded) not to overwrite existing items in CrawlDb
+- [NUTCH-2751](https://issues.apache.org/jira/browse/NUTCH-2751) - nutch clean does not work with secured solr cloud
+- [NUTCH-2753](https://issues.apache.org/jira/browse/NUTCH-2753) - Add -listen option to command-line help of CrawlDbReader and LinkDbReader
+- [NUTCH-2754](https://issues.apache.org/jira/browse/NUTCH-2754) - fetcher.max.crawl.delay ignored if exceeding 5 min. / 300 sec.
+- [NUTCH-2760](https://issues.apache.org/jira/browse/NUTCH-2760) - protocol-okhttp: properly record HTTP version in request message header
+- [NUTCH-2761](https://issues.apache.org/jira/browse/NUTCH-2761) - ivy jar fails to download
+- [NUTCH-2763](https://issues.apache.org/jira/browse/NUTCH-2763) - protocol-okhttp (store.http.headers): add whitespace in status line after status code also when message is empty
+- [NUTCH-2770](https://issues.apache.org/jira/browse/NUTCH-2770) - Subcollection logic allows empty string as a whitelist value, thus matching every incoming document.
+- [NUTCH-2778](https://issues.apache.org/jira/browse/NUTCH-2778) - indexer-elastic to properly log errors
+- [NUTCH-2787](https://issues.apache.org/jira/browse/NUTCH-2787) - CrawlDb JSON dump does not export metadata primitive data types correctly
+- [NUTCH-2789](https://issues.apache.org/jira/browse/NUTCH-2789) - Documentation: update links to point to cwiki
+- [NUTCH-2790](https://issues.apache.org/jira/browse/NUTCH-2790) - CSVIndexWriter does not escape leading quotes properly
+- [NUTCH-2791](https://issues.apache.org/jira/browse/NUTCH-2791) - domainstats, protocolstats and crawlcomplete do not handle GCS URLs
+
+### New Feature
+
+- [NUTCH-1863](https://issues.apache.org/jira/browse/NUTCH-1863) - Add JSON format dump output to readdb command
+
+### Improvement
+
+- [NUTCH-1194](https://issues.apache.org/jira/browse/NUTCH-1194) - Generator: CrawlDB lock should be released earlier
+- [NUTCH-2002](https://issues.apache.org/jira/browse/NUTCH-2002) - ParserChecker and IndexingFiltersChecker to check robots.txt
+- [NUTCH-2184](https://issues.apache.org/jira/browse/NUTCH-2184) - Enable IndexingJob to function with no crawldb
+- [NUTCH-2495](https://issues.apache.org/jira/browse/NUTCH-2495) - Use -deleteGone instead of clean job in crawler script while indexing
+- [NUTCH-2496](https://issues.apache.org/jira/browse/NUTCH-2496) - Speed up link inversion step in crawling script
+- [NUTCH-2501](https://issues.apache.org/jira/browse/NUTCH-2501) - allow to set Java heap size when using crawl script in distributed mode
+- [NUTCH-2649](https://issues.apache.org/jira/browse/NUTCH-2649) - Optionally skip TLS/SSL certificate validation for protocol-selenium and protocol-htmlunit
+- [NUTCH-2733](https://issues.apache.org/jira/browse/NUTCH-2733) - protocol-okhttp: add support for Brotli compression (Content-Encoding)
+- [NUTCH-2739](https://issues.apache.org/jira/browse/NUTCH-2739) - indexer-elastic: Upgrade ES and migrate to REST client
+- [NUTCH-2743](https://issues.apache.org/jira/browse/NUTCH-2743) - Add list of Nutch properties (nutch-default.xml) to documentation
+- [NUTCH-2746](https://issues.apache.org/jira/browse/NUTCH-2746) - Basic URL normalizer to normalize Unicode domain names
+- [NUTCH-2747](https://issues.apache.org/jira/browse/NUTCH-2747) - Replace remaining o.a.commons.logging by org.slf4j
+- [NUTCH-2750](https://issues.apache.org/jira/browse/NUTCH-2750) - Improve CrawlDbReader & LinkDbReader reader handling
+- [NUTCH-2752](https://issues.apache.org/jira/browse/NUTCH-2752) - indexer-solr: Upgrade to latest Solr version
+- [NUTCH-2755](https://issues.apache.org/jira/browse/NUTCH-2755) - Remove obsolete plugin indexer-elastic-rest
+- [NUTCH-2757](https://issues.apache.org/jira/browse/NUTCH-2757) - indexer-elastic: add authentication options
+- [NUTCH-2758](https://issues.apache.org/jira/browse/NUTCH-2758) - Add plugin READMEs to binary release packages
+- [NUTCH-2759](https://issues.apache.org/jira/browse/NUTCH-2759) - bin/crawl: Rename option --num-slaves
+- [NUTCH-2762](https://issues.apache.org/jira/browse/NUTCH-2762) - Replace http:// URLs by https:// (build files and documentation)
+- [NUTCH-2767](https://issues.apache.org/jira/browse/NUTCH-2767) - Fetcher to stop filling queues skipped due to repeated exceptions
+- [NUTCH-2768](https://issues.apache.org/jira/browse/NUTCH-2768) - FetcherThread: unnecessary usage of class casts
+- [NUTCH-2772](https://issues.apache.org/jira/browse/NUTCH-2772) - Debugging parse filter to show serialized DOM tree
+- [NUTCH-2773](https://issues.apache.org/jira/browse/NUTCH-2773) - SegmentReader (-dump or -get): show HTML content as UTF-8
+- [NUTCH-2774](https://issues.apache.org/jira/browse/NUTCH-2774) - Annotate methods implementing the Hadoop API by @Override
+- [NUTCH-2775](https://issues.apache.org/jira/browse/NUTCH-2775) - Fetcher to guarantee minimum delay even if robots.txt defines shorter Crawl-delay
+- [NUTCH-2776](https://issues.apache.org/jira/browse/NUTCH-2776) - Fetcher to temporarily deduplicate followed redirects
+- [NUTCH-2777](https://issues.apache.org/jira/browse/NUTCH-2777) - Upgrade to Hadoop 3.1
+- [NUTCH-2779](https://issues.apache.org/jira/browse/NUTCH-2779) - Upgrade to Tika 1.24.1
+- [NUTCH-2780](https://issues.apache.org/jira/browse/NUTCH-2780) - Upgrade index-solr to use Solr 8.5.1
+- [NUTCH-2781](https://issues.apache.org/jira/browse/NUTCH-2781) - Increase default Java heap size
+- [NUTCH-2783](https://issues.apache.org/jira/browse/NUTCH-2783) - Use (more) parametrized logging
+- [NUTCH-2784](https://issues.apache.org/jira/browse/NUTCH-2784) - Add tool to list Nutch and Hadoop properties
+- [NUTCH-2785](https://issues.apache.org/jira/browse/NUTCH-2785) - FreeGenerator: command-line option to define number of generated fetch lists
+- [NUTCH-2788](https://issues.apache.org/jira/browse/NUTCH-2788) - ParseData: improve presentation of Metadata in method toString()
+- [NUTCH-2794](https://issues.apache.org/jira/browse/NUTCH-2794) - Add additional ciphers to HTTP base's default cipher suite
+
+### Test
+
+- [NUTCH-1945](https://issues.apache.org/jira/browse/NUTCH-1945) - Test for XLSX parser
+
+### Task
+
+- [NUTCH-2434](https://issues.apache.org/jira/browse/NUTCH-2434) - Add methods to reset parameters HTMLMetaTags
+
+### Sub-task
+
+- [NUTCH-2735](https://issues.apache.org/jira/browse/NUTCH-2735) - Update the indexer-solr documentation about the schema.xml usage
+
+
+## Nutch 1.16 Release 02/10/2019 (dd/mm/yyyy)
+Release Report: https://s.apache.org/l2j94
+
+### Comments
+
+- schema.xml has been moved to indexer-solr plugin directory. This file is provided as a
+ reference/guide for Solr users (NUTCH-2654)
+
+### Breaking Changes
+
+- The value of crawl.gen.delay is now read in milliseconds as stated in the description in nutch-default.xml.
+ Previously, the value has been read in days, see [NUTCH-1842](https://issues.apache.org/jira/browse/NUTCH-1842)
+ for further information.
+
+- HostDB entries have been moved from Integer to Long in order to accomodate very large
+ hosts. Remove your existing HostDB and recreate it with bin/nutch updatehostdb, see
+ [NUTCH-2694](https://issues.apache.org/jira/browse/NUTCH-2694) for additional information.
+
+- The signature class TextProfileSignature has been improved to be stable over
+ consecutive runs by sorting tokens by frequency first and secondarily in lexicographic
+ order. If an existing CrawlDb contains signatures generated by TextProfileSignature
+ these are likely to change when upgrading to Nutch 1.16. The previous behavior relying
+ on a semi-stable pseudo-random hash sorting could be restored setting the property
+ `db.signature.text_profile.sec_sort_lex` to `false`. See also NUTCH-2381.
+
+### Bug
+
+- [NUTCH-1063](https://issues.apache.org/jira/browse/NUTCH-1063) - OutlinkExtractor test generates an exception but does not fail
+- [NUTCH-1842](https://issues.apache.org/jira/browse/NUTCH-1842) - crawl.gen.delay has a wrong default value in nutch-default.xml or is being parsed incorrectly
+- [NUTCH-2279](https://issues.apache.org/jira/browse/NUTCH-2279) - LinkRank fails when using Hadoop MR output compression
+- [NUTCH-2381](https://issues.apache.org/jira/browse/NUTCH-2381) - In some situations the class TextProfileSignature gives different signatures for the same text "profile" page.
+- [NUTCH-2387](https://issues.apache.org/jira/browse/NUTCH-2387) - Nutch should not index document with "noindex" meta
+- [NUTCH-2457](https://issues.apache.org/jira/browse/NUTCH-2457) - Embedded documents likely not correctly parsed by Tika
+- [NUTCH-2475](https://issues.apache.org/jira/browse/NUTCH-2475) - If and else-if branches has the same condition
+- [NUTCH-2482](https://issues.apache.org/jira/browse/NUTCH-2482) - index-geoip not to add null values to document fields
+- [NUTCH-2585](https://issues.apache.org/jira/browse/NUTCH-2585) - NPE in TrieStringMatcher
+- [NUTCH-2598](https://issues.apache.org/jira/browse/NUTCH-2598) - URLNormalizerChecker fails on invalid URLs in input
+- [NUTCH-2606](https://issues.apache.org/jira/browse/NUTCH-2606) - MIME detection is wrong for plain-text documents send as Content-Type "application/msword"
+- [NUTCH-2635](https://issues.apache.org/jira/browse/NUTCH-2635) - Generator writes unneeded temporary output
+- [NUTCH-2639](https://issues.apache.org/jira/browse/NUTCH-2639) - bin/nutch fails to set native library path on Cygwin causing jobs to fail with UnsatisfiedLinkError
+- [NUTCH-2641](https://issues.apache.org/jira/browse/NUTCH-2641) - ClassCastException in webui
+- [NUTCH-2642](https://issues.apache.org/jira/browse/NUTCH-2642) - MoreIndexingFilter parses ISO 8601 UTC dates in local time zone
+- [NUTCH-2643](https://issues.apache.org/jira/browse/NUTCH-2643) - ant target "resolve-default" to depend on "init"
+- [NUTCH-2644](https://issues.apache.org/jira/browse/NUTCH-2644) - CrawlDbReader -dump ignores filter options
+- [NUTCH-2645](https://issues.apache.org/jira/browse/NUTCH-2645) - Webgraph tools ignore command-line options
+- [NUTCH-2650](https://issues.apache.org/jira/browse/NUTCH-2650) - -addBinaryContent -base64 flags are causing "String length must be a multiple of four" error in IndexingJob
+- [NUTCH-2652](https://issues.apache.org/jira/browse/NUTCH-2652) - Fetcher launches more fetch tasks than fetch lists
+- [NUTCH-2655](https://issues.apache.org/jira/browse/NUTCH-2655) - Update Solr schema.xml for Solr 7.x
+- [NUTCH-2656](https://issues.apache.org/jira/browse/NUTCH-2656) - Update description to configure Solr 7.x in tutorial
+- [NUTCH-2673](https://issues.apache.org/jira/browse/NUTCH-2673) - EOFException protocol-http
+- [NUTCH-2674](https://issues.apache.org/jira/browse/NUTCH-2674) - HostDb: dump shows wrong column headers
+- [NUTCH-2680](https://issues.apache.org/jira/browse/NUTCH-2680) - Documentation: https supported by multiple protocol plugins not only httpclient
+- [NUTCH-2687](https://issues.apache.org/jira/browse/NUTCH-2687) - Regex for reading title from Content-Disposition is wrong
+- [NUTCH-2694](https://issues.apache.org/jira/browse/NUTCH-2694) - HostDB to aggregate by long instead of integer
+- [NUTCH-2696](https://issues.apache.org/jira/browse/NUTCH-2696) - Nutch SegmentReader does not dump non-ASCII characters with Hadoop 3.x
+- [NUTCH-2699](https://issues.apache.org/jira/browse/NUTCH-2699) - Protocol-okhttp: needless loops to increment requested bytes counter when more content is already buffered
+- [NUTCH-2703](https://issues.apache.org/jira/browse/NUTCH-2703) - parse-tika: Boilerpipe should not run for non-(X)HTML pages
+- [NUTCH-2706](https://issues.apache.org/jira/browse/NUTCH-2706) - -addBinaryContent flag can cause "String length must be a multiple of four" error in IndexingJob
+- [NUTCH-2715](https://issues.apache.org/jira/browse/NUTCH-2715) - WARCExporter fails on large records
+- [NUTCH-2716](https://issues.apache.org/jira/browse/NUTCH-2716) - protocol-http: Response headers are not stored for a compressed response
+- [NUTCH-2717](https://issues.apache.org/jira/browse/NUTCH-2717) - Generator cannot open hostDB
+- [NUTCH-2722](https://issues.apache.org/jira/browse/NUTCH-2722) - Fetch dependencies via https
+- [NUTCH-2723](https://issues.apache.org/jira/browse/NUTCH-2723) - Indexer Solr not to decode URLs before deletion
+- [NUTCH-2724](https://issues.apache.org/jira/browse/NUTCH-2724) - Metadata indexer not to emit empty values
+- [NUTCH-2729](https://issues.apache.org/jira/browse/NUTCH-2729) - protocol-okhttp: fix marking of truncated content
+- [NUTCH-2731](https://issues.apache.org/jira/browse/NUTCH-2731) - Solr Cleanup Step Fails when Authentication is Required
+- [NUTCH-2738](https://issues.apache.org/jira/browse/NUTCH-2738) - Generator: document property generate.restrict.status
+- [NUTCH-2740](https://issues.apache.org/jira/browse/NUTCH-2740) - Generator: generate.max.count overflow not logged
+
+### New Feature
+
+- [NUTCH-2676](https://issues.apache.org/jira/browse/NUTCH-2676) - Update to the latest selenium and add code to use chrome and firefox headless mode with the remote web driver
+
+### Improvement
+
+- [NUTCH-1014](https://issues.apache.org/jira/browse/NUTCH-1014) - Migrate from Apache ORO to java.util.regex
+- [NUTCH-1021](https://issues.apache.org/jira/browse/NUTCH-1021) - Migrate OutlinkExtractor from Apache ORO to java.util.regex
+- [NUTCH-1982](https://issues.apache.org/jira/browse/NUTCH-1982) - Make Git ignore IDE project files and add note about IDE setup
+- [NUTCH-2460](https://issues.apache.org/jira/browse/NUTCH-2460) - use the headless option of firefox and chrome in protocol-selenium
+- [NUTCH-2602](https://issues.apache.org/jira/browse/NUTCH-2602) - Configuration values in the description of index writers
+- [NUTCH-2612](https://issues.apache.org/jira/browse/NUTCH-2612) - Support for sitemap processing by hostname
+- [NUTCH-2623](https://issues.apache.org/jira/browse/NUTCH-2623) - Fetcher to guarantee delay for same host/domain/ip independent of http/https protocol
+- [NUTCH-2625](https://issues.apache.org/jira/browse/NUTCH-2625) - ProtocolFactory.getProtocol(url) may create multiple plugin instances
+- [NUTCH-2626](https://issues.apache.org/jira/browse/NUTCH-2626) - bin/crawl: remove option -noParsing from fetch command
+- [NUTCH-2627](https://issues.apache.org/jira/browse/NUTCH-2627) - Fetcher to optionally filter URLs
+- [NUTCH-2628](https://issues.apache.org/jira/browse/NUTCH-2628) - Fetcher: optionally generate signature of unparsed content
+- [NUTCH-2629](https://issues.apache.org/jira/browse/NUTCH-2629) - Documentation for CSV Index Writer
+- [NUTCH-2630](https://issues.apache.org/jira/browse/NUTCH-2630) - Fetcher to log skipped records by robots.txt
+- [NUTCH-2631](https://issues.apache.org/jira/browse/NUTCH-2631) - KafkaIndexWriter
+- [NUTCH-2632](https://issues.apache.org/jira/browse/NUTCH-2632) - protocol-okhttp doesn't accept proxy authentication
+- [NUTCH-2633](https://issues.apache.org/jira/browse/NUTCH-2633) - Fix deprecation warnings when building Nutch master branch under JDK 10.0.2+13
+- [NUTCH-2647](https://issues.apache.org/jira/browse/NUTCH-2647) - Skip TLS certificate checks in protocol-http plugin
+- [NUTCH-2648](https://issues.apache.org/jira/browse/NUTCH-2648) - Make configurable whether TLS/SSL certificates are checked by protocol plugins
+- [NUTCH-2651](https://issues.apache.org/jira/browse/NUTCH-2651) - Upgrade to Tika 1.19.1 (from 1.18)
+- [NUTCH-2653](https://issues.apache.org/jira/browse/NUTCH-2653) - ProtocolFactory.getProtocol(url) creates separate plugin instances for http/https
+- [NUTCH-2654](https://issues.apache.org/jira/browse/NUTCH-2654) - Remove obsolete index-writer configuration in conf/
+- [NUTCH-2657](https://issues.apache.org/jira/browse/NUTCH-2657) - Protocol-http to store HTTP response header with "\r\n"
+- [NUTCH-2658](https://issues.apache.org/jira/browse/NUTCH-2658) - Add README file to all plugins in src/plugin
+- [NUTCH-2659](https://issues.apache.org/jira/browse/NUTCH-2659) - Add missing Apache license headers
+- [NUTCH-2660](https://issues.apache.org/jira/browse/NUTCH-2660) - Unit tests of plugins parse-js, headings, index-jexl-filter to be executed during build
+- [NUTCH-2661](https://issues.apache.org/jira/browse/NUTCH-2661) - Move TestOutlinks to the proper path
+- [NUTCH-2663](https://issues.apache.org/jira/browse/NUTCH-2663) - Improve index-jexl-filter syntax for scripts
+- [NUTCH-2666](https://issues.apache.org/jira/browse/NUTCH-2666) - Increase default value for http.content.limit / ftp.content.limit / file.content.limit
+- [NUTCH-2668](https://issues.apache.org/jira/browse/NUTCH-2668) - Integrate OWASP dependency checks as ant target
+- [NUTCH-2678](https://issues.apache.org/jira/browse/NUTCH-2678) - Allow for per-host configurable protocol plugin
+- [NUTCH-2682](https://issues.apache.org/jira/browse/NUTCH-2682) - Upgrade to Tika 1.20
+- [NUTCH-2683](https://issues.apache.org/jira/browse/NUTCH-2683) - DeduplicationJob: add option to prefer https:// over http://
+- [NUTCH-2686](https://issues.apache.org/jira/browse/NUTCH-2686) - Separate field for mime types mapped by index-more plugin
+- [NUTCH-2688](https://issues.apache.org/jira/browse/NUTCH-2688) - Unify the licence headers
+- [NUTCH-2689](https://issues.apache.org/jira/browse/NUTCH-2689) - Speed up urlfilter-regex and urlfilter-automaton
+- [NUTCH-2690](https://issues.apache.org/jira/browse/NUTCH-2690) - Configurable and fast URL filter
+- [NUTCH-2691](https://issues.apache.org/jira/browse/NUTCH-2691) - Improve logging from scoring-depth plugin
+- [NUTCH-2692](https://issues.apache.org/jira/browse/NUTCH-2692) - Subcollection to support case-insensitive white and black lists
+- [NUTCH-2693](https://issues.apache.org/jira/browse/NUTCH-2693) - Misspelled configuration property names in documentation
+- [NUTCH-2695](https://issues.apache.org/jira/browse/NUTCH-2695) - Fix some alerts raised by LGTM
+- [NUTCH-2700](https://issues.apache.org/jira/browse/NUTCH-2700) - Indexchecker: improve command-line help
+- [NUTCH-2701](https://issues.apache.org/jira/browse/NUTCH-2701) - Fetcher: log dates and times also in human-readable form
+- [NUTCH-2702](https://issues.apache.org/jira/browse/NUTCH-2702) - Fetcher: suppress stack for frequent exceptions
+- [NUTCH-2704](https://issues.apache.org/jira/browse/NUTCH-2704) - Upgrade crawler-commons dependency to 1.0
+- [NUTCH-2708](https://issues.apache.org/jira/browse/NUTCH-2708) - urlfilter-automaton: update library dependency (dk.brics.automaton)
+- [NUTCH-2709](https://issues.apache.org/jira/browse/NUTCH-2709) - Remove unused properties and code related to HTTP protocol
+- [NUTCH-2718](https://issues.apache.org/jira/browse/NUTCH-2718) - Names of index writers and exchanges configuration files to be configurable
+- [NUTCH-2719](https://issues.apache.org/jira/browse/NUTCH-2719) - NPE if exchanges.xml uses index writer not available
+- [NUTCH-2725](https://issues.apache.org/jira/browse/NUTCH-2725) - Plugin lib-http to support per-host configurable cookies
+- [NUTCH-2726](https://issues.apache.org/jira/browse/NUTCH-2726) - Upgrade to Tika 1.22
+- [NUTCH-2727](https://issues.apache.org/jira/browse/NUTCH-2727) - Upgrade Hadoop dependencies to 2.9.2
+- [NUTCH-2728](https://issues.apache.org/jira/browse/NUTCH-2728) - protocol-okhttp: upgrade okhttp dependency to 3.14.2
+- [NUTCH-2732](https://issues.apache.org/jira/browse/NUTCH-2732) - Ignored and tracked configuration files by git
+- [NUTCH-2736](https://issues.apache.org/jira/browse/NUTCH-2736) - Upgrade Dockerfile to be based on recent Ubuntu LTS version
+- [NUTCH-2737](https://issues.apache.org/jira/browse/NUTCH-2737) - Generator: count and log reason of rejections during selection
+
+### Task
+
+- [NUTCH-2192](https://issues.apache.org/jira/browse/NUTCH-2192) - Get rid of oro
+- [NUTCH-2613](https://issues.apache.org/jira/browse/NUTCH-2613) - Documentation for exchange component
+- [NUTCH-2698](https://issues.apache.org/jira/browse/NUTCH-2698) - Remove sonar build task from build.xml
+
+### Sub-task
+
+- [NUTCH-1121](https://issues.apache.org/jira/browse/NUTCH-1121) - JUnit test for parse-js
+- [NUTCH-2621](https://issues.apache.org/jira/browse/NUTCH-2621) - Generate report of third-party licenses
+- [NUTCH-2684](https://issues.apache.org/jira/browse/NUTCH-2684) - Add README.md file to all indexer writers plugins
+- [NUTCH-2685](https://issues.apache.org/jira/browse/NUTCH-2685) - Add README.md file to all exchange plugins
+
+
+## Nutch 1.15 Release (25/07/2018)
+Release Report: https://s.apache.org/nczS
+
+### Breaking Changes
+
+- indexer plugins are now configured in a single XML file (conf/index-writers.xml),
+ see https://cwiki.apache.org/confluence/display/NUTCH/IndexWriters - setting or overwriting configuration
+ parameters via Nutch properties is not possible anymore.
+
+### Bug
+
+- [NUTCH-1993](https://issues.apache.org/jira/browse/NUTCH-1993) - Nutch does not use backup parsers
+- [NUTCH-2071](https://issues.apache.org/jira/browse/NUTCH-2071) - A parser failure on a single document may fail crawling job if parser.timeout=-1
+- [NUTCH-2145](https://issues.apache.org/jira/browse/NUTCH-2145) - parse/index checker fail to fetch valid percent-encoded URLs
+- [NUTCH-2161](https://issues.apache.org/jira/browse/NUTCH-2161) - Interrupted failed and/or killed tasks fail to clean up temp directories in HDFS
+- [NUTCH-2273](https://issues.apache.org/jira/browse/NUTCH-2273) - Selenium and InteractiveSelenium Do Not Support HTTPS
+- [NUTCH-2310](https://issues.apache.org/jira/browse/NUTCH-2310) - Protocol-Selenium does not support HTTPS protocol
+- [NUTCH-2321](https://issues.apache.org/jira/browse/NUTCH-2321) - Indexing filter checker leaks threads
+- [NUTCH-2324](https://issues.apache.org/jira/browse/NUTCH-2324) - Issue in setting default linkdb path
+- [NUTCH-2447](https://issues.apache.org/jira/browse/NUTCH-2447) - Work-around SSLProtocolException: handshake alert: unrecognized_name
+- [NUTCH-2454](https://issues.apache.org/jira/browse/NUTCH-2454) - REST API fix for usage of hostdb in generator
+- [NUTCH-2461](https://issues.apache.org/jira/browse/NUTCH-2461) - Generate passes the data to when maxCount == 0
+- [NUTCH-2466](https://issues.apache.org/jira/browse/NUTCH-2466) - Sitemap processor to follow redirects
+- [NUTCH-2467](https://issues.apache.org/jira/browse/NUTCH-2467) - Sitemap type field can be null
+- [NUTCH-2485](https://issues.apache.org/jira/browse/NUTCH-2485) - ParserFactory swallows exception
+- [NUTCH-2486](https://issues.apache.org/jira/browse/NUTCH-2486) - Compiler Warning: Unchecked / unsafe operations in MimeTypeIndexingFilter
+- [NUTCH-2489](https://issues.apache.org/jira/browse/NUTCH-2489) - Dependency collision with lucene-analyzers-common in scoring-similarity plugin
+- [NUTCH-2490](https://issues.apache.org/jira/browse/NUTCH-2490) - Sitemap processing: Sitemap index files not working
+- [NUTCH-2494](https://issues.apache.org/jira/browse/NUTCH-2494) - Fetcher: java.lang.IllegalArgumentException: Wrong FS: s3
+- [NUTCH-2499](https://issues.apache.org/jira/browse/NUTCH-2499) - Elastic REST Indexer: Duplicate values
+- [NUTCH-2505](https://issues.apache.org/jira/browse/NUTCH-2505) - nutch does not delete the .locked file, when the generator partition got an exception
+- [NUTCH-2508](https://issues.apache.org/jira/browse/NUTCH-2508) - Misleading documentation about http.proxy.exception.list
+- [NUTCH-2509](https://issues.apache.org/jira/browse/NUTCH-2509) - Inconsistent behavior in SitemapProcessor
+- [NUTCH-2513](https://issues.apache.org/jira/browse/NUTCH-2513) - ant eclipse target fails with "protocol switch unsafe"
+- [NUTCH-2517](https://issues.apache.org/jira/browse/NUTCH-2517) - mergesegs corrupts segment data
+- [NUTCH-2518](https://issues.apache.org/jira/browse/NUTCH-2518) - Must check return value of job.waitForCompletion()
+- [NUTCH-2520](https://issues.apache.org/jira/browse/NUTCH-2520) - Wrong Accept-Charset sent when http.accept.charset is not defined
+- [NUTCH-2521](https://issues.apache.org/jira/browse/NUTCH-2521) - SitemapProcessor to use property sitemap.redir.max
+- [NUTCH-2523](https://issues.apache.org/jira/browse/NUTCH-2523) - UpdateHostDB blocks usage of plugins unintentionally
+- [NUTCH-2524](https://issues.apache.org/jira/browse/NUTCH-2524) - bin/crawl: fix check for HostDb in distributed mode
+- [NUTCH-2533](https://issues.apache.org/jira/browse/NUTCH-2533) - Injector: NullPointerException if seed URL dir contains non-file entries
+- [NUTCH-2535](https://issues.apache.org/jira/browse/NUTCH-2535) - CrawlDbReader -stats: ClassCastException
+- [NUTCH-2544](https://issues.apache.org/jira/browse/NUTCH-2544) - Nutch 1.15 no longer compatible with AWS EMR and S3
+- [NUTCH-2547](https://issues.apache.org/jira/browse/NUTCH-2547) - urlnormalizer-basic fails on special characters in path/query
+- [NUTCH-2549](https://issues.apache.org/jira/browse/NUTCH-2549) - protocol-http does not behave the same as browsers
+- [NUTCH-2550](https://issues.apache.org/jira/browse/NUTCH-2550) - Fetcher fails to follow redirects
+- [NUTCH-2551](https://issues.apache.org/jira/browse/NUTCH-2551) - NullPointerException in generator
+- [NUTCH-2552](https://issues.apache.org/jira/browse/NUTCH-2552) - CrawlDbReader -topN fails
+- [NUTCH-2553](https://issues.apache.org/jira/browse/NUTCH-2553) - Fetcher not to modify URLs to be fetched
+- [NUTCH-2554](https://issues.apache.org/jira/browse/NUTCH-2554) - parserchecker can't fetch some URLs
+- [NUTCH-2565](https://issues.apache.org/jira/browse/NUTCH-2565) - MergeDB incorrectly handles unfetched CrawlDatums
+- [NUTCH-2568](https://issues.apache.org/jira/browse/NUTCH-2568) - Caught exception is immediately rethrown
+- [NUTCH-2569](https://issues.apache.org/jira/browse/NUTCH-2569) - ClassNotFoundException when running in (pseudo-)distributed mode
+- [NUTCH-2570](https://issues.apache.org/jira/browse/NUTCH-2570) - Deduplication job fails to install deduplicated CrawlDb
+- [NUTCH-2571](https://issues.apache.org/jira/browse/NUTCH-2571) - SegmentReader -list fails to read segment
+- [NUTCH-2572](https://issues.apache.org/jira/browse/NUTCH-2572) - HostDb: updatehostdb does not set values
+- [NUTCH-2574](https://issues.apache.org/jira/browse/NUTCH-2574) - Generator: hostCount >= maxCount comparison wrong
+- [NUTCH-2581](https://issues.apache.org/jira/browse/NUTCH-2581) - Caching of redirected robots.txt may overwrite correct robots.txt rules
+- [NUTCH-2589](https://issues.apache.org/jira/browse/NUTCH-2589) - HTML redirections are not followed when using parse-tika
+- [NUTCH-2590](https://issues.apache.org/jira/browse/NUTCH-2590) - SegmentReader -get fails
+- [NUTCH-2592](https://issues.apache.org/jira/browse/NUTCH-2592) - Fetcher to log reason of failed fetches
+- [NUTCH-2593](https://issues.apache.org/jira/browse/NUTCH-2593) - Single mode doesn't work in RabbitMQ indexer
+- [NUTCH-2597](https://issues.apache.org/jira/browse/NUTCH-2597) - NPE in updatehostdb
+- [NUTCH-2601](https://issues.apache.org/jira/browse/NUTCH-2601) - Elasticsearch Rest and Amazon CloudSearch have the same implementation class in indexer-writers.xml
+- [NUTCH-2607](https://issues.apache.org/jira/browse/NUTCH-2607) - ParserChecker should call ScoringFilters.passScoreAfterParsing() on all parses
+- [NUTCH-2609](https://issues.apache.org/jira/browse/NUTCH-2609) - urlnormalizer-basic to normalize path of file: URLs
+- [NUTCH-2614](https://issues.apache.org/jira/browse/NUTCH-2614) - NPE in CrawlDbReader -stats on empty CrawlDb
+- [NUTCH-2616](https://issues.apache.org/jira/browse/NUTCH-2616) - Review routing of deletions by Exchange component
+- [NUTCH-2618](https://issues.apache.org/jira/browse/NUTCH-2618) - protocol-okhttp not to use http.timeout for max duration to fetch document
+- [NUTCH-2620](https://issues.apache.org/jira/browse/NUTCH-2620) - urlfilter-validator incorrectly assumes that top-level domains are not longer than 4 characters
+- [NUTCH-2624](https://issues.apache.org/jira/browse/NUTCH-2624) - protocol-okhttp resource leak
+
+### New Feature
+
+- [NUTCH-1129](https://issues.apache.org/jira/browse/NUTCH-1129) - Any23 Nutch plugin
+- [NUTCH-1541](https://issues.apache.org/jira/browse/NUTCH-1541) - Indexer plugin to write CSV
+- [NUTCH-2412](https://issues.apache.org/jira/browse/NUTCH-2412) - Exchange component for indexing job
+- [NUTCH-2492](https://issues.apache.org/jira/browse/NUTCH-2492) - Add more configuration parameters to crawl script
+
+### Improvement
+
+- [NUTCH-1106](https://issues.apache.org/jira/browse/NUTCH-1106) - Options to skip url's based on length
+- [NUTCH-1480](https://issues.apache.org/jira/browse/NUTCH-1480) - SolrIndexer to write to multiple servers.
+- [NUTCH-2012](https://issues.apache.org/jira/browse/NUTCH-2012) - Merge parsechecker and indexchecker
+- [NUTCH-2375](https://issues.apache.org/jira/browse/NUTCH-2375) - Upgrade the code base from org.apache.hadoop.mapred to org.apache.hadoop.mapreduce
+- [NUTCH-2390](https://issues.apache.org/jira/browse/NUTCH-2390) - No documentation on pluggable indexing
+- [NUTCH-2411](https://issues.apache.org/jira/browse/NUTCH-2411) - Index-metadata to support indexing multiple values for a field
+- [NUTCH-2416](https://issues.apache.org/jira/browse/NUTCH-2416) - Fetcher to log thread ID
+- [NUTCH-2432](https://issues.apache.org/jira/browse/NUTCH-2432) - Protocol httpclient to disable cookies if http.enable.cookie.header is false
+- [NUTCH-2441](https://issues.apache.org/jira/browse/NUTCH-2441) - ARG_SEGMENT usage
+- [NUTCH-2491](https://issues.apache.org/jira/browse/NUTCH-2491) - Integrate sitemap processing and HostDB into crawl script
+- [NUTCH-2493](https://issues.apache.org/jira/browse/NUTCH-2493) - Add configuration parameter for sitemap processing to crawler script
+- [NUTCH-2497](https://issues.apache.org/jira/browse/NUTCH-2497) - Elastic REST Indexer: Allow multiple hosts
+- [NUTCH-2502](https://issues.apache.org/jira/browse/NUTCH-2502) - Any23 Plugin: Add Content-Type filtering
+- [NUTCH-2503](https://issues.apache.org/jira/browse/NUTCH-2503) - Add option to run tests for a single plugin
+- [NUTCH-2510](https://issues.apache.org/jira/browse/NUTCH-2510) - Crawl script modification. HostDb : generate, optional usage and description
+- [NUTCH-2516](https://issues.apache.org/jira/browse/NUTCH-2516) - Hadoop imports use wildcards
+- [NUTCH-2519](https://issues.apache.org/jira/browse/NUTCH-2519) - Log mapreduce job counters in local mode
+- [NUTCH-2526](https://issues.apache.org/jira/browse/NUTCH-2526) - NPE in scoring-opic when indexing document without CrawlDb datum
+- [NUTCH-2527](https://issues.apache.org/jira/browse/NUTCH-2527) - URL filter: provide rules to exclude localhost and private address spaces
+- [NUTCH-2530](https://issues.apache.org/jira/browse/NUTCH-2530) - Rename property db.max.anchor.length > linkdb.max.anchor.length
+- [NUTCH-2534](https://issues.apache.org/jira/browse/NUTCH-2534) - CrawlDbReader -stats: make score quantiles configurable
+- [NUTCH-2539](https://issues.apache.org/jira/browse/NUTCH-2539) - Not correct naming of db.url.filters and db.url.normalizers in nutch-default.xml
+- [NUTCH-2543](https://issues.apache.org/jira/browse/NUTCH-2543) - readdb & readlinkdb to implement AbstractChecker
+- [NUTCH-2545](https://issues.apache.org/jira/browse/NUTCH-2545) - Upgrade to Any23 2.2
+- [NUTCH-2566](https://issues.apache.org/jira/browse/NUTCH-2566) - Fix exception log messages
+- [NUTCH-2576](https://issues.apache.org/jira/browse/NUTCH-2576) - HTTP protocol plugin based on okhttp
+- [NUTCH-2577](https://issues.apache.org/jira/browse/NUTCH-2577) - protocol-selenium can't handle https
+- [NUTCH-2578](https://issues.apache.org/jira/browse/NUTCH-2578) - Avoid lock by MimeUtil in constructor of protocol.Content
+- [NUTCH-2579](https://issues.apache.org/jira/browse/NUTCH-2579) - Fetcher to use parsed URL to call ProtocolFactory.getProtocol(url)
+- [NUTCH-2580](https://issues.apache.org/jira/browse/NUTCH-2580) - Improvements for Rabbitmq support
+- [NUTCH-2583](https://issues.apache.org/jira/browse/NUTCH-2583) - Upgrading Nutch's dependencies
+- [NUTCH-2584](https://issues.apache.org/jira/browse/NUTCH-2584) - Upgrade parse-tika to use Tika 1.18
+- [NUTCH-2594](https://issues.apache.org/jira/browse/NUTCH-2594) - Documentation for indexer plugins
+- [NUTCH-2595](https://issues.apache.org/jira/browse/NUTCH-2595) - Upgrade crawler-commons dependency to 0.10
+- [NUTCH-2600](https://issues.apache.org/jira/browse/NUTCH-2600) - Refactoring indexer-solr
+- [NUTCH-2611](https://issues.apache.org/jira/browse/NUTCH-2611) - Add line-breaks when parsing HTML block-level elements
+- [NUTCH-2617](https://issues.apache.org/jira/browse/NUTCH-2617) - Disable Exchange component by default
+- [NUTCH-2619](https://issues.apache.org/jira/browse/NUTCH-2619) - protocol-okhttp: allow to keep partially fetched docs as truncated
+
+### Task
+
+- [NUTCH-1219](https://issues.apache.org/jira/browse/NUTCH-1219) - Upgrade all jobs to new MapReduce API
+- [NUTCH-1228](https://issues.apache.org/jira/browse/NUTCH-1228) - Change mapred.task.timeout to mapreduce.task.timeout in fetcher
+
+### Sub-task
+
+- [NUTCH-1223](https://issues.apache.org/jira/browse/NUTCH-1223) - Migrate WebGraph to MapReduce API
+- [NUTCH-1224](https://issues.apache.org/jira/browse/NUTCH-1224) - Migrate FreeGenerator to MapReduce API
+- [NUTCH-1226](https://issues.apache.org/jira/browse/NUTCH-1226) - Migrate CrawlDbReader to MapReduce API
+- [NUTCH-2152](https://issues.apache.org/jira/browse/NUTCH-2152) - CommonCrawl dump via Service endpoint
+- [NUTCH-2555](https://issues.apache.org/jira/browse/NUTCH-2555) - URL normalization problem: path not starting with a '/'
+- [NUTCH-2556](https://issues.apache.org/jira/browse/NUTCH-2556) - protocol-http makes invalid HTTP/1.0 requests
+- [NUTCH-2557](https://issues.apache.org/jira/browse/NUTCH-2557) - protocol-http fails to follow redirections when an HTTP response body is invalid
+- [NUTCH-2558](https://issues.apache.org/jira/browse/NUTCH-2558) - protocol-http cannot handle a missing HTTP status line
+- [NUTCH-2559](https://issues.apache.org/jira/browse/NUTCH-2559) - protocol-http cannot handle colons after the HTTP status code
+- [NUTCH-2560](https://issues.apache.org/jira/browse/NUTCH-2560) - protocol-http throws an error when an http header spans over multiple lines
+- [NUTCH-2561](https://issues.apache.org/jira/browse/NUTCH-2561) - protocol-http can be made to read arbitrarily large HTTP responses
+- [NUTCH-2562](https://issues.apache.org/jira/browse/NUTCH-2562) - protocol-http fails to read large chunked HTTP responses
+- [NUTCH-2563](https://issues.apache.org/jira/browse/NUTCH-2563) - HTTP header spellchecking issues
+- [NUTCH-2575](https://issues.apache.org/jira/browse/NUTCH-2575) - protocol-http does not respect the maximum content-size for chunked responses
+- [NUTCH-2622](https://issues.apache.org/jira/browse/NUTCH-2622) - Unbundle LGPL-licensed jars from binary release
+
+
+## Nutch 1.14 Release 18/12/2017 (dd/mm/yyyy)
+
+### Breaking Changes
+
+- the bin/crawl script now expects the path to the seed to be preceded by -s
+ ([NUTCH-2046](https://issues.apache.org/jira/browse/NUTCH-2046))
+
+### Bug
+
+- [NUTCH-2071](https://issues.apache.org/jira/browse/NUTCH-2071) - A parser failure on a single document may fail crawling job
+- [NUTCH-2235](https://issues.apache.org/jira/browse/NUTCH-2235) - Classpath discrepancy with protocol-selenium in deploy mode
+- [NUTCH-2269](https://issues.apache.org/jira/browse/NUTCH-2269) - Clean not working after crawl
+- [NUTCH-2295](https://issues.apache.org/jira/browse/NUTCH-2295) - Nutch master docker container broken
+- [NUTCH-2297](https://issues.apache.org/jira/browse/NUTCH-2297) - CrawlDbReader -stats wrong values for earliest fetch time and shortest interval
+- [NUTCH-2316](https://issues.apache.org/jira/browse/NUTCH-2316) - Library conflict with Parser-Tika Plugin and Lib Folder
+- [NUTCH-2317](https://issues.apache.org/jira/browse/NUTCH-2317) - Plugin jars don't get added to classpath while running in local
+- [NUTCH-2322](https://issues.apache.org/jira/browse/NUTCH-2322) - URL not available for Jexl operations
+- [NUTCH-2354](https://issues.apache.org/jira/browse/NUTCH-2354) - Upgrade Hadoop dependencies to 2.7.4
+- [NUTCH-2365](https://issues.apache.org/jira/browse/NUTCH-2365) - HTTP Redirects to SubDomains don't get crawled if db.ignore.external.links.mode == byDomain
+- [NUTCH-2371](https://issues.apache.org/jira/browse/NUTCH-2371) - Injector to support noFilter and noNormalize
+- [NUTCH-2372](https://issues.apache.org/jira/browse/NUTCH-2372) - Javadocs build failing.
+- [NUTCH-2386](https://issues.apache.org/jira/browse/NUTCH-2386) - BasicURLNormalizer does not encode curly braces
+- [NUTCH-2391](https://issues.apache.org/jira/browse/NUTCH-2391) - Spurious Duplications for MD5
+- [NUTCH-2394](https://issues.apache.org/jira/browse/NUTCH-2394) - Possible bugs in the source code
+- [NUTCH-2398](https://issues.apache.org/jira/browse/NUTCH-2398) - Fetcher saving redirected robots.txt under redirect target URL
+- [NUTCH-2399](https://issues.apache.org/jira/browse/NUTCH-2399) - indexer-elastic does not index multi-value fields (only the first value is indexed)
+- [NUTCH-2401](https://issues.apache.org/jira/browse/NUTCH-2401) - headings plugin does not trim values
+- [NUTCH-2403](https://issues.apache.org/jira/browse/NUTCH-2403) - Nutch Selenium: Wrong documentation about PhantomJS
+- [NUTCH-2413](https://issues.apache.org/jira/browse/NUTCH-2413) - Parsing fetcher to respect property "parse.filter.urls"
+- [NUTCH-2420](https://issues.apache.org/jira/browse/NUTCH-2420) - Bug in variable generate.max.count and fetcher.server.delay
+- [NUTCH-2436](https://issues.apache.org/jira/browse/NUTCH-2436) - Remove empty comment, and redundant semicolon from CommandRunner
+- [NUTCH-2442](https://issues.apache.org/jira/browse/NUTCH-2442) - Injector to stop if job fails to avoid loss of CrawlDb
+- [NUTCH-2444](https://issues.apache.org/jira/browse/NUTCH-2444) - HostDB CSV dumper to emit field header by default
+- [NUTCH-2446](https://issues.apache.org/jira/browse/NUTCH-2446) - URLFiltersCheck fix
+- [NUTCH-2448](https://issues.apache.org/jira/browse/NUTCH-2448) - Allow Sending an empty http.agent.version
+- [NUTCH-2451](https://issues.apache.org/jira/browse/NUTCH-2451) - protocol-ftp to resolve relative URL when following redirects
+- [NUTCH-2452](https://issues.apache.org/jira/browse/NUTCH-2452) - Problem retrieving encoded URLs via FTP?
+- [NUTCH-2456](https://issues.apache.org/jira/browse/NUTCH-2456) - Allow to index pages/URLs not contained in CrawlDb
+- [NUTCH-2458](https://issues.apache.org/jira/browse/NUTCH-2458) - TikaParser doesn't work with tika-config.xml set
+- [NUTCH-2464](https://issues.apache.org/jira/browse/NUTCH-2464) - Plugin headings: Headers That Contain HTML Elements Are Not Parsed
+- [NUTCH-2465](https://issues.apache.org/jira/browse/NUTCH-2465) - Broken Eclipse project. Classpaths and interactiveselenium should be fixed.
+- [NUTCH-2472](https://issues.apache.org/jira/browse/NUTCH-2472) - Sitemap processor does not honour db.ignore.external.links
+- [NUTCH-2473](https://issues.apache.org/jira/browse/NUTCH-2473) - Elasticsearch REST Indexer broken due to wrong depenency
+- [NUTCH-2474](https://issues.apache.org/jira/browse/NUTCH-2474) - CrawlDbReader -stats fails with ClassCastException
+- [NUTCH-2478](https://issues.apache.org/jira/browse/NUTCH-2478) - // is not a valid base URL
+- [NUTCH-2483](https://issues.apache.org/jira/browse/NUTCH-2483) - Remove/replace indirect dependencies to org.json
+
+### Improvement
+
+- [NUTCH-1763](https://issues.apache.org/jira/browse/NUTCH-1763) - Improving comments on the Injector Class
+- [NUTCH-2034](https://issues.apache.org/jira/browse/NUTCH-2034) - CrawlDB filtered documents counter.
+- [NUTCH-2035](https://issues.apache.org/jira/browse/NUTCH-2035) - Regex filter using case sensitive rules.
+- [NUTCH-2046](https://issues.apache.org/jira/browse/NUTCH-2046) - The crawl script should be able to skip an initial injection.
+- [NUTCH-2135](https://issues.apache.org/jira/browse/NUTCH-2135) - Ant Eclipse build does not include protocol-interactiveselenium
+- [NUTCH-2193](https://issues.apache.org/jira/browse/NUTCH-2193) - Upgrade feed parser plugin to use rome 1.5
+- [NUTCH-2216](https://issues.apache.org/jira/browse/NUTCH-2216) - db.ignore.*.links to optionally follow internal redirects
+- [NUTCH-2281](https://issues.apache.org/jira/browse/NUTCH-2281) - Support non-default FileSystem
+- [NUTCH-2296](https://issues.apache.org/jira/browse/NUTCH-2296) - Elasticsearch Indexing Over Rest
+- [NUTCH-2320](https://issues.apache.org/jira/browse/NUTCH-2320) - URLFilterChecker to run as TCP Telnet service
+- [NUTCH-2335](https://issues.apache.org/jira/browse/NUTCH-2335) - Injector not to filter and normalize existing URLs in CrawlDb
+- [NUTCH-2362](https://issues.apache.org/jira/browse/NUTCH-2362) - Upgrade MaxMind GeoIP version in index-geoip
+- [NUTCH-2368](https://issues.apache.org/jira/browse/NUTCH-2368) - Variable generate.max.count and fetcher.server.delay
+- [NUTCH-2370](https://issues.apache.org/jira/browse/NUTCH-2370) - FileDumper: save JSON mapping file -> URL
+- [NUTCH-2376](https://issues.apache.org/jira/browse/NUTCH-2376) - Improve configurability of HTTP Accept* header fields
+- [NUTCH-2378](https://issues.apache.org/jira/browse/NUTCH-2378) - ChildFirst plugin classloader
+- [NUTCH-2380](https://issues.apache.org/jira/browse/NUTCH-2380) - indexer-elastic version upgrade to 5.3.0
+- [NUTCH-2397](https://issues.apache.org/jira/browse/NUTCH-2397) - Parser to add paragraph line breaks
+- [NUTCH-2400](https://issues.apache.org/jira/browse/NUTCH-2400) - Solr 6.6.0 compatibility
+- [NUTCH-2406](https://issues.apache.org/jira/browse/NUTCH-2406) - Sum up constants, make minor changes
+- [NUTCH-2408](https://issues.apache.org/jira/browse/NUTCH-2408) - CrawlDb: allow update from unparsed segments
+- [NUTCH-2409](https://issues.apache.org/jira/browse/NUTCH-2409) - Injector: complete command-line help and counters
+- [NUTCH-2414](https://issues.apache.org/jira/browse/NUTCH-2414) - Allow LanguageIndexingFilter to actually filter documents by language.
+- [NUTCH-2430](https://issues.apache.org/jira/browse/NUTCH-2430) - Complete plugin build configuration
+- [NUTCH-2431](https://issues.apache.org/jira/browse/NUTCH-2431) - URLFilterchecker to implement Tool-interface
+- [NUTCH-2439](https://issues.apache.org/jira/browse/NUTCH-2439) - Upgrade to Apache Tika 1.17
+- [NUTCH-2443](https://issues.apache.org/jira/browse/NUTCH-2443) - Extract links from the video tag with the parse-html plugin
+- [NUTCH-2445](https://issues.apache.org/jira/browse/NUTCH-2445) - Fetcher following outlinks to keep track of already fetched items
+- [NUTCH-2463](https://issues.apache.org/jira/browse/NUTCH-2463) - Enable sampling CrawlDB
+- [NUTCH-2468](https://issues.apache.org/jira/browse/NUTCH-2468) - should filter out invalid URLs by default
+- [NUTCH-2470](https://issues.apache.org/jira/browse/NUTCH-2470) - CrawlDbReader -stats to show quantiles of score
+- [NUTCH-2477](https://issues.apache.org/jira/browse/NUTCH-2477) - Refactor *Checker classes to use base class for common code
+- [NUTCH-2480](https://issues.apache.org/jira/browse/NUTCH-2480) - Upgrade crawler-commons dependency to 0.9
+
+### New Feature
+
+- [NUTCH-1465](https://issues.apache.org/jira/browse/NUTCH-1465) - Support sitemaps in Nutch
+- [NUTCH-1932](https://issues.apache.org/jira/browse/NUTCH-1932) - Automatically remove orphaned pages
+- [NUTCH-2333](https://issues.apache.org/jira/browse/NUTCH-2333) - Indexer for RabbitMQ
+- [NUTCH-2338](https://issues.apache.org/jira/browse/NUTCH-2338) - URLNormalizerChecker to run as TCP Telnet service
+- [NUTCH-2415](https://issues.apache.org/jira/browse/NUTCH-2415) - Create a JEXL based IndexingFilter
+- [NUTCH-2433](https://issues.apache.org/jira/browse/NUTCH-2433) - Html Parser: keep htmltag where the outlinks are found
+- [NUTCH-2435](https://issues.apache.org/jira/browse/NUTCH-2435) - New configuration allowing to choose whether to store 'parse_text' directory or not.
+- [NUTCH-2484](https://issues.apache.org/jira/browse/NUTCH-2484) - Extend indexer-elastic-rest to support languages
+
+### Task
+
+- [NUTCH-2181](https://issues.apache.org/jira/browse/NUTCH-2181) - Add Webpage for 3rd Party Connectors/Libraries to Apache Nutch
+
+
+## Nutch 1.13 Release 28/03/2017 (dd/mm/yyyy)
+Release Report: https://s.apache.org/wq3x
+
+### Sub-task
+
+- [NUTCH-2246](https://issues.apache.org/jira/browse/NUTCH-2246) - Refactor /seed endpoint for backward compatibility
+
+### Bug
+
+- [NUTCH-1553](https://issues.apache.org/jira/browse/NUTCH-1553) - Property 'indexer.delete.robots.noindex' not working when using parser-html.
+- [NUTCH-2242](https://issues.apache.org/jira/browse/NUTCH-2242) - lastModified not always set
+- [NUTCH-2291](https://issues.apache.org/jira/browse/NUTCH-2291) - Fix mrunit dependencies
+- [NUTCH-2337](https://issues.apache.org/jira/browse/NUTCH-2337) - urlnormalizer-basic to strip empty port
+- [NUTCH-2345](https://issues.apache.org/jira/browse/NUTCH-2345) - FetchItemQueue logs are logged with wrong class name
+- [NUTCH-2349](https://issues.apache.org/jira/browse/NUTCH-2349) - urlnormalizer-basic NPE for ill-formed URL "http:/"
+- [NUTCH-2357](https://issues.apache.org/jira/browse/NUTCH-2357) - Index metadata throw Exception because writable object cannot be cast to Text
+- [NUTCH-2359](https://issues.apache.org/jira/browse/NUTCH-2359) - Parsefilter-regex raises IndexOutOfBoundsException when rules are ill-formed
+- [NUTCH-2364](https://issues.apache.org/jira/browse/NUTCH-2364) - http.agent.rotate: IllegalArgumentException / last element of agent names ignored
+- [NUTCH-2366](https://issues.apache.org/jira/browse/NUTCH-2366) - Deprecated Job constructor in hostdb/ReadHostDb.java
+
+### Improvement
+
+- [NUTCH-1308](https://issues.apache.org/jira/browse/NUTCH-1308) - Add main() to ZipParser
+- [NUTCH-2164](https://issues.apache.org/jira/browse/NUTCH-2164) - Inconsistent 'Modified Time' in crawl db
+- [NUTCH-2234](https://issues.apache.org/jira/browse/NUTCH-2234) - Upgrade to elasticsearch 2.3.3
+- [NUTCH-2236](https://issues.apache.org/jira/browse/NUTCH-2236) - Upgrade to Hadoop 2.7.2
+- [NUTCH-2262](https://issues.apache.org/jira/browse/NUTCH-2262) - Utilize parameterized logging notation across Fetcher
+- [NUTCH-2272](https://issues.apache.org/jira/browse/NUTCH-2272) - Index checker server to optionally keep client connection open
+- [NUTCH-2286](https://issues.apache.org/jira/browse/NUTCH-2286) - CrawlDbReader -stats to show fetch time and interval
+- [NUTCH-2287](https://issues.apache.org/jira/browse/NUTCH-2287) - Indexer-elastic plugin should use Elasticsearch BulkProcessor and BackoffPolicy
+- [NUTCH-2299](https://issues.apache.org/jira/browse/NUTCH-2299) - Remove obsolete properties protocol.plugin.check.*
+- [NUTCH-2300](https://issues.apache.org/jira/browse/NUTCH-2300) - Fetcher to optionally save robots.txt
+- [NUTCH-2327](https://issues.apache.org/jira/browse/NUTCH-2327) - Seeds injected in REST workflow must be ingested into HDFS
+- [NUTCH-2329](https://issues.apache.org/jira/browse/NUTCH-2329) - Update Slf4j logging for Java 8 and upgrade miredot plugin version
+- [NUTCH-2336](https://issues.apache.org/jira/browse/NUTCH-2336) - SegmentReader to implement Tool
+- [NUTCH-2352](https://issues.apache.org/jira/browse/NUTCH-2352) - Log with Generic Class Name at Nutch 1.x
+- [NUTCH-2355](https://issues.apache.org/jira/browse/NUTCH-2355) - Protocol plugins to set cookie if Cookie metadata field is present
+- [NUTCH-2367](https://issues.apache.org/jira/browse/NUTCH-2367) - Get single record from HostDB
+
+### New Feature
+
+- [NUTCH-2132](https://issues.apache.org/jira/browse/NUTCH-2132) - Publisher/Subscriber model for Nutch to emit events
+
+### Task
+
+- [NUTCH-2171](https://issues.apache.org/jira/browse/NUTCH-2171) - Upgrade Nutch Trunk to Java 1.8
+
+
+## Nutch 1.12 Release 28/05/2016 (dd/mm/yyyy)
+Release Report: https://s.apache.org/nutch1.12
+
+### Breaking Changes
+
+Fellow committers, Nutch 1.12 contains a breaking change [NUTCH-2220](https://issues.apache.org/jira/browse/NUTCH-2220).
+Please use the note below and in the release announcement and keep it on top in this CHANGES.txt for the Nutch 1.12 release.
+
+* replace your old conf/nutch-default.xml with the conf/nutch-default.xml from Nutch 1.12 release
+* if you use LinkDB (e.g. invertlinks) and modified parameters db.max.inlinks and/or db.max.anchor.length
+ and/or db.ignore.internal.links, rename those parameters to linkdb.max.inlinks and
+ linkdb.max.anchor.length and linkdb.ignore.internal.links
+* db.ignore.internal.links and db.ignore.external.links now operate on the CrawlDB only
+* linkdb.ignore.internal.links and linkdb.ignore.external.links now operate on the LinkDB only
+
+### Sub-task
+
+- [NUTCH-2250](https://issues.apache.org/jira/browse/NUTCH-2250) - CommonCrawlDumper : Invalid format + skipped parts
+
+### Bug
+
+- [NUTCH-2042](https://issues.apache.org/jira/browse/NUTCH-2042) - parse-html increase chunk size used to detect charset
+- [NUTCH-2180](https://issues.apache.org/jira/browse/NUTCH-2180) - FileDumper dumps data, but breaks midway on corrupt segments
+- [NUTCH-2189](https://issues.apache.org/jira/browse/NUTCH-2189) - Domain filter must deactivate if no rules are present
+- [NUTCH-2203](https://issues.apache.org/jira/browse/NUTCH-2203) - Suffix URL filter can't handle trailing/leading whitespaces
+- [NUTCH-2206](https://issues.apache.org/jira/browse/NUTCH-2206) - Provide example scoring.similarity.stopword.file
+- [NUTCH-2213](https://issues.apache.org/jira/browse/NUTCH-2213) - CommonCrawlDataDumper saves gzipped body in extracted form
+- [NUTCH-2223](https://issues.apache.org/jira/browse/NUTCH-2223) - Upgrade xercesImpl to 2.11.0 to fix hang on issue in tika mimetype detection
+- [NUTCH-2224](https://issues.apache.org/jira/browse/NUTCH-2224) - Average bytes/second calculated incorrectly in fetcher
+- [NUTCH-2225](https://issues.apache.org/jira/browse/NUTCH-2225) - Parsed time calculated incorrectly
+- [NUTCH-2228](https://issues.apache.org/jira/browse/NUTCH-2228) - Plugin index-replace unit test broken on Java 8
+- [NUTCH-2232](https://issues.apache.org/jira/browse/NUTCH-2232) - DeduplicationJob should decode URL's before length is compared
+- [NUTCH-2241](https://issues.apache.org/jira/browse/NUTCH-2241) - Unstable Selenium plugin in Nutch. Fixed bugs and enhanced configuration
+- [NUTCH-2256](https://issues.apache.org/jira/browse/NUTCH-2256) - Inconsistent log level practice
+
+### Improvement
+
+- [NUTCH-1233](https://issues.apache.org/jira/browse/NUTCH-1233) - Rely on Tika for outlink extraction
+- [NUTCH-1712](https://issues.apache.org/jira/browse/NUTCH-1712) - Use MultipleInputs in Injector to make it a single mapreduce job
+- [NUTCH-2172](https://issues.apache.org/jira/browse/NUTCH-2172) - index-more: document format of contenttype-mapping.txt
+- [NUTCH-2178](https://issues.apache.org/jira/browse/NUTCH-2178) - DeduplicationJob to optionally group on host or domain
+- [NUTCH-2182](https://issues.apache.org/jira/browse/NUTCH-2182) - Make reverseUrlDirs file dumper option hash the URL for consistency
+- [NUTCH-2183](https://issues.apache.org/jira/browse/NUTCH-2183) - Improvement to SegmentChecker for skipping non-segments present in segments directory
+- [NUTCH-2187](https://issues.apache.org/jira/browse/NUTCH-2187) - Change FileDumper SHAs to all uppercase
+- [NUTCH-2195](https://issues.apache.org/jira/browse/NUTCH-2195) - IndexingFilterChecker to optionally follow N redirects
+- [NUTCH-2196](https://issues.apache.org/jira/browse/NUTCH-2196) - IndexingFilterChecker to optionally normalize
+- [NUTCH-2197](https://issues.apache.org/jira/browse/NUTCH-2197) - Add solr5 solrcloud indexer support
+- [NUTCH-2204](https://issues.apache.org/jira/browse/NUTCH-2204) - Remove junit lib from runtime
+- [NUTCH-2218](https://issues.apache.org/jira/browse/NUTCH-2218) - Switch CrawlCompletion arg parsing to Commons CLI
+- [NUTCH-2221](https://issues.apache.org/jira/browse/NUTCH-2221) - Introduce db.ignore.internal.links to FetcherThread
+- [NUTCH-2229](https://issues.apache.org/jira/browse/NUTCH-2229) - Allow Jexl expressions on CrawlDatum's fixed attributes
+- [NUTCH-2231](https://issues.apache.org/jira/browse/NUTCH-2231) - Jexl support in generator job
+- [NUTCH-2252](https://issues.apache.org/jira/browse/NUTCH-2252) - Allow phantomjs as a browser for selenium options
+- [NUTCH-2263](https://issues.apache.org/jira/browse/NUTCH-2263) - Support for mingram and maxgram at Unigram Cosine Similarity Model
+
+### New Feature
+
+- [NUTCH-961](https://issues.apache.org/jira/browse/NUTCH-961) - Expose Tika's boilerpipe support
+- [NUTCH-1325](https://issues.apache.org/jira/browse/NUTCH-1325) - HostDB for Nutch
+- [NUTCH-2144](https://issues.apache.org/jira/browse/NUTCH-2144) - Plugin to override db.ignore.external to exempt interesting external domain URLs
+- [NUTCH-2190](https://issues.apache.org/jira/browse/NUTCH-2190) - Protocol normalizer
+- [NUTCH-2191](https://issues.apache.org/jira/browse/NUTCH-2191) - Add protocol-htmlunit
+- [NUTCH-2194](https://issues.apache.org/jira/browse/NUTCH-2194) - Run IndexingFilterChecker as simple Telnet server
+- [NUTCH-2219](https://issues.apache.org/jira/browse/NUTCH-2219) - Criteria order to be configurable in DeduplicationJob
+- [NUTCH-2227](https://issues.apache.org/jira/browse/NUTCH-2227) - RegexParseFilter
+- [NUTCH-2245](https://issues.apache.org/jira/browse/NUTCH-2245) - Developed the NGram Model on the existing Unigram Cosine Similarity Model
+
+### Task
+
+- [NUTCH-2201](https://issues.apache.org/jira/browse/NUTCH-2201) - Remove loops program from webgraph package
+- [NUTCH-2211](https://issues.apache.org/jira/browse/NUTCH-2211) - Filter and normalizer checkers missing in bin/nutch
+- [NUTCH-2220](https://issues.apache.org/jira/browse/NUTCH-2220) - Rename db.* options used only by the linkdb to linkdb.*
+
+
+## Nutch 1.11 Release 03/12/2015 (dd/mm/yyyy)
+Release Report: http://s.apache.org/nutch11
+
+- [NUTCH-2176](https://issues.apache.org/jira/browse/NUTCH-2176) - Clean up of log4j.properties (markus)
+
+- [NUTCH-2107](https://issues.apache.org/jira/browse/NUTCH-2107) - plugin.xml to validate against plugin.dtd (snagel)
+
+- [NUTCH-2177](https://issues.apache.org/jira/browse/NUTCH-2177) - Generator produces only one partition even in distributed mode (jnioche, snagel)
+
+- [NUTCH-2158](https://issues.apache.org/jira/browse/NUTCH-2158) - Upgrade to Tika 1.11 (jnioche, snagel)
+
+- [NUTCH-2175](https://issues.apache.org/jira/browse/NUTCH-2175) - Typos in property descriptions in nutch-default.xml (Roannel Fernández Hernández via snagel)
+
+- [NUTCH-2069](https://issues.apache.org/jira/browse/NUTCH-2069) - Ignore external links based on domain (jnioche)
+
+- [NUTCH-2173](https://issues.apache.org/jira/browse/NUTCH-2173) - String.join in FileDumper breaks the build (joyce)
+
+- [NUTCH-2166](https://issues.apache.org/jira/browse/NUTCH-2166) - Add reverse URL format to dump tool (joyce)
+
+- [NUTCH-2157](https://issues.apache.org/jira/browse/NUTCH-2157) - Addressing Miredot REST API Warnings (Sujen Shah)
+
+- [NUTCH-2165](https://issues.apache.org/jira/browse/NUTCH-2165) - FileDumper Util hard codes part-# folder name (joyce)
+
+- [NUTCH-2167](https://issues.apache.org/jira/browse/NUTCH-2167) - Backport TableUtil from 2.x for URL reversing (joyce)
+
+- [NUTCH-2160](https://issues.apache.org/jira/browse/NUTCH-2160) - Upgrade Selenium Java to 2.48.2 (lewismc, kwhitehall)
+
+- [NUTCH-2120](https://issues.apache.org/jira/browse/NUTCH-2120) - Remove MapWritable from trunk codebase (lewismc)
+
+- [NUTCH-1911](https://issues.apache.org/jira/browse/NUTCH-1911) - Improve DomainStatistics tool command line parsing (joyce)
+
+- [NUTCH-2064](https://issues.apache.org/jira/browse/NUTCH-2064) - URLNormalizer basic to encode reserved chars and decode non-reserved chars (markus, snagel)
+
+- [NUTCH-2159](https://issues.apache.org/jira/browse/NUTCH-2159) - Ensure that all WebApp files are copied into generated artifacts for 1.X Webapp (lewismc)
+
+- [NUTCH-2154](https://issues.apache.org/jira/browse/NUTCH-2154) - Nutch REST API (DB) suffering NullPointerException (Aron Ahmadia, Sujen Shah via mattmann)
+
+- [NUTCH-2150](https://issues.apache.org/jira/browse/NUTCH-2150) - Add protocolstats utility (Michael Joyce via mattmann)
+
+- [NUTCH-2146](https://issues.apache.org/jira/browse/NUTCH-2146) - hashCode on the Outlink class (jorgelbg via mattmann)
+
+- [NUTCH-2155](https://issues.apache.org/jira/browse/NUTCH-2155) - Create a "crawl completeness" utility (Michael Joyce via mattmann)
+
+- [NUTCH-1988](https://issues.apache.org/jira/browse/NUTCH-1988) - Make nested output directory dump optional... again (Michael Joyce via lewismc)
+
+- [NUTCH-1800](https://issues.apache.org/jira/browse/NUTCH-1800) - Documentation for Nutch 1.X and 2.X REST APIs (lewismc)
+
+- [NUTCH-2149](https://issues.apache.org/jira/browse/NUTCH-2149) - REST endpoint to read Nutch sequence files (Sujen Shah)
+
+- [NUTCH-2139](https://issues.apache.org/jira/browse/NUTCH-2139) - Basic plugin to index inlinks and outlinks (jorgelbg)
+
+- [NUTCH-2128](https://issues.apache.org/jira/browse/NUTCH-2128) - Review and update mapred --> mapreduce config params in crawl script (lewismc)
+
+- [NUTCH-2141](https://issues.apache.org/jira/browse/NUTCH-2141) - Change the InteractiveSelenium plugin handler Interface to return page content (Balaji Gurumurthy via mattmann)
+
+- [NUTCH-2129](https://issues.apache.org/jira/browse/NUTCH-2129) - Add protocol status tracking to crawl datum (Michael Joyce via mattmann)
+
+- [NUTCH-2142](https://issues.apache.org/jira/browse/NUTCH-2142) - Nutch File Dump - FileNotFoundException (Invalid Argument) Error (Karanjeet Singh via mattmann)
+
+- [NUTCH-2136](https://issues.apache.org/jira/browse/NUTCH-2136) - Implement a different version of Naive Bayes Parse Filter (Asitang Mishra)
+
+- [NUTCH-2109](https://issues.apache.org/jira/browse/NUTCH-2109) - Create a brute force click-all-ajax-links utility fucntion for selenium interactive plugin (Asitang Mishra)
+
+- [NUTCH-2108](https://issues.apache.org/jira/browse/NUTCH-2108) - Add a function to the selenium interactive plugin interface to do multiple manipulation of driver and then return the data (Asitang Mishra)
+
+- [NUTCH-2124](https://issues.apache.org/jira/browse/NUTCH-2124) - Fetcher following same redirect again and again (Yogendra Kumar Soni via snagel)
+
+- [NUTCH-2123](https://issues.apache.org/jira/browse/NUTCH-2123) - Seed List REST API returns Text but headers indicate/require JSON (Aron Ahmadia, Sujen Shah via mattmann)
+
+- [NUTCH-2086](https://issues.apache.org/jira/browse/NUTCH-2086) - Nutch 1.X Webui (Sujen Shah, mattmann via lewismc)
+
+- [NUTCH-2121](https://issues.apache.org/jira/browse/NUTCH-2121) - Update javadoc link for Hadoop 2.4.0 in default.properties (Sujen Shah)
+
[... 2416 lines stripped ...]