You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@stormcrawler.apache.org by dp...@apache.org on 2026/08/18 15:27:12 UTC
(stormcrawler-site) branch refresh updated (7901a99 -> 3ed2bd8)
This is an automated email from the ASF dual-hosted git repository.
dpol1 pushed a change to branch refresh
in repository https://gitbox.apache.org/repos/asf/stormcrawler-site.git
discard 7901a99 Restore the dynamic ASF event banner in the footer
discard 4b0766c Template release versions, restore the full FAQ and clean up page copy
new a079948 Template release versions, restore the full FAQ and clean up page copy
new 3ed2bd8 Restore the dynamic ASF event banner in the footer
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (7901a99)
\
N -- N -- N refs/heads/refresh (3ed2bd8)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
_includes/footer.html | 2 +-
css/main.scss | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
(stormcrawler-site) 02/02: Restore the dynamic ASF event banner in the footer
Posted by dp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dpol1 pushed a commit to branch refresh
in repository https://gitbox.apache.org/repos/asf/stormcrawler-site.git
commit 3ed2bd82f02dbf44eccefa9a319eeb76bf97c3cc
Author: Davide Polato <dp...@apache.org>
AuthorDate: Tue Aug 18 16:31:29 2026 +0200
Restore the dynamic ASF event banner in the footer
The restyle had replaced the acevent element with a static Community
Over Code link. The banner image and target are managed remotely by ASF
Marketing and Publicity via snippet.js, so the footer now carries the
same acevent markup as the previous site and the static pill is gone.
The EVENTS link in the ASF band points to communityovercode.org, as the
old pill did. Also switches the Stack Overflow icon link to https.
---
_includes/footer.html | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/_includes/footer.html b/_includes/footer.html
index 31b0511..098b3a0 100644
--- a/_includes/footer.html
+++ b/_includes/footer.html
@@ -14,17 +14,18 @@
<a class="icon-btn" href="https://github.com/apache/stormcrawler" target="_blank" rel="noopener" title="GitHub" aria-label="Apache StormCrawler on GitHub">
<svg width="18" height="18" viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27s1.36.09 2 .27c1.53-1.04 2.2-.82 2. [...]
</a>
- <a class="icon-btn" href="https://infra.apache.org/slack.html" target="_blank" rel="noopener" title="Slack — #stormcrawler" aria-label="ASF Slack">
+ <a class="icon-btn" href="https://infra.apache.org/slack.html" target="_blank" rel="noopener" title="Slack · #stormcrawler" aria-label="ASF Slack">
<svg width="15" height="15" viewBox="0 0 24 24" fill="currentColor" aria-hidden="true"><path d="M5.04 15.17a2.52 2.52 0 1 1-2.52-2.53h2.52v2.53Zm1.27 0a2.52 2.52 0 0 1 5.05 0v6.31a2.52 2.52 0 1 1-5.05 0v-6.31ZM8.83 5.04a2.52 2.52 0 1 1 2.53-2.52v2.52H8.83Zm0 1.27a2.52 2.52 0 0 1 0 5.05H2.52a2.52 2.52 0 1 1 0-5.05h6.31Zm10.13 2.52a2.52 2.52 0 1 1 2.52 2.53h-2.52V8.83Zm-1.27 0a2.52 2.52 0 0 1-5.05 0V2.52a2.52 2.52 0 1 1 5.05 0v6.31Zm-2.52 10.13a2.52 2.52 0 1 1-2.53 2.52v-2.52h2 [...]
</a>
- <a class="icon-btn" href="http://stackoverflow.com/questions/tagged/stormcrawler" target="_blank" rel="noopener" title="Stack Overflow — tag stormcrawler" aria-label="Stack Overflow">
+ <a class="icon-btn" href="https://stackoverflow.com/questions/tagged/stormcrawler" target="_blank" rel="noopener" title="Stack Overflow · tag stormcrawler" aria-label="Stack Overflow">
<svg width="15" height="15" viewBox="0 0 24 24" fill="currentColor" aria-hidden="true"><path d="M17.4 20.4H6.6v-5H5v6.6h14v-6.6h-1.6v5ZM8.7 14.2h6.6v1.6H8.7v-1.6Zm.2-3 6.4 1.4.3-1.6-6.4-1.3-.3 1.5Zm.9-3.5 6 2.8.6-1.4-6-2.8-.6 1.4Zm1.9-3.2 5 4.2 1-1.2-5-4.2-1 1.2Z"/></svg>
</a>
<a class="icon-btn" href="mailto:dev-subscribe@stormcrawler.apache.org" title="Subscribe to the dev mailing list" aria-label="Subscribe to the dev mailing list">
<svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><rect x="3" y="5" width="18" height="14" rx="2"/><path d="m3 7 9 6 9-6"/></svg>
</a>
- <a class="coc-pill" href="https://communityovercode.org/" target="_blank" rel="noopener"><span class="coc-dot"></span>COMMUNITY OVER CODE ↗</a>
</div>
+ <!-- Current ASF event banner: image and target managed remotely by ASF M&P via snippet.js -->
+ <a class="acevent" data-format="wide" data-mode="dark" target="_blank" rel="noopener"></a>
</div>
<div class="fcol">
<h3>Project</h3>
@@ -54,7 +55,7 @@
<a href="https://www.apache.org/security/" target="_blank" rel="noopener">SECURITY</a>
<a href="https://www.apache.org/foundation/sponsorship" target="_blank" rel="noopener">SPONSORSHIP</a>
<a href="https://www.apache.org/foundation/sponsors" target="_blank" rel="noopener">SPONSORS</a>
- <a href="https://www.apache.org/events/current-event.html" target="_blank" rel="noopener">EVENTS</a>
+ <a href="https://communityovercode.org/" target="_blank" rel="noopener">EVENTS</a>
</span>
</div>
<p class="colophon">© 2026 The Apache Software Foundation · Licensed under the Apache License, Version 2.0<br>Apache StormCrawler, StormCrawler and the Apache logo are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
(stormcrawler-site) 01/02: Template release versions, restore the full FAQ and clean up page copy
Posted by dp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dpol1 pushed a commit to branch refresh
in repository https://gitbox.apache.org/repos/asf/stormcrawler-site.git
commit a079948439b0cf3e0deadb2342b82b32fd723cc6
Author: Davide Polato <dp...@apache.org>
AuthorDate: Tue Aug 18 16:31:15 2026 +0200
Template release versions, restore the full FAQ and clean up page copy
The hardcoded 3.7.0 references on the download and docs pages now come
from site.version and _data/versions.yml, so a release bump is a one
line change. The docs FAQ regains the five entries from the live site
that were lost in the restyle (Storm concepts, why Storm, crawl speed
with its #howfast anchor, Nutch comparison, external storage) and its
questions are prefixed with Q: instead of a shell prompt. The migration
guide is restyled with the current components and linked from the
download page. Em dashes are replaced with middots or rewritten, labels
and prose use full trademarked names, remaining http links move to
https, the PMC contact gains a mailto and the security page gets a CVE
list template for future advisories.
---
contribute/index.html | 8 +++---
css/main.scss | 2 +-
docs/index.html | 44 ++++++++++++++++++++++--------
download/3.0/migration-guide.html | 57 +++++++++++++++++++--------------------
download/index.html | 35 +++++++++++++-----------
security/index.html | 16 +++++++----
support/index.html | 12 ++++-----
7 files changed, 102 insertions(+), 72 deletions(-)
diff --git a/contribute/index.html b/contribute/index.html
index 5d0a864..0a3fab1 100644
--- a/contribute/index.html
+++ b/contribute/index.html
@@ -12,25 +12,25 @@ description: Questions, bug reports and pull requests are all welcome.
<h2>Your thread into the web</h2>
<div class="vchain">
<div class="vlink">
- <p class="slabel">01 — DISCUSS</p>
+ <p class="slabel">01 · DISCUSS</p>
<h3>Join the conversation</h3>
<p>Development discussions happen on the <b>dev</b> mailing list and GitHub discussions.</p>
<div class="chip-cluster"><a class="chip" href="mailto:dev-subscribe@stormcrawler.apache.org">✉ SUBSCRIBE</a><a class="chip" href="https://lists.apache.org/list.html?dev@stormcrawler.apache.org" target="_blank" rel="noopener">ARCHIVE ↗</a><a class="chip" href="https://github.com/apache/stormcrawler/discussions" target="_blank" rel="noopener">DISCUSSIONS ↗</a></div>
</div>
<div class="vlink">
- <p class="slabel">02 — REPORT</p>
+ <p class="slabel">02 · REPORT</p>
<h3>File what you find</h3>
<p>Bug reports and feature requests go to GitHub issues. Please search existing issues before creating new ones.</p>
<div class="chip-cluster"><a class="chip" href="https://github.com/apache/stormcrawler/issues" target="_blank" rel="noopener">ISSUES ↗</a></div>
</div>
<div class="vlink">
- <p class="slabel">03 — PATCH</p>
+ <p class="slabel">03 · PATCH</p>
<h3>Open a pull request</h3>
<p>All code merges require approval from at least one StormCrawler committer.</p>
<div class="chip-cluster"><a class="chip" href="https://github.com/apache/stormcrawler/pulls" target="_blank" rel="noopener">PULL REQUESTS ↗</a></div>
</div>
<div class="vlink">
- <p class="slabel">04 — RELEASE</p>
+ <p class="slabel">04 · RELEASE</p>
<h3>Ship it together</h3>
<p>Releases require that at least three StormCrawler PMC members approve the release.</p>
<div class="chip-cluster"><a class="chip" href="{{ site.baseurl }}/download/">DOWNLOAD →</a></div>
diff --git a/css/main.scss b/css/main.scss
index 8cd3963..92381c3 100644
--- a/css/main.scss
+++ b/css/main.scss
@@ -1307,7 +1307,7 @@ html:not(.no-js) .hero__webbox .hero__mark {
}
.faq .faq__q::before {
- content: "$ ";
+ content: "Q: ";
color: var(--accent-ink);
}
diff --git a/docs/index.html b/docs/index.html
index a62a128..d95f713 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -18,13 +18,13 @@ description: Guides, configuration reference and Javadoc for every release.
<div class="door">
<p class="slabel">TUNING</p>
<h3>Configuration</h3>
- <p>Seeds, politeness, fetch intervals, filters — every knob of the crawl, explained.</p>
+ <p>Seeds, politeness, fetch intervals, filters: every knob of the crawl, explained.</p>
<a class="slink" href="{{ site.baseurl }}/docs/latest/index.html#_configuration">CONFIGURATION <svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7M9 7h8v8"/></svg></a>
</div>
<div class="door">
<p class="slabel">INTEGRATING</p>
<h3>External modules</h3>
- <p><a href="https://opensearch.org/" target="_blank" rel="noopener">OpenSearch®</a>, <a href="https://solr.apache.org/" target="_blank" rel="noopener">Apache Solr®</a>, <a href="https://tika.apache.org/" target="_blank" rel="noopener">Apache Tika®</a>, sitemaps, feeds — plug-in resources maintained by the project.</p>
+ <p><a href="https://opensearch.org/" target="_blank" rel="noopener">OpenSearch®</a>, <a href="https://solr.apache.org/" target="_blank" rel="noopener">Apache Solr®</a>, <a href="https://tika.apache.org/" target="_blank" rel="noopener">Apache Tika®</a>, sitemaps, feeds: plug-in resources maintained by the project.</p>
<a class="slink" href="https://github.com/apache/stormcrawler/tree/main/external" target="_blank" rel="noopener">MODULES <svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7M9 7h8v8"/></svg></a>
</div>
</div>
@@ -32,30 +32,52 @@ description: Guides, configuration reference and Javadoc for every release.
<h2>User Documentation</h2>
<p>Additional documentation, including configuration guides and usage examples, is available in the versioned documentation pages:</p>
<div class="chip-cluster" style="margin-bottom:26px;">
- <a class="chip" href="{{ site.baseurl }}/docs/latest/index.html">LATEST (3.7.0) →</a>
- <a class="chip" href="{{ site.baseurl }}/docs/3.7.0/index.html">3.7.0</a>
- <a class="chip" href="{{ site.baseurl }}/docs/3.6.0/index.html">3.6.0</a>
- <a class="chip" href="{{ site.baseurl }}/docs/3.5.1/index.html">3.5.1</a>
+ <a class="chip" href="{{ site.baseurl }}/docs/latest/index.html">LATEST ({{ site.version }}) →</a>
+ {% for v in site.data.versions.docs %}<a class="chip" href="{{ site.baseurl }}/docs/{{ v }}/index.html">{{ v }}</a>
+ {% endfor %}
</div>
<h2>Javadoc</h2>
<p>The full API reference for Apache StormCrawler on javadoc.io:</p>
<div class="chip-cluster" style="margin-bottom:26px;">
- <a class="chip" href="https://javadoc.io/doc/org.apache.stormcrawler/stormcrawler-core/3.7.0" target="_blank" rel="noopener">3.7.0 (CURRENT) ↗</a>
- <a class="chip" href="https://javadoc.io/doc/org.apache.stormcrawler/stormcrawler-core/3.6.0" target="_blank" rel="noopener">3.6.0 ↗</a>
- <a class="chip" href="https://javadoc.io/doc/org.apache.stormcrawler/stormcrawler-core/3.5.1" target="_blank" rel="noopener">3.5.1 ↗</a>
- <a class="chip" href="https://javadoc.io/doc/org.apache.stormcrawler/stormcrawler-core/3.5.0" target="_blank" rel="noopener">3.5.0 ↗</a>
- <a class="chip" href="https://javadoc.io/doc/org.apache.stormcrawler/stormcrawler-core/3.4.0" target="_blank" rel="noopener">3.4.0 ↗</a>
+ {% for v in site.data.versions.javadoc %}<a class="chip" href="https://javadoc.io/doc/org.apache.stormcrawler/stormcrawler-core/{{ v }}" target="_blank" rel="noopener">{{ v }}{% if v == site.version %} (CURRENT){% endif %} ↗</a>
+ {% endfor %}
</div>
<h2>Presentations</h2>
<p>The <a href="{{ site.baseurl }}/docs/latest/presentations.html">Presentations</a> page contains links to some recent presentations made about this project.</p>
<h2>FAQ</h2>
+<div class="faq">
+ <p class="faq__q">Topologies? Spouts? Bolts? I'm confused!</p>
+ <p class="faq__a">If you're new to these concepts, it's worth starting with <a href="https://storm.apache.org/" target="_blank" rel="noopener">Apache Storm®</a>. The <a href="https://storm.apache.org/releases/current/Tutorial.html" target="_blank" rel="noopener">tutorial</a> and <a href="https://storm.apache.org/releases/current/Concepts.html" target="_blank" rel="noopener">concept pages</a> provide a good introduction. In addition, you can have a look at our own documentation, whi [...]
+</div>
<div class="faq">
<p class="faq__q">Do I need an <a href="https://storm.apache.org/" target="_blank" rel="noopener">Apache Storm®</a> cluster to run StormCrawler?</p>
<p class="faq__a">Not necessarily. StormCrawler can run in local mode, using Storm libraries as dependencies. However, installing <a href="https://storm.apache.org/" target="_blank" rel="noopener">Storm</a> in pseudo-distributed mode is useful if you want to use its UI to monitor topologies.</p>
</div>
+<div class="faq">
+ <p class="faq__q">Why use Apache Storm®?</p>
+ <p class="faq__a">Apache Storm® is a robust, fault-tolerant framework for distributed stream processing. It guarantees data processing, is simple to understand, actively maintained, and licensed under ASF 2.0.</p>
+</div>
+<div class="faq" id="howfast">
+ <p class="faq__q">How fast is StormCrawler?</p>
+ <p class="faq__a">Speed depends on the diversity of hostnames, your politeness settings and execution environment. For example, if you have 1 million URLs from the same host and set a 1-second delay between requests, you can fetch a maximum of ~86,400 pages per day. Actual performance will vary depending on network speed, document size, parsing, and indexing overhead. This is true for any crawler.</p>
+</div>
+<div class="faq">
+ <p class="faq__q">Why choose StormCrawler over Apache Nutch?</p>
+ <p class="faq__a">StormCrawler processes URLs as a continuous stream, indexing them as they are fetched. Nutch uses batch steps, which can slow down as the crawl grows and resources are unevenly used. StormCrawler can handle streaming URLs or low-latency use cases efficiently. It is also more modern, modular, and actively maintained, though Nutch excels in advanced scoring and deduplication.</p>
+ <p class="faq__a">Tutorials comparing both are available here: <a href="https://digitalpebble.blogspot.co.uk/2015/09/index-web-with-aws-cloudsearch.html" target="_blank" rel="noopener">Nutch & SC with CloudSearch</a> and a benchmark study: <a href="https://digitalpebble.blogspot.co.uk/2017/01/the-battle-of-crawlers-apache-nutch-vs.html" target="_blank" rel="noopener">Crawlers benchmark</a>.</p>
+</div>
+<div class="faq">
+ <p class="faq__q">Do I need external storage? Which type?</p>
+ <p class="faq__a">Yes, StormCrawler needs storage for URLs. The type depends on your crawl:</p>
+ <ul>
+ <li><strong>Non-recursive crawls:</strong> Messaging queues like <a href="https://www.rabbitmq.com/" target="_blank" rel="noopener">RabbitMQ</a>, <a href="https://aws.amazon.com/sqs/" target="_blank" rel="noopener">AWS SQS</a>, or <a href="https://kafka.apache.org/" target="_blank" rel="noopener">Apache Kafka®</a> work well. Use a Spout implementation to read from the queue.</li>
+ <li><strong>Recursive crawls:</strong> Use storage with unique keys (e.g., a database) to avoid duplicate URLs. StormCrawler provides external modules for <a href="https://github.com/apache/stormcrawler/tree/main/external" target="_blank" rel="noopener">Apache Solr®, OpenSearch® and SQL</a>.</li>
+ </ul>
+ <p class="faq__a">The modularity of StormCrawler lets you plug in almost any storage backend.</p>
+</div>
<div class="faq">
<p class="faq__q">Is StormCrawler polite?</p>
<p class="faq__a">Yes. It respects the <a href="https://en.wikipedia.org/wiki/Robots.txt" target="_blank" rel="noopener">robots.txt</a> protocol and can be configured with a politeness delay between requests to the same host or domain.</p>
diff --git a/download/3.0/migration-guide.html b/download/3.0/migration-guide.html
index 45f9ff7..7d772b2 100644
--- a/download/3.0/migration-guide.html
+++ b/download/3.0/migration-guide.html
@@ -1,12 +1,12 @@
---
layout: default
slug: migration-guide
-title: Migration Guide
+title: Migration Guide - Apache StormCrawler
+eyebrow: DOWNLOAD
+heading: Migration Guide
+description: Moving from a pre-Apache (DigitalPebble) StormCrawler release to 3.x under the Apache umbrella.
---
-<div class="row row-col">
-<h1>Apache StormCrawler Migration Guide</h1>
-<h2>Introduction</h2>
<p>This guide provides step-by-step instructions for migrating your project from older versions of StormCrawler to the new version under the Apache umbrella. Key changes include updates to the group and artifact IDs, as well as the removal of the Elasticsearch module.</p>
<h2>Group ID and Artifact ID Changes</h2>
@@ -18,62 +18,61 @@ title: Migration Guide
<p>The artifact ID has changed from <code>storm-crawler</code> to <code>stormcrawler</code>.</p>
<h3>Maven Configuration</h3>
-<p>Update your <code>pom.xml</code> to reflect these changes. Below is an example of the updated dependency configuration:</p>
+<p>Update your <code>pom.xml</code> to reflect these changes. Old configuration:</p>
-<div class="code-block">
- <p><strong>Old Configuration:</strong></p>
- <pre><code><dependency>
+<div class="codebox">
+ <pre><dependency>
<groupId>com.digitalpebble.stormcrawler</groupId>
<artifactId>storm-crawler</artifactId>
<version>OLD_VERSION</version>
-</dependency></code></pre>
+</dependency></pre>
</div>
-<div class="code-block">
- <p><strong>New Configuration:</strong></p>
- <pre><code><dependency>
+<p>New configuration:</p>
+
+<div class="codebox">
+ <pre><dependency>
<groupId>org.apache.stormcrawler</groupId>
<artifactId>stormcrawler</artifactId>
- <version>NEW_VERSION</version>
-</dependency></code></pre>
+ <version>{{ site.version }}</version>
+</dependency></pre>
</div>
-<p>Replace <code>OLD_VERSION</code> with the version you are currently using and <code>NEW_VERSION</code> with the latest version of Apache StormCrawler.</p>
+<p>Replace <code>OLD_VERSION</code> with the version you are currently using; the current Apache StormCrawler release is <code>{{ site.version }}</code>.</p>
<h2>Removal of Elasticsearch Module</h2>
<p>The Elasticsearch module has been removed in the latest version of StormCrawler. You have two options to handle this change:</p>
<ol>
- <li><strong>Fork the Elasticsearch Module:</strong> You can fork the Elasticsearch module from the older version of StormCrawler and maintain it independently.</li>
- <li><strong>Migrate to OpenSearch Module:</strong> Alternatively, you can migrate your code to use the OpenSearch module provided by Apache StormCrawler.</li>
+ <li><strong>Fork the Elasticsearch Module:</strong> You can fork the Elasticsearch module from the older version of StormCrawler and maintain it independently.</li>
+ <li><strong>Migrate to OpenSearch Module:</strong> Alternatively, you can migrate your code to use the OpenSearch module provided by Apache StormCrawler.</li>
</ol>
<h3>Forking the Elasticsearch Module</h3>
<ol>
- <li>Clone the repository containing the last version of StormCrawler that includes the Elasticsearch module.</li>
- <li>Copy the Elasticsearch module into your project's repository.</li>
- <li>Update your project's dependencies to include this local version of the Elasticsearch module.</li>
+ <li>Clone the repository containing the last version of StormCrawler that includes the Elasticsearch module.</li>
+ <li>Copy the Elasticsearch module into your project's repository.</li>
+ <li>Update your project's dependencies to include this local version of the Elasticsearch module.</li>
</ol>
<h3>Migrating to OpenSearch Module</h3>
<p>If you choose to migrate to the OpenSearch module, you will need to update your code to use the new module. Here are the steps:</p>
<ol>
- <li>Add the OpenSearch dependency to your <code>pom.xml</code>:</li>
+ <li>Add the OpenSearch dependency to your <code>pom.xml</code>:</li>
</ol>
-<div class="code-block">
- <pre><code><dependency>
+<div class="codebox">
+ <pre><dependency>
<groupId>org.apache.stormcrawler</groupId>
<artifactId>stormcrawler-opensearch</artifactId>
- <version>NEW_VERSION</version>
-</dependency></code></pre>
+ <version>{{ site.version }}</version>
+</dependency></pre>
</div>
<ol start="2">
- <li>Update your code to replace any references to the Elasticsearch module with the corresponding OpenSearch module classes and methods. The API for OpenSearch is similar to Elasticsearch, so the changes should be straightforward.</li>
- <li>Test your application thoroughly to ensure that the migration does not introduce any issues.</li>
+ <li>Update your code to replace any references to the Elasticsearch module with the corresponding OpenSearch module classes and methods. The API for OpenSearch is similar to Elasticsearch, so the changes should be straightforward.</li>
+ <li>Test your application thoroughly to ensure that the migration does not introduce any issues.</li>
</ol>
<h2>Summary</h2>
<p>By following the steps outlined in this guide, you should be able to migrate your project to the latest version of Apache StormCrawler with minimal effort. Ensure you update your Maven dependencies and handle the removal of the Elasticsearch module by either forking it or migrating to the OpenSearch module.</p>
-<p>For any further assistance, reach out to the community via mailing lists or GitHub discussions.</p>
-</div>
\ No newline at end of file
+<p>For any further assistance, reach out to the community via the <a href="https://lists.apache.org/list.html?dev@stormcrawler.apache.org" target="_blank" rel="noopener">mailing lists</a> or <a href="https://github.com/apache/stormcrawler/discussions" target="_blank" rel="noopener">GitHub discussions</a>.</p>
diff --git a/download/index.html b/download/index.html
index e110647..841e306 100644
--- a/download/index.html
+++ b/download/index.html
@@ -7,48 +7,48 @@ heading: Download
description: Verified source releases of StormCrawler, signed by the release managers.
---
-<p>StormCrawler™ software is an open source SDK for building low-latency, scalable web crawlers based on <a href="https://storm.apache.org/" target="_blank" rel="noopener">Apache Storm®</a>. You can <a href="https://www.apache.org/dyn/closer.lua/stormcrawler/stormcrawler-3.7.0/apache-stormcrawler-3.7.0-source-release.tar.gz">download StormCrawler™ software here</a> (latest release: 3.7.0), or pick individual artifacts below.</p>
+<p>StormCrawler™ software is an open source SDK for building low-latency, scalable web crawlers based on <a href="https://storm.apache.org/" target="_blank" rel="noopener">Apache Storm®</a>. You can <a href="https://www.apache.org/dyn/closer.lua/stormcrawler/stormcrawler-{{ site.version }}/apache-stormcrawler-{{ site.version }}-source-release.tar.gz">download StormCrawler™ software here</a> (latest release: {{ site.version }}), or pick individual artifacts below.</p>
<div class="release-card">
- <h3>Apache StormCrawler 3.7.0 <span class="badge">LATEST</span></h3>
+ <h3>Apache StormCrawler {{ site.version }} <span class="badge">LATEST</span></h3>
<p style="margin:0;">Source release, signed and checksummed:</p>
<div class="chip-cluster">
- <a class="chip" href="https://www.apache.org/dyn/closer.lua/stormcrawler/stormcrawler-3.7.0/apache-stormcrawler-3.7.0-source-release.tar.gz">SOURCE TAR.GZ</a>
- <a class="chip" href="https://github.com/apache/stormcrawler/releases/tag/stormcrawler-3.7.0" target="_blank" rel="noopener">RELEASE NOTES ↗</a>
+ <a class="chip" href="https://www.apache.org/dyn/closer.lua/stormcrawler/stormcrawler-{{ site.version }}/apache-stormcrawler-{{ site.version }}-source-release.tar.gz">SOURCE TAR.GZ</a>
+ <a class="chip" href="https://github.com/apache/stormcrawler/releases/tag/stormcrawler-{{ site.version }}" target="_blank" rel="noopener">RELEASE NOTES ↗</a>
<a class="chip" href="https://search.maven.org/search?q=org.apache.stormcrawler" target="_blank" rel="noopener">MAVEN CENTRAL ↗</a>
</div>
</div>
<h2>Chain of trust</h2>
-<p>You <b>must</b> <a href="https://www.apache.org/info/verification.html">verify</a> the integrity of the downloaded files — every release ships with everything you need to prove it came from us. The chain has four links:</p>
+<p>You <b>must</b> <a href="https://www.apache.org/info/verification.html">verify</a> the integrity of the downloaded files: every release ships with everything you need to prove it came from us. The chain has four links:</p>
<div class="vchain">
<div class="vlink">
- <p class="slabel">01 — GET</p>
+ <p class="slabel">01 · GET</p>
<h3>Source archive</h3>
<p>Fetch the tar.gz from an Apache mirror.</p>
<div class="chip-cluster">
- <a class="chip" href="https://www.apache.org/dyn/closer.lua/stormcrawler/stormcrawler-3.7.0/apache-stormcrawler-3.7.0-source-release.tar.gz">TAR.GZ</a>
+ <a class="chip" href="https://www.apache.org/dyn/closer.lua/stormcrawler/stormcrawler-{{ site.version }}/apache-stormcrawler-{{ site.version }}-source-release.tar.gz">TAR.GZ</a>
</div>
</div>
<div class="vlink">
- <p class="slabel">02 — CHECK</p>
+ <p class="slabel">02 · CHECK</p>
<h3>Integrity</h3>
<p>Your SHA-512 checksum must match ours.</p>
<div class="chip-cluster">
- <a class="chip" href="https://downloads.apache.org/stormcrawler/stormcrawler-3.7.0/apache-stormcrawler-3.7.0-source-release.tar.gz.sha512">SHA512</a>
+ <a class="chip" href="https://downloads.apache.org/stormcrawler/stormcrawler-{{ site.version }}/apache-stormcrawler-{{ site.version }}-source-release.tar.gz.sha512">SHA512</a>
</div>
</div>
<div class="vlink">
- <p class="slabel">03 — PROVE</p>
+ <p class="slabel">03 · PROVE</p>
<h3>Authenticity</h3>
<p>The OpenPGP signature (*.asc) is made by a release manager.</p>
<div class="chip-cluster">
- <a class="chip" href="https://downloads.apache.org/stormcrawler/stormcrawler-3.7.0/apache-stormcrawler-3.7.0-source-release.tar.gz.asc">ASC</a>
+ <a class="chip" href="https://downloads.apache.org/stormcrawler/stormcrawler-{{ site.version }}/apache-stormcrawler-{{ site.version }}-source-release.tar.gz.asc">ASC</a>
</div>
</div>
<div class="vlink">
- <p class="slabel">04 — TRUST</p>
+ <p class="slabel">04 · TRUST</p>
<h3>Key ring</h3>
<p>Match the signature against the project's KEYS file.</p>
<div class="chip-cluster">
@@ -59,10 +59,10 @@ description: Verified source releases of StormCrawler, signed by the release man
<h3>Verify like a release manager</h3>
<div class="codebox">
- <pre id="verify-cmd">$ sha512sum -c apache-stormcrawler-3.7.0-source-release.tar.gz.sha512
+ <pre id="verify-cmd">$ sha512sum -c apache-stormcrawler-{{ site.version }}-source-release.tar.gz.sha512
$ gpg --import KEYS
-$ gpg --verify apache-stormcrawler-3.7.0-source-release.tar.gz.asc</pre>
- <button class="copy-chip" data-copy="#verify-cmd" hidden style="position:absolute;top:10px;right:10px;">copy</button>
+$ gpg --verify apache-stormcrawler-{{ site.version }}-source-release.tar.gz.asc</pre>
+ <button class="copy-chip" data-copy="#verify-cmd" hidden>copy</button>
</div>
<p>More information about release signing and verifying signatures can be found <a href="https://www.apache.org/dev/release-signing.html">here</a>.</p>
@@ -71,4 +71,7 @@ $ gpg --verify apache-stormcrawler-3.7.0-source-release.tar.gz.asc</pre>
<p>Artifacts are available via <a href="https://search.maven.org/search?q=org.apache.stormcrawler">Maven Central</a>.</p>
<h3>Older Releases</h3>
-<p>Previous <b>non</b>-ASF releases can be found <a href="https://search.maven.org/search?q=g:com.digitalpebble.stormcrawler">here</a>.</p>
+<p>Earlier ASF releases are listed on the <a href="https://github.com/apache/stormcrawler/releases" target="_blank" rel="noopener">GitHub releases page</a> and remain available from the <a href="https://archive.apache.org/dist/stormcrawler/" target="_blank" rel="noopener">Apache archive</a>. Previous <b>non</b>-ASF releases (group <code>com.digitalpebble.stormcrawler</code>) can be found <a href="https://search.maven.org/search?q=g:com.digitalpebble.stormcrawler" target="_blank" rel="noo [...]
+
+<h3>Migrating from 2.x</h3>
+<p>Coming from a pre-Apache (DigitalPebble) release? The <a href="{{ site.baseurl }}/download/3.0/migration-guide.html">migration guide</a> covers the new Maven coordinates, package renames and removed modules.</p>
diff --git a/security/index.html b/security/index.html
index 829a7c0..e2edfb1 100644
--- a/security/index.html
+++ b/security/index.html
@@ -8,24 +8,24 @@ description: How to report vulnerabilities privately, and the StormCrawler threa
---
<div class="callout callout--alert">
- <p><b>Found a vulnerability?</b> Report it privately to <a href="mailto:security@apache.org">security@apache.org</a> — never in a public forum. The security list is only for undisclosed vulnerabilities and managing their fixes; regular bug reports and other queries cannot be accepted at this address.</p>
+ <p><b>Found a vulnerability?</b> Report it privately to <a href="mailto:security@apache.org">security@apache.org</a>, never in a public forum. The security list is only for undisclosed vulnerabilities and managing their fixes; regular bug reports and other queries cannot be accepted at this address.</p>
</div>
<h2>How disclosure works</h2>
<p>The Apache Software Foundation takes a very active stance in eliminating security problems and denial of service attacks against its products. Every report follows the <a href="https://www.apache.org/security/" target="_blank" rel="noopener">ASF security process</a>:</p>
<div class="vchain vchain--3">
<div class="vlink">
- <p class="slabel">01 — REPORT</p>
+ <p class="slabel">01 · REPORT</p>
<h3>Privately</h3>
<p>You write to the ASF Security Team; the report stays confidential.</p>
</div>
<div class="vlink">
- <p class="slabel">02 — FIX</p>
+ <p class="slabel">02 · FIX</p>
<h3>Quietly</h3>
<p>The PMC develops, reviews and releases a fix before any disclosure.</p>
</div>
<div class="vlink">
- <p class="slabel">03 — DISCLOSE</p>
+ <p class="slabel">03 · DISCLOSE</p>
<h3>Publicly</h3>
<p>The vulnerability is announced once a fixed release is available.</p>
</div>
@@ -49,4 +49,10 @@ description: How to report vulnerabilities privately, and the StormCrawler threa
<p>Questions about whether a vulnerability applies to your application, further information on a published vulnerability, or availability of patches should be addressed to the <a href="https://lists.apache.org/list.html?dev@stormcrawler.apache.org" target="_blank" rel="noopener">dev mailing list</a> (<a href="mailto:dev-subscribe@stormcrawler.apache.org">subscribe</a> · <a href="mailto:dev-unsubscribe@stormcrawler.apache.org">unsubscribe</a>).</p>
<h2>Known Security Vulnerabilities</h2>
-<p>No known security vulnerability yet.</p>
+<p>No security vulnerabilities have been disclosed for Apache StormCrawler to date. When one is published, it will be listed here with its CVE identifier, the affected versions and the fixed release.</p>
+<!-- CVE list template — uncomment and fill in when the first advisory lands:
+<div class="faq">
+ <p class="faq__q">CVE-XXXX-XXXXX — short title</p>
+ <p class="faq__a">Affects: x.y.z and earlier · Fixed in: x.y.z · <a href="https://www.cve.org/CVERecord?id=CVE-XXXX-XXXXX">advisory</a></p>
+</div>
+-->
diff --git a/support/index.html b/support/index.html
index 4ac77c2..1dce5fd 100644
--- a/support/index.html
+++ b/support/index.html
@@ -9,24 +9,24 @@ description: Community help on GitHub and the mailing lists, plus commercial sup
<div class="lanes">
<div class="lane">
- <p class="slabel">COMMUNITY — FREE</p>
+ <p class="slabel">COMMUNITY · FREE</p>
<h3>Ask the community</h3>
<p>Questions go to GitHub discussions or Stack Overflow (tag 'stormcrawler'); bugs to GitHub issues. There is also a <b>#stormcrawler</b> channel on the ASF Slack: committers join directly with their @apache.org address, everyone else can ask for a guest invite on the <a href="mailto:dev@stormcrawler.apache.org">dev mailing list</a> (<a href="mailto:dev-subscribe@stormcrawler.apache.org">subscribe</a>).</p>
<div class="chip-cluster" style="margin-top:14px;">
<a class="chip" href="https://github.com/apache/stormcrawler/discussions" target="_blank" rel="noopener">DISCUSSIONS ↗</a>
- <a class="chip" href="http://stackoverflow.com/questions/tagged/stormcrawler" target="_blank" rel="noopener">STACK OVERFLOW ↗</a>
+ <a class="chip" href="https://stackoverflow.com/questions/tagged/stormcrawler" target="_blank" rel="noopener">STACK OVERFLOW ↗</a>
<a class="chip" href="https://github.com/apache/stormcrawler/issues" target="_blank" rel="noopener">ISSUES ↗</a>
<a class="chip" href="https://infra.apache.org/slack.html" target="_blank" rel="noopener">ASF SLACK ↗</a>
</div>
</div>
<div class="lane">
- <p class="slabel">COMMERCIAL — PAID HELP</p>
+ <p class="slabel">COMMERCIAL · PAID HELP</p>
<h3>Hire an expert</h3>
- <p>The companies below offer paid consulting, custom development and production support for StormCrawler deployments — for when you need guaranteed response times or someone to build and run the crawler for you.</p>
- <p style="margin-top:12px;"><a href="http://digitalpebble.com" target="_blank" rel="noopener">DigitalPebble Ltd:</a> Commercial Support and Consulting Services</p>
+ <p>The companies below offer paid consulting, custom development and production support for StormCrawler deployments, for when you need guaranteed response times or someone to build and run the crawler for you.</p>
+ <p style="margin-top:12px;"><a href="https://digitalpebble.com" target="_blank" rel="noopener">DigitalPebble Ltd:</a> Commercial Support and Consulting Services</p>
<p style="margin-top:12px;font-size:12.5px;color:var(--muted);">The Apache StormCrawler PMC does not endorse or recommend any of the products or services on this page. We love all our supporters equally.</p>
</div>
</div>
<h2>Want to be added to this page?</h2>
-<p>All submitted information must be factual and informational in nature and not be a marketing statement. Statements that promote your products and services over other offerings on the page will not be tolerated and will be removed. When in doubt, email the Apache StormCrawler PMC and ask. We are happy to help.</p>
+<p>All submitted information must be factual and informational in nature and not be a marketing statement. Statements that promote your products and services over other offerings on the page will not be tolerated and will be removed. When in doubt, email the <a href="mailto:private@stormcrawler.apache.org">Apache StormCrawler PMC</a> and ask. We are happy to help.</p>