You are viewing a plain text version of this content. The canonical link for it is here.
- Re: [VOTE] Apache StormCrawler 3.7.0 Release Candidate - posted by Davide Polato <dp...@apache.org> on 2026/08/01 07:59:48 UTC, 0 replies.
- [RESULT] [VOTE] Apache StormCrawler 3.7.0 - posted by Davide Polato <dp...@apache.org> on 2026/08/01 15:58:41 UTC, 0 replies.
- [PR] Release Apache StormCrawler 3.7.0 [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/08/01 16:15:19 UTC, 1 replies.
- [PR] Release 3.7.0 [stormcrawler-site] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/08/01 16:23:13 UTC, 1 replies.
- Re: [PR] Fix typo and broken link [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/01 16:27:33 UTC, 0 replies.
- Re: [PR] Fix broken anchor in Asciidoc [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/01 16:27:44 UTC, 0 replies.
- [ANNOUNCE] Apache StormCrawler 3.7.0 released - posted by Davide Polato <dp...@apache.org> on 2026/08/02 07:42:00 UTC, 0 replies.
- Re: [PR] Upgrade to Apache Storm 3.0.0 and Java 25 [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/08/02 08:09:34 UTC, 1 replies.
- Re: [I] Delete redirected pages [stormcrawler] - posted by "tomatotomata (via GitHub)" <gi...@apache.org> on 2026/08/02 16:06:07 UTC, 0 replies.
- [PR] build(deps): bump com.mysql:mysql-connector-j from 9.7.0 to 26.7.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/03 04:03:24 UTC, 1 replies.
- [PR] build(deps): bump software.amazon.awssdk:bom from 2.49.3 to 2.50.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/03 04:03:50 UTC, 1 replies.
- [PR] build(deps): bump org.jsoup:jsoup from 1.22.2 to 1.23.1 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/03 04:03:57 UTC, 1 replies.
- [PR] build(deps): bump langchain4j.version from 1.18.0 to 1.18.1 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/03 04:04:04 UTC, 1 replies.
- [PR] build(deps): bump commons-codec:commons-codec from 1.22.0 to 1.22.1 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/03 04:04:13 UTC, 1 replies.
- [PR] build(deps): bump actions/setup-java from 5.6.0 to 5.7.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/03 14:49:03 UTC, 1 replies.
- [GH] (stormcrawler/dependabot/github_actions/actions/setup-java-5.7.0): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/08/03 18:33:51 UTC, 0 replies.
- [GH] (stormcrawler/dependabot/github_actions/actions/setup-java-5.7.0): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/08/05 13:11:11 UTC, 0 replies.
- [GH] (stormcrawler/main): Workflow run "CI Main - Regenerate License" failed! - posted by GitBox <gi...@apache.org> on 2026/08/06 09:56:17 UTC, 1 replies.
- [PR] Regenerated License file after dependency upgrades [stormcrawler] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/08/06 09:56:33 UTC, 14 replies.
- [GH] (stormcrawler/main): Workflow run "CI Main - Regenerate License" is working again! - posted by GitBox <gi...@apache.org> on 2026/08/06 09:56:41 UTC, 1 replies.
- Re: [I] Upgrade to Apache Storm 3.0.0 [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/06 09:57:29 UTC, 0 replies.
- [PR] Upgrade to Storm 3 [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/06 10:16:51 UTC, 1 replies.
- Re: [PR] Bump git-code-format-maven-plugin.version from 5.4 to 6.0 [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/08/06 14:32:26 UTC, 1 replies.
- [GH] (stormcrawler/main): Workflow run "Publish SNAPSHOTs" failed! - posted by GitBox <gi...@apache.org> on 2026/08/07 01:26:20 UTC, 10 replies.
- [PR] refresh NOTICE copyright year to 2026 [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/08/07 14:52:50 UTC, 1 replies.
- [PR] build(deps): bump com.microsoft.playwright:playwright from 1.61.0 to 1.62.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/10 04:02:43 UTC, 1 replies.
- [PR] build(deps): bump software.amazon.awssdk:bom from 2.50.0 to 2.51.2 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/10 04:03:25 UTC, 1 replies.
- Re: Feedback on Website Refresh - posted by Davide Polato <dp...@apache.org> on 2026/08/10 09:16:27 UTC, 7 replies.
- [PR] Fix deletion of redirected URLs [stormcrawler] - posted by "Atmuri-SatyaPrakash (via GitHub)" <gi...@apache.org> on 2026/08/10 16:12:29 UTC, 10 replies.
- [GH] (stormcrawler/fix-delete-redirected-pages): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/08/11 10:16:19 UTC, 6 replies.
- [GH] (stormcrawler/fix-delete-redirected-pages): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/08/11 17:40:06 UTC, 2 replies.
- Re: [PR] #1998 WARC writer: WARC-Protocol header to follow WARC field proposals [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/08/12 15:06:27 UTC, 1 replies.
- [PR] [Issue - 315] Adaptative URL filter to normalize URLs based on canonical tag [stormcrawler] - posted by "akash-manna-sky (via GitHub)" <gi...@apache.org> on 2026/08/16 12:02:34 UTC, 0 replies.
- Re: [PR] #315 Adaptative URL filter to normalize URLs based on canonical tag [stormcrawler] - posted by "akash-manna-sky (via GitHub)" <gi...@apache.org> on 2026/08/16 12:05:10 UTC, 3 replies.
- [PR] build(deps): bump selenium.version from 4.46.0 to 4.47.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/17 04:02:48 UTC, 3 replies.
- [PR] build(deps): bump junit.version from 6.1.2 to 6.1.3 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/17 04:03:11 UTC, 1 replies.
- [PR] build(deps): bump software.amazon.awssdk:bom from 2.51.2 to 2.53.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/17 04:03:18 UTC, 1 replies.
- [PR] bugfix set Java 25 for snapshots workflow [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/08/17 10:01:05 UTC, 1 replies.
- [I] Refactor FetcherBolt [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/08/17 12:34:01 UTC, 2 replies.
- [I] fetcher.timeout.queue silently drops URLs [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/08/17 12:34:35 UTC, 2 replies.
- [PR] Remove Selenium module [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/08/17 13:45:04 UTC, 1 replies.
- Re: [PR] build(deps): bump git-code-format-maven-plugin.version from 5.4 to 6.0 [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/17 18:22:27 UTC, 0 replies.
- [GH] (stormcrawler/main): Workflow run "Publish SNAPSHOTs" is working again! - posted by GitBox <gi...@apache.org> on 2026/08/18 00:24:05 UTC, 0 replies.
- [PR] Expand security model with network-facing threat model [stormcrawler-site] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/18 17:52:02 UTC, 1 replies.
- [GH] (stormcrawler/issue-315): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/08/19 11:51:41 UTC, 0 replies.
- [I] [Improvement] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/08/19 14:52:50 UTC, 1 replies.
- [I] Upgrade to Tika 4.0 [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/08/22 08:59:08 UTC, 1 replies.
- [PR] build(deps): bump langchain4j.version from 1.18.1 to 1.19.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/24 04:06:29 UTC, 1 replies.
- [PR] build(deps): bump okhttp.version from 5.4.0 to 5.5.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/24 04:06:41 UTC, 1 replies.
- [PR] build(deps): bump tika.version from 3.3.2 to 4.0.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/24 04:07:01 UTC, 3 replies.
- [PR] build(deps): bump com.google.guava:guava from 33.6.0-jre to 33.7.1-jre [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/24 04:07:11 UTC, 1 replies.
- [PR] build(deps): bump software.amazon.awssdk:bom from 2.53.0 to 2.54.1 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/24 04:07:16 UTC, 1 replies.
- [PR] build(deps): bump com.puppycrawl.tools:checkstyle from 12.3.1 to 14.0.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/24 04:07:27 UTC, 1 replies.
- [GH] (stormcrawler/dependabot/maven/tika.version-4.0.0): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/08/24 04:11:51 UTC, 0 replies.
- [GH] (stormcrawler/dependabot/maven/langchain4j.version-1.19.0): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/08/24 04:24:10 UTC, 0 replies.
- Re: [D] Deprecate Selenium module? [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/08/25 06:53:20 UTC, 0 replies.
- [PR] Removed Apache http protocol implementation [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/08/25 07:13:07 UTC, 1 replies.
- [PR] Say when a defect leads to an advisory and when it is hardening [stormcrawler-site] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/25 18:40:55 UTC, 5 replies.
- [D] Proposal & Feedback: Native Integration with OpenCrawling for High-Throughput RAG & AI Vector Ingestion [stormcrawler] - posted by "OpenPj (via GitHub)" <gi...@apache.org> on 2026/08/26 14:21:06 UTC, 1 replies.
- [PR] Restyle the site and the versioned docs [stormcrawler-site] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/08/27 09:29:14 UTC, 3 replies.
- [PR] Honour ignoreOutsideDomain when set on its own in HostURLFilter [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/27 12:25:11 UTC, 3 replies.
- [PR] Scope cookies to the host that set them [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/27 12:26:29 UTC, 5 replies.
- [PR] Escape the metadata column of the SQL status store [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/27 12:26:51 UTC, 2 replies.
- [PR] Only follow robots.txt redirects on the same host [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/27 12:27:57 UTC, 4 replies.
- [GH] (stormcrawler/fix/hosturlfilter-domain-config): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/08/27 13:03:58 UTC, 0 replies.
- [PR] build(deps): bump actions/setup-java from 5.7.0 to 6.0.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/27 13:48:25 UTC, 1 replies.
- [GH] (stormcrawler/dependabot/github_actions/actions/setup-java-6.0.0): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/08/27 14:08:35 UTC, 1 replies.
- [GH] (stormcrawler/dependabot/github_actions/actions/setup-java-6.0.0): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/08/28 07:39:42 UTC, 0 replies.
- [GH] (stormcrawler/feature/store-request-url): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/08/28 15:32:33 UTC, 0 replies.
- [I] Credentials are sent over connections the okhttp protocol never authenticated, and the trust-all path uses an "SSL" context with an always-true hostname verifier [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:26:55 UTC, 1 replies.
- [I] Nothing in the shipped configuration keeps the fetcher off private address space [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:10 UTC, 1 replies.
- [I] The file scheme is enabled by default and FileProtocol reads any path the worker can open [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:12 UTC, 1 replies.
- [I] Politeness queues and the robots.txt cache key on the raw host string, so percent-escaped host aliases get their own queue and their own robots.txt fetch [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:14 UTC, 1 replies.
- [I] SiteMapParserBolt sniffs every document, parses sitemaps non-strict, and turns a failed sitemap parse into a terminal ERROR [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:15 UTC, 1 replies.
- [I] URLFilters.filter() returns the URL as accepted when a filter in the chain throws [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:17 UTC, 1 replies.
- [I] Persistence spouts emit stored URLs without checking the scheme [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:19 UTC, 1 replies.
- [I] IPFilterRules has no keyword for link-local addresses and the documented example leaves them allowed [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:20 UTC, 1 replies.
- [I] The robots.txt fetch replaces the configured http.content.limit with no limit at all [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:22 UTC, 1 replies.
- [I] Redirect hops taken inside okhttp are not run through the URL filters [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:31 UTC, 1 replies.
- [I] http.basicauth credentials are attached to every request, whatever the host [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:34 UTC, 0 replies.
- [I] A response header named set-header becomes a request header directive [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:36 UTC, 0 replies.
- [I] Response headers can populate the reserved verbatim request and response metadata keys [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:37 UTC, 0 replies.
- [I] CharsetIdentification retries the meta charset lookup ten bytes at a time by recursing, and the call sits outside JSoupParserBolt's catch block [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:38 UTC, 0 replies.
- [I] AbstractLLMTextExtractor pastes page HTML into the prompt unescaped and returns the reply unchecked [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:40 UTC, 0 replies.
- [I] Playwright browser context stops validating certificates as soon as a proxy is configured [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:42 UTC, 0 replies.
- [I] Playwright protocol applies neither the IP filter nor the URL filters to the requests it makes [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:44 UTC, 0 replies.
- [I] SQL IndexerBolt interpolates metadata labels into the statement as unquoted identifiers [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:46 UTC, 0 replies.
- [I] Tika ParserBolt puts no limit on the extracted text and no time limit on the parse [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:47 UTC, 0 replies.
- [I] URLFrontier channels are always plaintext and the module has no TLS option [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:57 UTC, 0 replies.
- [I] Response headers named metrics.* are fed straight into the fetcher metric registry, and a non-numeric one fails the fetch [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:27:59 UTC, 0 replies.
- [I] Gzipped sitemap files are expanded with no size bound, and SiteMapParserBolt reads Content-Type under a key that is never present [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:28:00 UTC, 0 replies.
- [I] HostURLFilter memoizes the source host in plain fields shared by all fetcher threads [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:28:02 UTC, 0 replies.
- [I] SCProxy.toString() rebuilds the proxy URL with user and password, and the protocol logs it [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:28:04 UTC, 0 replies.
- [I] OpenSearch addresses default to http and Basic credentials are registered for any host [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:28:06 UTC, 0 replies.
- [I] Tika mimetype whitelist is checked against the HTTP header while the parser is chosen from the bytes [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:28:07 UTC, 1 replies.
- [I] WARC metadata records write metadata values into warc-fields without escaping CR LF [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:28:09 UTC, 2 replies.
- [I] WARC digest algorithm is hard coded to SHA-1 with no way to choose another [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:28:11 UTC, 2 replies.
- [I] CommaSeparatedToMultivaluedMetadata appends one token at a time, which is quadratic [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:28:11 UTC, 1 replies.
- [I] ParseResult.get() inserts an entry for a URL that was never parsed [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/08/29 11:28:14 UTC, 1 replies.
- [PR] Escape CR and LF in WARC metadata records and resource Content-Type [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/08/29 16:51:31 UTC, 0 replies.
- [PR] Make the WARC digest algorithm configurable [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/08/29 17:41:41 UTC, 0 replies.
- [PR] CommaSeparatedToMultivaluedMetadata: replace quadratic token append with a single bulk write [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/08/29 18:34:58 UTC, 0 replies.
- [PR] ParseResult: non-mutating lookup, explicit getOrCreate() and no empty documents emitted [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/08/29 19:50:13 UTC, 0 replies.
- [PR] build(deps): bump software.amazon.awssdk:bom from 2.54.1 to 2.54.6 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/31 04:02:48 UTC, 1 replies.
- [PR] build(deps): bump org.jsoup:jsoup from 1.23.1 to 1.23.2 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/08/31 04:03:07 UTC, 1 replies.