You are viewing a plain text version of this content. The canonical link for it is here.
- [PR] feat: implement MIME type whitelist filtering in ParserBolt for Tika-… [stormcrawler] - posted by "Riddhish1 (via GitHub)" <gi...@apache.org> on 2026/09/01 05:21:15 UTC, 6 replies.
- [GH] (stormcrawler/fix/tika-whitelist-detected-mimetype): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/01 07:02:07 UTC, 3 replies.
- Re: [PR] #315 Adaptative URL filter to normalize URLs based on canonical tag [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/01 07:09:18 UTC, 1 replies.
- Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/01 07:12:12 UTC, 13 replies.
- Re: [PR] CommaSeparatedToMultivaluedMetadata: replace quadratic token append with a single bulk write [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/01 07:14:04 UTC, 4 replies.
- Re: [PR] ParseResult: non-mutating lookup, explicit getOrCreate() and no empty documents emitted [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/01 07:17:45 UTC, 6 replies.
- Re: [PR] Regenerated License file after dependency upgrades [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/01 07:18:51 UTC, 22 replies.
- [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/01 07:18:51 UTC, 20 replies.
- [GH] (stormcrawler/issue-2107-quadratic-comma-split): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/01 07:29:34 UTC, 1 replies.
- Re: [PR] Escape CR and LF in WARC metadata records and resource Content-Type [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/01 07:50:00 UTC, 6 replies.
- [GH] (stormcrawler/warc-configurable-digest-algorithm): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/01 07:54:44 UTC, 1 replies.
- [GH] (stormcrawler/fix/warc-crlf-escaping-2105): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/01 08:30:36 UTC, 0 replies.
- Re: [PR] #108 Normalize misspelled HTTP headers [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/01 14:53:58 UTC, 2 replies.
- Re: [I] WARC metadata records write metadata values into warc-fields without escaping CR LF [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/01 16:08:32 UTC, 0 replies.
- Re: [I] CommaSeparatedToMultivaluedMetadata appends one token at a time, which is quadratic [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/01 16:32:56 UTC, 0 replies.
- [GH] (stormcrawler/issue-2108-parse-result-lookup): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/02 15:56:26 UTC, 0 replies.
- [GH] (stormcrawler/issue-2108-parse-result-lookup): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/02 18:01:09 UTC, 0 replies.
- [PR] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/02 22:06:03 UTC, 9 replies.
- [PR] Upgrade to Tika 4.0.0 [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/03 05:54:57 UTC, 11 replies.
- Re: [I] ParseResult.get() inserts an entry for a URL that was never parsed [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/03 07:17:46 UTC, 0 replies.
- [GH] (stormcrawler/issue-2063-tika-4): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/03 19:30:57 UTC, 0 replies.
- [GH] (stormcrawler/issue-2063-tika-4): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/03 19:45:43 UTC, 1 replies.
- Re: [PR] Restyle the site and the versioned docs [stormcrawler-site] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/04 13:24:44 UTC, 2 replies.
- [PR] IPFilterRules: add anylocal and multicast keywords, fail on unparseable rules [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/05 22:22:06 UTC, 7 replies.
- [PR] URLFilters: treat an exception from a filter as a rejection [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/05 22:33:22 UTC, 7 replies.
- [PR] Robots.txt fetch no longer replaces the configured content limit [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/05 22:37:49 UTC, 5 replies.
- [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/05 22:43:10 UTC, 18 replies.
- [PR] Spouts check the scheme of stored URLs before emitting [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/05 22:47:05 UTC, 15 replies.
- [PR] File scheme off by default; reads confined to file.protocol.root [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/05 22:51:12 UTC, 8 replies.
- [PR] SiteMapParserBolt: strict parsing, opt-in sniffing, non-terminal parse failure [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/05 23:02:58 UTC, 12 replies.
- [PR] OKHttp: authenticated TLS by default, credentials withheld from unauthenticated servers [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/05 23:16:54 UTC, 9 replies.
- [PR] Keep the fetcher off private address space by default [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/05 23:29:42 UTC, 9 replies.
- [PR] Run redirect hops through the URL filters [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/06 07:43:53 UTC, 11 replies.
- [I] FetcherBolt: global lock and linear scan in FetchItemQueues stall the executor with many hosts [stormcrawler] - posted by "GGraziadei (via GitHub)" <gi...@apache.org> on 2026/09/06 13:13:38 UTC, 1 replies.
- [I] MetadataTransfer: avoid an intermediate key set per wildcard for every outlink [stormcrawler] - posted by "GGraziadei (via GitHub)" <gi...@apache.org> on 2026/09/06 13:13:40 UTC, 1 replies.
- [I] Enable Storm dynamic producer batching in the archetype configurations [stormcrawler] - posted by "GGraziadei (via GitHub)" <gi...@apache.org> on 2026/09/06 13:27:51 UTC, 0 replies.
- [PR] #2129 FetcherBolt: lock-free fetch queues ordered by next fetch time [stormcrawler] - posted by "GGraziadei (via GitHub)" <gi...@apache.org> on 2026/09/06 13:32:17 UTC, 9 replies.
- Re: [I] WARC digest algorithm is hard coded to SHA-1 with no way to choose another [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/06 13:40:09 UTC, 0 replies.
- [PR] #2130 MetadataTransfer: single pass over the metadata for wildcard keys [stormcrawler] - posted by "GGraziadei (via GitHub)" <gi...@apache.org> on 2026/09/06 13:42:09 UTC, 4 replies.
- [GH] (stormcrawler/perf/fetch-queues-lock-free): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/06 13:44:52 UTC, 0 replies.
- [GH] (stormcrawler/issue-2085-spout-scheme-check): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/06 14:06:56 UTC, 2 replies.
- [GH] (stormcrawler/issue-2084-urlfilters-exception): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/06 14:06:59 UTC, 1 replies.
- [GH] (stormcrawler/issue-2080-private-address-default): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/06 14:07:05 UTC, 4 replies.
- [GH] (stormcrawler/issue-2082-host-canonicalisation): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/06 14:07:07 UTC, 3 replies.
- [GH] (stormcrawler/issue-2062-urlfrontier-batch-discovered): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/06 14:07:13 UTC, 1 replies.
- [GH] (stormcrawler/issue-2081-file-protocol-root): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/06 14:07:14 UTC, 1 replies.
- [GH] (stormcrawler/issue-2083-sitemapparser-strict): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/06 14:07:24 UTC, 2 replies.
- [GH] (stormcrawler/issue-2088-redirect-filtering): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/06 14:07:49 UTC, 2 replies.
- [GH] (stormcrawler/issue-2086-ipfilter-linklocal): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/06 14:18:15 UTC, 0 replies.
- [GH] (stormcrawler/fix/108-spellchecked-http-headers): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/06 16:45:36 UTC, 0 replies.
- [I] fetcher.thread.timeout: cancel the okhttp call instead of interrupting a helper thread [stormcrawler] - posted by "GGraziadei (via GitHub)" <gi...@apache.org> on 2026/09/06 20:25:35 UTC, 1 replies.
- [PR] #2134 fetcher.thread.timeout: cancel the okhttp call, bounded helper pool for other protocols [stormcrawler] - posted by "GGraziadei (via GitHub)" <gi...@apache.org> on 2026/09/06 21:59:46 UTC, 17 replies.
- [GH] (stormcrawler/perf/fetch-timeout-call-cancel): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/06 22:12:21 UTC, 1 replies.
- [PR] build(deps): bump org.apache.maven.plugins:maven-compiler-plugin from 3.15.0 to 3.16.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/07 04:02:13 UTC, 1 replies.
- [PR] build(deps): bump org.yaml:snakeyaml from 2.6 to 2.7 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/07 04:02:19 UTC, 1 replies.
- [PR] build(deps): bump com.puppycrawl.tools:checkstyle from 14.0.0 to 14.1.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/07 04:02:30 UTC, 1 replies.
- [PR] build(deps): bump software.amazon.awssdk:bom from 2.54.6 to 2.54.12 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/07 04:02:38 UTC, 1 replies.
- [PR] build(deps): bump org.opensearch.client:opensearch-java from 3.9.0 to 3.10.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/07 04:02:49 UTC, 1 replies.
- [PR] build(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.5.6 to 3.6.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/07 04:03:05 UTC, 1 replies.
- [PR] build(deps): bump org.netpreserve:jwarc from 0.36.0 to 0.37.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/07 04:03:09 UTC, 1 replies.
- [GH] (stormcrawler/main): Workflow run "CI Main - Regenerate License" failed! - posted by GitBox <gi...@apache.org> on 2026/09/07 17:18:57 UTC, 0 replies.
- [GH] (stormcrawler/main): Workflow run "CI Main - Regenerate License" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/07 17:19:01 UTC, 0 replies.
- [GH] (stormcrawler/issue-2079-okhttp-trust-v2): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/07 17:48:35 UTC, 0 replies.
- [I] Politeness keys do not collapse IDN host spellings (unicode vs punycode) [stormcrawler] - posted by "abhinav-phi (via GitHub)" <gi...@apache.org> on 2026/09/07 18:42:48 UTC, 0 replies.
- [GH] (stormcrawler/issue-2086-ipfilter-linklocal): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/07 19:17:26 UTC, 0 replies.
- [GH] (stormcrawler/issue-2087-robots-content-limit): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/07 19:17:27 UTC, 0 replies.
- [GH] (stormcrawler/issue-2080-private-address-default): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/08 06:39:09 UTC, 1 replies.
- Re: [PR] #2104 feat: implement MIME type whitelist filtering in ParserBolt for Tika [stormcrawler] - posted by "Riddhish1 (via GitHub)" <gi...@apache.org> on 2026/09/08 10:51:57 UTC, 7 replies.
- [GH] (stormcrawler/fix/tika-whitelist-detected-mimetype): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/08 13:26:53 UTC, 0 replies.
- [GH] (stormcrawler/issue-2079-okhttp-trust-v2): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/08 18:47:29 UTC, 2 replies.
- Re: [I] IPFilterRules has no keyword for link-local addresses and the documented example leaves them allowed [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/08 19:07:41 UTC, 0 replies.
- [GH] (stormcrawler/issue-2082-host-canonicalisation): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/08 20:20:02 UTC, 0 replies.
- Re: [I] The robots.txt fetch replaces the configured http.content.limit with no limit at all [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/09 09:08:58 UTC, 0 replies.
- Re: [I] URLFilters.filter() returns the URL as accepted when a filter in the chain throws [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/09 09:51:08 UTC, 0 replies.
- Re: [I] The file scheme is enabled by default and FileProtocol reads any path the worker can open [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/09 17:27:50 UTC, 0 replies.
- [PR] SQL IndexerBolt: only plain identifiers may be used as column names, fixes #2096 [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/09 17:49:17 UTC, 3 replies.
- [GH] (stormcrawler/fix/sql-indexer-identifier-injection): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/09 21:09:08 UTC, 3 replies.
- [GH] (stormcrawler/main): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/09 22:38:47 UTC, 6 replies.
- [GH] (stormcrawler/fix/sql-indexer-identifier-injection): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/09 22:45:34 UTC, 0 replies.
- Re: [I] Upgrade to Tika 4.0 [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/10 07:48:05 UTC, 0 replies.
- Re: [I] SQL IndexerBolt interpolates metadata labels into the statement as unquoted identifiers [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/10 07:50:32 UTC, 0 replies.
- Re: [I] [Improvement] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/10 08:00:29 UTC, 0 replies.
- [GH] (stormcrawler/main): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/10 08:16:34 UTC, 3 replies.
- [PR] Minor: Fix formatting in ColumnNameValidationTest [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/10 09:26:15 UTC, 1 replies.
- [GH] (stormcrawler/fix-main-formatting): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/10 10:16:12 UTC, 0 replies.
- Re: [I] Tika mimetype whitelist is checked against the HTTP header while the parser is chosen from the bytes [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/10 11:19:23 UTC, 0 replies.
- Re: [I] Credentials are sent over connections the okhttp protocol never authenticated, and the trust-all path uses an "SSL" context with an always-true hostname verifier [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/10 17:57:44 UTC, 0 replies.
- [PR #2135] fetcher.thread.timeout: cancel the okhttp call, bounded helper pool for other protocols - posted by Gianluca Graziadei <gr...@gmail.com> on 2026/09/10 19:32:03 UTC, 0 replies.
- Re: [I] Nothing in the shipped configuration keeps the fetcher off private address space [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/11 19:02:38 UTC, 0 replies.
- [PR] URLFrontier StatusUpdaterBolt: make lock objects transient so the bolt can be serialized [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/11 19:42:00 UTC, 1 replies.
- [GH] (stormcrawler/sc-1998-warc-protocol-header): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/12 13:24:37 UTC, 0 replies.
- Re: [PR] #1998 WARC writer: WARC-Protocol header to follow WARC field proposals [stormcrawler] - posted by "sebastian-nagel (via GitHub)" <gi...@apache.org> on 2026/09/12 13:34:27 UTC, 2 replies.
- [GH] (stormcrawler/sc-1998-warc-protocol-header): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/12 13:40:35 UTC, 0 replies.
- [PR] Upgrade to Apache Storm 3.1.0 and Java 25 [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/13 06:02:18 UTC, 1 replies.
- Re: [I] [Improvement] WARC writer: WARC-Protocol header to follow WARC field proposals [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/13 06:06:49 UTC, 0 replies.
- Re: [I] Politeness queues and the robots.txt cache key on the raw host string, so percent-escaped host aliases get their own queue and their own robots.txt fetch [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/13 06:07:31 UTC, 0 replies.
- Re: [I] SiteMapParserBolt sniffs every document, parses sitemaps non-strict, and turns a failed sitemap parse into a terminal ERROR [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/13 06:10:50 UTC, 0 replies.
- Re: [I] SCProxy.toString() rebuilds the proxy URL with user and password, and the protocol logs it [stormcrawler] - posted by "Riddhish1 (via GitHub)" <gi...@apache.org> on 2026/09/13 06:16:37 UTC, 2 replies.
- Re: [I] Persistence spouts emit stored URLs without checking the scheme [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/13 06:37:01 UTC, 0 replies.
- Re: [I] Redirect hops taken inside okhttp are not run through the URL filters [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/13 06:53:44 UTC, 0 replies.
- [PR] #2102 Redact proxy credentials from worker logs [stormcrawler] - posted by "Riddhish1 (via GitHub)" <gi...@apache.org> on 2026/09/13 13:30:23 UTC, 3 replies.
- Re: [I] URLFrontier channels are always plaintext and the module has no TLS option [stormcrawler] - posted by "Riddhish1 (via GitHub)" <gi...@apache.org> on 2026/09/13 13:47:42 UTC, 1 replies.
- Re: [I] HostURLFilter memoizes the source host in plain fields shared by all fetcher threads [stormcrawler] - posted by "Riddhish1 (via GitHub)" <gi...@apache.org> on 2026/09/13 17:27:34 UTC, 2 replies.
- [PR] build(deps): bump storm-client.version from 3.0.0 to 3.1.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/14 04:02:31 UTC, 3 replies.
- [PR] build(deps): bump langchain4j.version from 1.19.0 to 1.20.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/14 04:02:40 UTC, 1 replies.
- [PR] build(deps): bump org.slf4j:slf4j-simple from 2.0.18 to 2.0.19 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/14 04:02:54 UTC, 1 replies.
- [PR] build(deps): bump software.amazon.awssdk:bom from 2.54.12 to 2.54.16 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/14 04:03:10 UTC, 1 replies.
- [PR] build(deps): bump actions/setup-java from 6.0.0 to 6.0.1 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/14 13:23:33 UTC, 1 replies.
- [PR] #2092 Bound the meta charset look-ahead and catch charset detection failures in JSoupParserBolt [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/15 07:37:50 UTC, 1 replies.
- [GH] (stormcrawler/meta-charset-overflow): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/15 07:43:28 UTC, 1 replies.
- [GH] (stormcrawler/fix/proxy-credential-redaction): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/15 07:52:50 UTC, 0 replies.
- [PR] #2094 Make ignoring HTTPS errors in the Playwright protocol configurable and independent of the proxy [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/15 08:10:57 UTC, 1 replies.
- [GH] (stormcrawler/meta-charset-overflow): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/15 08:38:21 UTC, 1 replies.
- [I] Scope credential headers in http.custom.headers to hosts [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/15 09:33:17 UTC, 1 replies.
- [PR] #2089 Send http.basicauth credentials only to the hosts in http.basicauth.hosts [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/15 09:33:22 UTC, 2 replies.
- [PR] #2162 Send credential headers from http.custom.headers only to the hosts in http.custom.headers.hosts [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/15 09:44:26 UTC, 1 replies.
- [PR] #2095 Apply the IP address filter to the requests made by the Playwright browser [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/15 10:58:09 UTC, 3 replies.
- Re: [I] Send host info on a specific stream [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/15 14:22:59 UTC, 1 replies.
- [GH] (stormcrawler/perf/fetch-timeout-call-cancel): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/16 17:29:08 UTC, 0 replies.
- Re: [I] Playwright browser context stops validating certificates as soon as a proxy is configured [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/18 17:30:55 UTC, 0 replies.
- Re: [I] http.basicauth credentials are attached to every request, whatever the host [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/18 17:37:21 UTC, 1 replies.
- Re: [I] CharsetIdentification retries the meta charset lookup ten bytes at a time by recursing, and the call sits outside JSoupParserBolt's catch block [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/18 18:37:30 UTC, 0 replies.
- [I] [Improvement] Add a cap on the number of outlinks emitted per sitemap [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/18 19:41:03 UTC, 1 replies.
- [PR] #2101 HostURLFilter: drop the source host memoization shared by all fetcher threads [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/18 19:50:47 UTC, 1 replies.
- [PR] #2090 Fetcher bolts: drop a set-header value coming from a response [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/18 19:50:58 UTC, 1 replies.
- [PR] #2100 SiteMapParserBolt: read the content type under the protocol prefix [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/18 19:51:04 UTC, 3 replies.
- [PR] #2099 Fetcher bolts: skip a metrics.* value which is not a number [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/18 19:51:13 UTC, 3 replies.
- [PR] #2091 Protocols: reject metadata keys arriving in a response header [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/18 19:51:18 UTC, 3 replies.
- [GH] (stormcrawler/fix-2091-reserved-metadata-keys): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/18 19:52:10 UTC, 1 replies.
- [GH] (stormcrawler/fix-2099-metrics-header-parse): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/18 20:03:32 UTC, 0 replies.
- [GH] (stormcrawler/2095-playwright-ip-filter): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/19 16:25:52 UTC, 0 replies.
- Re: [I] Playwright protocol applies neither the IP filter nor the URL filters to the requests it makes [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/19 16:35:37 UTC, 0 replies.
- Re: [I] A response header named set-header becomes a request header directive [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/19 16:36:09 UTC, 0 replies.
- Re: [I] Response headers named metrics.* are fed straight into the fetcher metric registry, and a non-numeric one fails the fetch [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/19 17:04:49 UTC, 0 replies.
- Re: [I] Response headers can populate the reserved verbatim request and response metadata keys [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/19 17:05:18 UTC, 0 replies.
- Re: [I] Gzipped sitemap files are expanded with no size bound, and SiteMapParserBolt reads Content-Type under a key that is never present [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/19 17:22:00 UTC, 0 replies.
- [PR] #2166 SiteMapParserBolt: cap the number of URLs emitted per sitemap [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/19 17:40:07 UTC, 1 replies.
- [PR] #2103 OpenSearchConnection: send the credentials only to the configured addresses [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/19 17:40:13 UTC, 2 replies.
- [PR] #2097 Tika ParserBolt: limit the extracted text with parser.tika.text.maxlength [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/19 17:40:19 UTC, 2 replies.
- [PR] #2098 URLFrontier: optional TLS for the channels to the frontier [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/19 17:40:26 UTC, 4 replies.
- [PR] build(deps): bump com.microsoft.playwright:playwright from 1.62.0 to 1.63.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/21 04:02:41 UTC, 1 replies.
- [PR] build(deps): bump software.amazon.awssdk:bom from 2.54.16 to 2.55.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/21 04:02:53 UTC, 1 replies.
- sending fetched pages to a remote tika over grpc - posted by Davide Polato <dp...@apache.org> on 2026/09/22 14:10:30 UTC, 0 replies.
- [GH] (stormcrawler/fix-2103-opensearch-credentials-scope): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/23 08:50:30 UTC, 0 replies.
- [GH] (stormcrawler/fix-2098-urlfrontier-tls): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/23 09:27:46 UTC, 0 replies.
- Re: [I] OpenSearch addresses default to http and Basic credentials are registered for any host [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/23 17:08:33 UTC, 0 replies.
- [PR] #2093 LLM text extractor: keep page markers out of the prompt and clean the reply [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/23 17:35:22 UTC, 4 replies.
- [PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/23 17:35:26 UTC, 11 replies.
- [GH] (stormcrawler/fix-2097-tika-parse-timeout): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/23 18:09:35 UTC, 0 replies.
- [PR] Tika pipes parser timeout [stormcrawler] - posted by "tballison (via GitHub)" <gi...@apache.org> on 2026/09/23 20:21:59 UTC, 0 replies.
- [GH] (stormcrawler/tika-pipes-parser-timeout): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/23 20:38:06 UTC, 2 replies.
- [GH] (stormcrawler/tika-pipes-parser-timeout): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/23 21:57:13 UTC, 2 replies.
- [PR] #2058 FetcherBolt: count the URLs dropped by fetcher.timeout.queue [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/24 08:00:32 UTC, 1 replies.
- [GH] (stormcrawler/fix/2058-queue-timeout-metric): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/24 08:01:31 UTC, 0 replies.
- [GH] (stormcrawler/fix/2058-queue-timeout-metric): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/09/24 08:18:22 UTC, 0 replies.
- Re: [PR] 2097: Isolate tika parsing via tika-pipes to ensure reliable timeouts and protection from OOMs [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/24 08:35:38 UTC, 19 replies.
- [I] URLFrontier Spout: depth metrics to observe and drive the breadth of the crawl [stormcrawler] - posted by "GGraziadei (via GitHub)" <gi...@apache.org> on 2026/09/24 09:36:27 UTC, 6 replies.
- [PR] #2185 URLFrontier Spout: depth metrics to observe the breadth of the crawl [stormcrawler] - posted by "GGraziadei (via GitHub)" <gi...@apache.org> on 2026/09/24 09:36:51 UTC, 2 replies.
- [PR] Bring back the logo in the README [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/25 11:13:10 UTC, 1 replies.
- Re: [I] fetcher.timeout.queue silently drops URLs [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/25 14:51:18 UTC, 0 replies.
- Re: [I] AbstractLLMTextExtractor pastes page HTML into the prompt unescaped and returns the reply unchecked [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/26 14:04:15 UTC, 0 replies.
- [PR] build(deps): bump org.slf4j:slf4j-simple from 2.0.19 to 2.0.20 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/28 04:02:16 UTC, 1 replies.
- [PR] build(deps-dev): bump de.thetaphi:forbiddenapis from 3.10 to 3.11 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/28 04:02:19 UTC, 1 replies.
- [PR] build(deps): bump git-code-format-maven-plugin.version from 6.0 to 6.1 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/28 04:02:44 UTC, 1 replies.
- [PR] build(deps): bump org.apache:apache from 39 to 40 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/28 04:02:47 UTC, 1 replies.
- [PR] build(deps): bump com.github.ben-manes.caffeine:caffeine from 3.2.4 to 3.3.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/28 04:02:50 UTC, 1 replies.
- [PR] build(deps): bump software.amazon.awssdk:bom from 2.55.0 to 2.55.5 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/28 04:02:59 UTC, 1 replies.
- [PR] build(deps): bump org.mockito:mockito-core from 5.23.0 to 5.24.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/09/28 04:03:19 UTC, 2 replies.
- [I] [Bug report] BasicURLNormalizer folds the query into the path when it rebuilds a URL [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/29 08:37:10 UTC, 0 replies.
- [I] [Bug report] BasicURLNormalizer unmangleQueryString inserts a second `?` when a query value contains a `/` [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/29 08:42:56 UTC, 0 replies.
- [I] URLNormalizer The query is split on `;` as well as `&`, which rewrites URLs [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/29 08:57:10 UTC, 0 replies.
- [PR] #2057 FetcherBolt: add characterization tests [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/29 09:46:45 UTC, 0 replies.
- [PR] #2057 FetcherBolt: move the fetch queue classes to their own files [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/29 09:48:13 UTC, 0 replies.
- [PR] #2057 FetcherBolt: extract the robots.txt lookup [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/29 09:50:22 UTC, 0 replies.
- [PR] #2057 FetcherBolt: extract the sitemap autodiscovery [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/29 09:50:25 UTC, 0 replies.
- [PR] #2057 FetcherBolt: extract the crawl-delay policy [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/29 09:50:28 UTC, 0 replies.
- [PR] #2057 FetcherBolt: extract the response and failure handling [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/29 09:50:31 UTC, 4 replies.
- [PR] #2197 BasicURLNormalizer keeps the query when rebuilding a URL [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/29 11:39:40 UTC, 1 replies.
- [I] Upgrade to Tika 4.1.0 [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/30 14:37:43 UTC, 2 replies.
- [PR] #2057 FetcherBolt: move the fetcher internals to o.a.s.fetcher [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/09/30 14:49:36 UTC, 0 replies.
- [PR] Upgrade to Tika 4.1.0 [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/30 14:57:42 UTC, 1 replies.
- [I] Migrate core from HttpClient 4.x to HttpClient 5.x [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/30 15:16:37 UTC, 0 replies.
- [I] Keep opensearch-java and remove opensearch module? [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/30 15:21:32 UTC, 0 replies.
- [PR] Add StatusUpdaterBenchmark to measure the performance of status updater backends [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/09/30 17:54:20 UTC, 1 replies.
- [GH] (stormcrawler/statusupdaterbenchmark): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/09/30 18:49:05 UTC, 0 replies.