You are viewing a plain text version of this content. The canonical link for it is here.
- Re: [PR] 2097: Isolate tika parsing via tika-pipes to ensure reliable timeouts and protection from OOMs [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/10/01 07:32:52 UTC, 2 replies.
- Re: [I] URLFrontier Spout: depth metrics to observe and drive the breadth of the crawl [stormcrawler] - posted by "GGraziadei (via GitHub)" <gi...@apache.org> on 2026/10/01 18:31:39 UTC, 1 replies.
- Re: [PR] #2185 URLFrontier Spout: depth metrics to observe the breadth of the crawl [stormcrawler] - posted by "GGraziadei (via GitHub)" <gi...@apache.org> on 2026/10/01 18:32:16 UTC, 0 replies.
- Re: [I] Migrate core from HttpClient 4.x to HttpClient 5.x [stormcrawler] - posted by "SCORP711 (via GitHub)" <gi...@apache.org> on 2026/10/01 19:09:33 UTC, 2 replies.
- Re: [I] Tika ParserBolt puts no limit on the extracted text and no time limit on the parse [stormcrawler] - posted by "tballison (via GitHub)" <gi...@apache.org> on 2026/10/01 20:16:57 UTC, 0 replies.
- [I] [Improvement] [stormcrawler] - posted by "tballison (via GitHub)" <gi...@apache.org> on 2026/10/01 20:20:12 UTC, 0 replies.
- [PR] #2214 Manage the Jackson dataformat/datatype versions in the root pom [stormcrawler] - posted by "tballison (via GitHub)" <gi...@apache.org> on 2026/10/01 20:23:02 UTC, 1 replies.
- Re: [I] [Improvement] Unify Jackson versions? [stormcrawler] - posted by "tballison (via GitHub)" <gi...@apache.org> on 2026/10/01 20:25:07 UTC, 1 replies.
- Re: [PR] #2057 FetcherBolt: move the fetcher internals to o.a.s.fetcher [stormcrawler] - posted by "rajucomp (via GitHub)" <gi...@apache.org> on 2026/10/01 21:59:36 UTC, 2 replies.
- Re: [PR] #2057 FetcherBolt: add characterization tests [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/10/02 18:53:41 UTC, 0 replies.
- Re: [PR] #2057 FetcherBolt: move the fetch queue classes to their own files [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/10/02 18:54:05 UTC, 0 replies.
- Re: [PR] #2057 FetcherBolt: extract the robots.txt lookup [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/10/02 18:54:23 UTC, 0 replies.
- Re: [PR] #2057 FetcherBolt: extract the sitemap autodiscovery [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/10/02 18:54:39 UTC, 0 replies.
- Re: [PR] #2057 FetcherBolt: extract the crawl-delay policy [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/10/02 18:54:52 UTC, 0 replies.
- Re: [PR] #2057 FetcherBolt: extract the response and failure handling [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/10/02 18:55:29 UTC, 0 replies.
- Re: [I] Refactor FetcherBolt [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/10/02 18:55:49 UTC, 0 replies.
- Re: [PR] Add StatusUpdaterBenchmark to measure the performance of status updater backends [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/03 06:50:06 UTC, 0 replies.
- [PR] build(deps): bump software.amazon.awssdk:bom from 2.55.5 to 2.55.10 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/10/05 04:02:41 UTC, 1 replies.
- [PR] build(deps): bump langchain4j.version from 1.20.0 to 1.20.2 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/10/05 04:02:45 UTC, 1 replies.
- [PR] build(deps): bump git-code-format-maven-plugin.version from 6.1 to 6.2 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/10/05 04:03:03 UTC, 1 replies.
- [PR] build(deps): bump com.google.guava:guava from 33.7.1-jre to 33.7.2-jre [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/10/05 04:03:16 UTC, 1 replies.
- [PR] build(deps): bump com.puppycrawl.tools:checkstyle from 14.1.0 to 14.3.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/10/05 04:03:20 UTC, 1 replies.
- [PR] build(deps): bump org.apache.commons:commons-lang3 from 3.20.0 to 3.21.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/10/05 04:03:35 UTC, 1 replies.
- [PR] StatusUpdaterBenchmark: run several instances of the status updater in parallel [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/05 12:50:10 UTC, 1 replies.
- [PR] URLFrontier StatusUpdaterBolt: cheaper bookkeeping of the acks [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/05 13:04:11 UTC, 1 replies.
- [PR] Regenerated License file after dependency upgrades [stormcrawler] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/10/05 13:20:05 UTC, 13 replies.
- Re: [I] Keep 'opensearch-java' and remove the older 'opensearch' module? [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/06 13:09:50 UTC, 7 replies.
- [PR] opensearch-java: send bulk requests with the OpenSearchAsyncClient [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/06 15:15:26 UTC, 3 replies.
- Re: [PR] #315 Adaptative URL filter to normalize URLs based on canonical tag [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/10/06 17:21:32 UTC, 7 replies.
- Re: [PR] Fix deletion of redirected URLs [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/10/06 17:25:31 UTC, 2 replies.
- [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/10/07 17:22:12 UTC, 8 replies.
- Re: [PR] #2197 BasicURLNormalizer keeps the query when rebuilding a URL [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/10/07 19:14:48 UTC, 2 replies.
- [GH] (stormcrawler/main): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/10/07 19:45:53 UTC, 3 replies.
- [GH] (stormcrawler/2197): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/10/08 11:04:08 UTC, 0 replies.
- [PR] Add SpoutBenchmark to measure the read performance of spouts [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/08 13:56:29 UTC, 1 replies.
- [GH] (stormcrawler/spout-benchmark): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/10/08 15:09:19 UTC, 0 replies.
- Re: [I] URLPartitionerBolt to partition on arbitrary metadata value [stormcrawler] - posted by "smilsimon-s (via GitHub)" <gi...@apache.org> on 2026/10/08 20:26:30 UTC, 2 replies.
- Re: [I] [Bug report] BasicURLNormalizer folds the query into the path when it rebuilds a URL [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/10/09 09:04:53 UTC, 0 replies.
- [PR] #1901 Tika ParserBolt: limit the pages parsed in a PDF with parser.tika.pdf.maxpages [stormcrawler] - posted by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/10/09 09:31:35 UTC, 3 replies.
- [PR] MultiProxyManager: thread-safe round-robin rotation [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/10/09 14:07:46 UTC, 1 replies.
- [GH] (stormcrawler/fix-delete-redirected-pages): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/10/09 14:12:26 UTC, 0 replies.
- Re: [PR] #108 Normalize misspelled HTTP headers [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/09 14:13:12 UTC, 1 replies.
- Re: [PR] Give the caller the possibility to access the actual effective URL used for fetching a resource [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/09 14:14:43 UTC, 3 replies.
- Re: [PR] #1662 - Migrate to Maven 4 [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/09 14:15:22 UTC, 4 replies.
- [GH] (stormcrawler/proxy-round-robin): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/10/09 15:52:53 UTC, 0 replies.
- [GH] (stormcrawler/issue-2063-tika-4): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/10/09 17:26:44 UTC, 0 replies.
- [GH] (stormcrawler/1901-pdf-max-pages): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/10/09 18:21:03 UTC, 0 replies.
- [GH] (stormcrawler/issue-315): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/10/10 11:02:08 UTC, 0 replies.
- [GH] (stormcrawler/feature/store-request-url): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/10/10 11:02:11 UTC, 0 replies.
- [PR] AbstractStatusUpdaterBolt: report cache hits/misses with a counter, not reset-on-read gauges [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/10 20:55:49 UTC, 1 replies.
- [PR] Add a Prometheus reporter for the Storm V2 metrics [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/10 21:00:40 UTC, 4 replies.
- Re: [I] [Improvement] Add configuration to stop parsing PDFs after X pages [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/10 21:00:46 UTC, 0 replies.
- [GH] (stormcrawler/status-updater-cache-counters): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/10/10 21:10:47 UTC, 0 replies.
- [PR] fix: preserve query strings containing slashes [stormcrawler] - posted by "sean0407 (via GitHub)" <gi...@apache.org> on 2026/10/10 22:35:23 UTC, 0 replies.
- Re: [PR] #2198 preserve query strings containing slashes [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/11 04:50:57 UTC, 2 replies.
- [GH] (stormcrawler/fix/108-spellchecked-http-headers): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/10/11 19:03:18 UTC, 0 replies.
- [PR] build(deps): bump software.amazon.awssdk:bom from 2.55.10 to 2.55.13 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/10/12 11:10:14 UTC, 1 replies.
- [PR] build(deps): bump langchain4j.version from 1.20.2 to 1.22.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/10/12 11:10:38 UTC, 1 replies.
- [PR] build(deps): bump commons-codec:commons-codec from 1.22.1 to 1.23.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/10/12 11:10:53 UTC, 1 replies.
- [GH] (stormcrawler/dependabot/maven/commons-codec-commons-codec-1.23.0): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/10/12 13:19:48 UTC, 0 replies.
- [PR] URLFrontier Spout: report the query time [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/12 14:14:32 UTC, 1 replies.
- [GH] (stormcrawler/codex/prevent-slash-query-unmangling): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/10/12 16:50:52 UTC, 0 replies.
- [GH] (stormcrawler/urlfrontier-spout-query-time): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/10/12 17:18:40 UTC, 0 replies.
- [PR] #2211 Migrate core to HttpClient 5 and preserve parsing behavior [stormcrawler] - posted by "SCORP711 (via GitHub)" <gi...@apache.org> on 2026/10/12 19:34:12 UTC, 9 replies.
- [PR] opensearch-java: fix the search_after values sent by HybridSpout [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/10/12 20:04:57 UTC, 2 replies.
- Re: [I] Politeness keys do not collapse IDN host spellings (unicode vs punycode) [stormcrawler] - posted by "omarbounawarapy (via GitHub)" <gi...@apache.org> on 2026/10/14 00:29:49 UTC, 4 replies.
- [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/10/14 13:03:58 UTC, 0 replies.
- [GH] (stormcrawler/migrate-httpclient5): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/10/15 12:30:44 UTC, 0 replies.
- [PR] #2212 retry 429 items and flush bulk requests by size [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/10/16 15:37:21 UTC, 4 replies.
- [PR] #2212 move the opensearch-java implementation into external/opensearch [stormcrawler] - posted by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/10/16 15:37:48 UTC, 4 replies.
- [PR] #2145 Punycode the host in politeness, partition and robots.txt keys [stormcrawler] - posted by "omarbounawarapy (via GitHub)" <gi...@apache.org> on 2026/10/17 13:25:32 UTC, 3 replies.
- [GH] (stormcrawler/2145-punycode-host-keys): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/10/18 12:46:51 UTC, 0 replies.
- [PR] build(deps): bump com.github.ben-manes.caffeine:caffeine from 3.3.0 to 3.3.1 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/10/19 04:21:41 UTC, 1 replies.
- [PR] build(deps): bump software.amazon.awssdk:bom from 2.55.13 to 2.56.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/10/19 04:21:43 UTC, 1 replies.
- [PR] build(deps): bump actions/setup-java from 6.0.1 to 6.1.0 [stormcrawler] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2026/10/19 15:03:08 UTC, 1 replies.
- [I] CharsetIdentification reads Content-Type under a key that is never present, so the HTTP charset is ignored [stormcrawler] - posted by "nicoscandolo (via GitHub)" <gi...@apache.org> on 2026/10/20 17:57:12 UTC, 1 replies.
- [PR] #2250 CharsetIdentification: read the content type under the protocol prefix [stormcrawler] - posted by "nicoscandolo (via GitHub)" <gi...@apache.org> on 2026/10/20 17:57:26 UTC, 10 replies.
- [GH] (stormcrawler/charset-protocol-prefix): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/10/21 19:23:04 UTC, 2 replies.
- [GH] (stormcrawler/codex/prevent-slash-query-unmangling): Workflow run "Java CI with Maven" is working again! - posted by GitBox <gi...@apache.org> on 2026/10/21 22:16:28 UTC, 0 replies.
- Re: [I] New XPATH implementation based on JSON file and organised per hostname or domain [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/22 09:29:56 UTC, 1 replies.
- Re: [I] Remove needsDOM() in ParseFilter [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/22 09:37:00 UTC, 0 replies.
- Re: [I] Capture stats about hosts on status stream post fetching [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/22 09:46:43 UTC, 1 replies.
- Re: [I] wish: WARCHdfsBolt with CDX index [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/22 09:49:19 UTC, 1 replies.
- [PR] #158 jsoup XPathFilter: convert to W3C DOM once per page, not once per expression [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/22 10:05:39 UTC, 1 replies.
- [PR] #158 remove the DOM-based LDJsonParseFilter and LinkParseFilter [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/22 10:13:58 UTC, 1 replies.
- [I] [Bug report] Logo not displayed in the documentation [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/22 10:22:25 UTC, 2 replies.
- [PR] docs: fix light-mode logo not displayed in versioned docs pages [stormcrawler-site] - posted by "suryakant-source (via GitHub)" <gi...@apache.org> on 2026/10/22 11:45:25 UTC, 2 replies.
- [PR] Tika ParserBolt: apply the JSoupFilters to the documents parsed by Tika [stormcrawler] - posted by "jnioche (via GitHub)" <gi...@apache.org> on 2026/10/23 06:19:33 UTC, 0 replies.
- [GH] (stormcrawler/tika-jsoup-filters): Workflow run "Java CI with Maven" failed! - posted by GitBox <gi...@apache.org> on 2026/10/23 07:45:23 UTC, 0 replies.