You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "FrankChen021 (via GitHub)" <gi...@apache.org> on 2026/09/21 02:39:33 UTC
[I] Flaky test Triage - 2026-09-21 (druid)
FrankChen021 opened a new issue, #20389:
URL: https://github.com/apache/druid/issues/20389
_This issue was generated automatically by Claude Code (Anthropic's AI coding agent) running a scheduled CI-triage routine on behalf of @FrankChen021. Analysis and suggested fixes are AI-produced; please verify before acting on them._
This triage covers the single commit merged to master on 2026-09-20 (b2b9baedca, #20383, a Dependabot bump of `aws.sdk.v2.version` from 2.54.18 to 2.54.19 in `pom.xml` and `licenses.yaml`). Its push-triggered workflows (Static Checks CI, CodeQL, Unit & Integration tests CI) all passed; the only failed job is the `security vulnerabilities` job of the scheduled Cron Job ITs run on that commit. Nothing in the failure is caused by the commit: the job has been red on every scheduled master run since 2026-09-17 with the same two problems (a Sonatype OSS Index HTTP 402 outage and the `elasticache-java-cluster-client` false positive, see #20366, #20378, #20385), and since 2026-09-19 it additionally reports three genuine ZooKeeper 3.8.6 CVEs published on 2026-09-16 (fixed upstream in ZooKeeper 3.8.7 / 3.9.6). The OSS Index part is infrastructure; the other two are persistent CI problems that need a suppression (#20236) and a dependency bump respectively. No re-runs had been triggered.
### Summary
| Commit | Failed job | Failure log | Root cause | Verdict |
|---|---|---|---|---|
| b2b9baedca (#20383, build(deps): bump aws.sdk.v2.version 2.54.18 to 2.54.19) | `security vulnerabilities (cron)` | [job 106012780651](https://github.com/apache/druid/actions/runs/35486148433/job/106012780651) | `dependency-check-maven:13.0.0:check`: on `druid-processing`, `AnalysisException: Sonatype OSS Index / Guide credits insufficient / payment required` (HTTP 402 from `api.guide.sonatype.com/api/v3/component-report`); the fallback's unescaped backticks then ran a second credential-less scan that failed on `druid-server` with `elasticache-java-cluster-client-1.2.4.jar` matched to `cpe:2.3:a:memcached:memcached:1.2.4` (10 memcached server CVEs >= 7.0, false positive) and `zookeeper-3.8.6.jar` flagged with CVE-2026-59739, CVE-2026-59969, CVE-2026-79993 (all 7.5, genuine, fixed in ZooKeeper 3.8.7) | Infra (OSS Index 402); Persistent (memcached false positive, fix in #20236; ZooKeeper 3.8.6 CVEs, needs bump to 3.8.7) |
### Analysis and suggested fixes
**1. `security vulnerabilities (cron)`: Sonatype OSS Index HTTP 402 Payment Required (infra)**
The job runs `mvn -B dependency-check:purge dependency-check:check` with the `OSS_INDEX_USERNAME` / `OSS_INDEX_PASSWORD` / `NVD_API_KEY` secrets. The NVD download finished in about 13 minutes, and the first module scanned, `druid-processing`, failed because the OSS Index analyzer received `402 Payment Required` from `api.guide.sonatype.com/api/v3/component-report` ("credits insufficient / payment required, disabling the analyzer"). dependency-check 13 treats this analyzer exception as fatal (`failOnError` defaults to true), so the Maven reactor stopped at `druid-processing`. The identical 402 hit the scheduled runs on 2026-09-17, 2026-09-18 and 2026-09-19 (#20366, #20378, #20385), so it is a credit/quota problem on the OSS Index account (Sonatype is migrating OSS Index to Sonatype Guide with paid credits), not anything in the commit, which only changes an AWS SDK version property.
The second half of the log is an artifact of the workflow itself: the `|| { echo "... `mvn dependency-check:check` ..." && false; }` fallback in `.github/workflows/cron-job-its.yml` puts `mvn dependency-check:check` in unescaped backticks inside a double-quoted string, so bash runs a second full scan (without the OSS Index and NVD credentials, hence the "Sonatype OSS Index Analyzer disabled due to missing credentials" warnings) as a command substitution before printing the message. That second scan is where the `druid-server` findings below come from; they would equally be reported by the first scan once the OSS Index problem is resolved.
Suggested fix: replenish the OSS Index credits or migrate the `OSS_INDEX_USERNAME` / `OSS_INDEX_PASSWORD` secrets to a Sonatype Guide personal access token, and pass `-DossIndexWarnOnlyOnRemoteErrors=true` (or set `<ossIndexWarnOnlyOnRemoteErrors>true</ossIndexWarnOnlyOnRemoteErrors>` in the plugin configuration in `pom.xml`) so a remote OSS Index error degrades to a warning instead of failing the build. In the workflow, single-quote the fallback message or escape the backticks so it stops launching a second scan. #20126 (keep the NVD database between runs instead of purging it) remains worthwhile.
**2. `elasticache-java-cluster-client-1.2.4.jar` matched to `cpe:/a:memcached:memcached` (druid-server)**
`com.amazonaws:elasticache-java-cluster-client` is a Java memcached client library (an AWS fork of spymemcached) that Druid uses only in the memcached cache implementation (`server/pom.xml`). dependency-check maps it to `cpe:2.3:a:memcached:memcached:1.2.4`, i.e. the C memcached server daemon, purely on the coinciding version string, and reports 10 server-side CVEs with CVSS >= 7.0 (CVE-2016-8704 9.8, CVE-2016-8705 9.8, CVE-2023-46853 9.8, CVE-2026-47783 8.1, CVE-2026-47784 8.1, CVE-2016-8706 8.1, CVE-2017-9951, CVE-2018-1000127, CVE-2019-11596, CVE-2023-46852 7.5). Druid neither runs nor embeds a memcached server, so these are false positives. `owasp-dependency-check-suppressions.xml` on master still has no entry for this artifact; the `38.0.0` branch got one via #20335 and its 2026-09-18 scheduled run passed (there the OSS Index 402 only surfaced as per-artifact warnings). The false positive has appeared in every scheduled master run since at least 2026-09-14 and is unrelated to
the AWS SDK bump.
Suggested fix: merge #20236, which adds a `<suppress>` entry with `<packageUrl regex="true">^pkg:maven/com\.amazonaws/elasticache-java-cluster-client@.*$</packageUrl>` and `<cpe>cpe:/a:memcached:memcached</cpe>`, or split that block into a standalone PR so the cron job can go green independently of the other suppressions in that PR.
**3. `zookeeper-3.8.6.jar`: CVE-2026-59739, CVE-2026-59969, CVE-2026-79993 (druid-server)**
Unlike item 2, these are genuine. All three CVEs were published on 2026-09-16 and, per NVD, affect ZooKeeper 3.8.0 to 3.8.6 and 3.9.0 to 3.9.5: CVE-2026-59739 (information disclosure via `SetWatches` reconnect replay bypassing the ACL check added for CVE-2024-23944), CVE-2026-59969 (quorum TLS not enforcing peer hostname verification in FIPS mode) and CVE-2026-79993 (`deleteContainer` opcode processed without an ACL check, letting an unauthenticated client delete empty persistent znodes). They first showed up in the 2026-09-19 scheduled run (noted in #20385) and will keep failing the job on every module that depends on `druid-server` even after the memcached suppression lands. Druid pins `zookeeper.version` to 3.8.6 in the root `pom.xml` (set by #19135 for CVE-2026-24308) and depends on it from `server/pom.xml`, `extensions-core/kafka-indexing-service/pom.xml` and a few extension exclusions; Curator is at 5.9.0, which works with ZooKeeper 3.8.x and 3.9.x. Druid itself is a ZooKeep
er client, so the server-side CVEs mostly affect the ZooKeeper ensemble operators deploy alongside Druid, but the scanner cannot tell that apart and the jar is shipped in the distribution.
Suggested fix: bump `zookeeper.version` to 3.8.7 (already on Maven Central, as is 3.9.6) in `pom.xml` and update `licenses.yaml`, then run `mvn dependency-check:check -pl server` to confirm the three CVEs disappear. If the bump has to wait, add a time-boxed `<suppress until="...">` entry for the three CVEs on `^pkg:maven/org\.apache\.zookeeper/zookeeper@3\.8\.6$` with a comment that Druid only uses the client side, so the cron job stops masking new findings in the meantime.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Flaky test Triage - 2026-09-21 (druid)
Posted by "FrankChen021 (via GitHub)" <gi...@apache.org>.
FrankChen021 commented on issue #20389:
URL: https://github.com/apache/druid/issues/20389#issuecomment-5755190147
Closing: the scheduled Cron Job ITs run is out of scope for the daily triage. The push-triggered CI for b2b9baedca was fully green. The ZooKeeper 3.8.6 CVEs and the elasticache false positive noted above are still real and tracked separately (#20236, #20385).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Flaky test Triage - 2026-09-21 (druid)
Posted by "FrankChen021 (via GitHub)" <gi...@apache.org>.
FrankChen021 closed issue #20389: Flaky test Triage - 2026-09-21
URL: https://github.com/apache/druid/issues/20389
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org