[jira] [Resolved] (FLINK-40592) PrometheusPushGatewayReporter fails to initialize with basic authentication

["Martijn Visser (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-40592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Martijn Visser resolved FLINK-40592.
------------------------------------
    Resolution: Fixed

> PrometheusPushGatewayReporter fails to initialize with basic authentication
> ---------------------------------------------------------------------------
>
>                 Key: FLINK-40592
>                 URL: https://issues.apache.org/jira/browse/FLINK-40592
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Metrics
>    Affects Versions: 2.3.0
>         Environment: Original reproduction: Ubuntu 24.04.4 LTS under WSL 2, x86_64; OpenJDK 17.0.20. Independent reproductions: macOS 26.0 arm64; OpenJDK 11.0.28 (Homebrew) and JetBrains Runtime / JDK 21.0.8. All reproductions use published Flink 2.3.0 artifacts from Maven Central and an explicit classpath of six checksum-verified JARs without JAXB. Direct reporter factory reproduction; no full Flink cluster test.
>            Reporter: Weicheng Ji
>            Assignee: Weicheng Ji
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 2.3.1, 2.4.0
>
>         Attachments: flink-pushgateway-reproducer.zip, java11-reproduction.txt
>
>
> h3. Problem
> With the published Flink 2.3.0 Prometheus reporter JAR on Java 11, 17 and 21, configuring both username and password causes {{PrometheusPushGatewayReporter}} construction to fail with {{NoClassDefFoundError: javax/xml/bind/DatatypeConverter}} when JAXB is not on the classpath. The same factory call without credentials succeeds.
> h3. Environment and version scope
> * Original reproduction: Ubuntu 24.04.4 LTS under WSL 2, x86_64; OpenJDK 17.0.20.
> * Independent reproduction: macOS 26.0 arm64; JetBrains Runtime / JDK 21.0.8.
> * Additional independent reproduction: macOS 26.0 arm64; OpenJDK 11.0.28 (Homebrew).
> * Published Flink 2.3.0 artifacts and SLF4J, using the six pinned JARs listed in the attachment's {{SHA256SUMS}}, without additional JAXB libraries.
> The authentication path was introduced by FLINK-27661 / [PR #27576|https://github.com/apache/flink/pull/27576] for 2.3.0. Source inspection also found it on release-2.3 and master (2.4-SNAPSHOT); release-2.2.0 does not contain this path. Runtime reproduction was performed on 2.3.0 only.
> These checks call the reporter factory directly. A full Flink distribution or cluster has not been tested.
> h3. Reproduction
> On Linux/WSL, extract {{flink-pushgateway-reproducer.zip}} and run {{bash run.sh}} with JDK 17, Bash, curl, sha256sum, and access to Maven Central. The script downloads and verifies the six pinned artifacts, then compiles and runs the following source. No running Pushgateway is required: the authenticated case fails during construction, before an HTTP request.
> {code:java}
> import org.apache.flink.metrics.MetricConfig;
> import org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporterFactory;
> public class ReporterRepro {
>     public static void main(String[] args) throws Exception {
>         MetricConfig config = new MetricConfig();
>         config.setProperty("hostUrl", "http://127.0.0.1:9091");
>         config.setProperty("deleteOnShutdown", "false");
>         PrometheusPushGatewayReporterFactory factory = new PrometheusPushGatewayReporterFactory();
>         factory.createMetricReporter(config).close();
>         System.out.println("PASS: reporter without credentials initializes and closes");
>         config.setProperty("username", "test-user");
>         config.setProperty("password", "test-password");
>         System.out.println("Creating reporter with credentials...");
>         factory.createMetricReporter(config).close();
>         System.out.println("PASS: reporter with credentials initializes and closes");
>     }
> }
> {code}
> The README includes the explicit classpath commands. The ZIP contains the original Java 17 logs and independent Java 21 logs; the Java 21 check used direct javac/java commands. Java 11 was reproduced with the same source and six checksum-verified artifacts using direct javac/java commands. Its environment and full output are in {{java11-reproduction.txt}}.
> h3. Expected and actual behavior
> Expected: initialization succeeds with or without credentials, without requiring an additional JAXB dependency solely for Base64 encoding.
> Actual: the unauthenticated control succeeds; the authenticated call throws the following error (excerpt from the Java 17 output):
> {code:none}
> PASS: reporter without credentials initializes and closes
> Creating reporter with credentials...
> Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/bind/DatatypeConverter
>     at io.prometheus.client.exporter.BasicAuthHttpConnectionFactory.encode(BasicAuthHttpConnectionFactory.java:31)
>     at io.prometheus.client.exporter.BasicAuthHttpConnectionFactory.<init>(BasicAuthHttpConnectionFactory.java:14)
>     at io.prometheus.client.exporter.BasicAuthHttpConnectionFactory.<init>(BasicAuthHttpConnectionFactory.java:18)
>     at org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporter.<init>(PrometheusPushGatewayReporter.java:60)
>     at org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporterFactory.createMetricReporter(PrometheusPushGatewayReporterFactory.java:86)
>     at ReporterRepro.main(ReporterRepro.java:15)
> Caused by: java.lang.ClassNotFoundException: javax.xml.bind.DatatypeConverter
> {code}
> The Java process exits with code 1. The wrapper script exits with code 0 when it confirms this known failure. Full output is included in the attachment.
> h3. Root cause and proposed approach
> The reporter constructs simpleclient's {{BasicAuthHttpConnectionFactory}} when both credentials are present. The [Flink module POM|https://github.com/apache/flink/blob/release-2.3.0/flink-metrics/flink-metrics-prometheus/pom.xml] uses Prometheus 0.8.1; in that version, [BasicAuthHttpConnectionFactory|https://github.com/prometheus/client_java/blob/parent-0.8.1/simpleclient_pushgateway/src/main/java/io/prometheus/client/exporter/BasicAuthHttpConnectionFactory.java] calls JAXB's {{DatatypeConverter.printBase64Binary}} from its constructor. JAXB was removed from the JDK in Java 11 ([JEP 320|https://openjdk.org/jeps/320]).
> One possible fix is an internal {{HttpConnectionFactory}} using {{java.util.Base64}} and UTF-8 to set the Basic Authorization header while preserving existing connection and credential configuration behavior. This would avoid adding JAXB solely for Base64 encoding. I am open to the approach preferred by maintainers.
> AI assistance: OpenAI Codex assisted with the investigation, reproducer, candidate fix and tests, and this report.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)