You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "yandrey321 (via GitHub)" <gi...@apache.org> on 2026/10/23 19:43:21 UTC
[PR] HDDS-17353. Register the BouncyCastle provider only when it is looked up by name [ozone]
yandrey321 opened a new pull request, #11725:
URL: https://github.com/apache/ozone/pull/11725
## What changes were proposed in this pull request?
**Problem.** Every Ozone client registers the BouncyCastle JCE provider at start-up, from two places: the
`SecurityConfig` constructor, which clients create just to read flags such as "is security enabled", and
`OzoneSecurityUtil.isSecurityEnabled()`. With the default `hdds.security.provider=BC`, that runs
`Security.addProvider(new BouncyCastleProvider())`. The JVM verifies the signed BouncyCastle jar and loads about 500
provider classes, which costs about 150 ms in a cold client (YARN containers, CLI commands, tasks).
A plain client uses the provider for one thing: parsing the CA certificates that OM returns
(`ServiceInfoEx.getCaCertificates()` → `CertificateCodec.readX509Certificate()`). It asks for BouncyCastle's X.509
factory by name, although the JDK's own factory parses those single PEM certificates.
**Implementation idea.** Register the provider where it's looked up by name, and nowhere else:
- `SecurityConfig.getProvider()` registers the configured provider before returning its name. Every service path
that names the provider goes through it: key generation, certificate requests, signing, the certificate client.
- `CertificateCodec.getCertFactory()` registers BouncyCastle before asking for it. It's still used for certificate
paths, including the "PEM" cert-path encoding that only BouncyCastle supports.
- `CertificateCodec.readX509Certificate()` parses with the JDK's X.509 factory.
- The `SecurityConfig` constructor and `OzoneSecurityUtil.isSecurityEnabled()` no longer register anything.
**Key decisions.**
- **The registration logic is unchanged:** it resolves the provider once per JVM, adds BouncyCastle for "BC", and
rejects an unknown name. It now also has an unsynchronized fast path, through
`initSecurityProvider(String providerName)`.
- **No registration at service start-up.** All main-code lookups of the provider by name go through `getProvider()`
or `getCertFactory()`. The explicit `initSecurityProvider` calls in `HddsServerUtil.getScmSecurityClient*` stay.
`STSTokenEncryption` uses its own provider instance.
- **`readX509Certificate` changes for services too.** The default lookup gets the JDK's implementation even when
BouncyCastle is registered, because `addProvider` appends it last. Certificate equality compares the encoded form,
so a certificate parsed by either provider is equal.
**Side effects.**
- An unknown `hdds.security.provider` is now reported at the first `getProvider()`, not in the `SecurityConfig`
constructor.
- Certificates read through `readX509Certificate` are JDK `X509Certificate` objects instead of BouncyCastle ones.
- An application that relied on Ozone registering BouncyCastle as a side effect, and then asked for it by name,
has to register it itself.
Generated-by: Claude Code (Claude Opus 5.5)
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-17353
## How was this patch tested?
Unit and integration tests.
- **Measurement:** cold `ofs://` client on the secure `ozonesecure` compose cluster (Kerberos, gRPC TLS; create the
client, getFileStatus, read 64 KB, close), JDK 17 (17.0.20.1), 10 interleaved rounds per arm. The arms differ only in
the `hdds-common` jar: this change, or the same jar built from the previous `SecurityConfig`, `OzoneSecurityUtil` and
`CertificateCodec`.
| | before | after |
|---|---|---|
| total | 1,320.5 ms (1,283–1,355 ms) | **1,211.5 ms (1,187–1,220 ms), −109 ms** |
| client creation | 576 ms | 496 ms (−80 ms) |
| first read, which parses the CA certificates | 447.5 ms | 422 ms (−25.5 ms) |
| classes loaded | 8,705 | 8,078 |
| of which BouncyCastle | 615 | 2 |
The ranges don't overlap.
- **TestDFSIO on secure YARN** (YARN images with JDK 17) against the same cluster: 3 NodeManagers with
`LinuxContainerExecutor`, 4 files, write and read, 6 interleaved rounds per arm, 24 of 24 jobs successful. The arms
differ only in those three classes: before each arm's jobs the shaded `ozone-filesystem-hadoop3` jar is swapped for
this build, or for the same jar with the previous classes.
| JVM | change |
|---|---|
| localizer: localization time (main band of 55 localizations at about 1.5–1.6 s) | −151 ms median, −139 ms mean (standard error about 0.013 s) |
| AppMaster start-up | −155 ms median, −144 ms mean |
| map start-up | −149 ms median, −164 ms mean |
Every cold client on a job's critical path gains (two localizers, the AppMaster, the maps): about 0.6 s per job.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-17353. Register the BouncyCastle provider only when it is looked up by name [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #11725:
URL: https://github.com/apache/ozone/pull/11725#issuecomment-6381978496
Thanks @yandrey321 for the patch, @jojochuang for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-17353. Register the BouncyCastle provider only when it is looked up by name [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #11725:
URL: https://github.com/apache/ozone/pull/11725#discussion_r4364864622
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/utils/CertificateCodec.java:
##########
@@ -184,6 +185,8 @@ public static X509Certificate firstCertificateFrom(CertPath certificatePath) {
}
public static CertificateFactory getCertFactory() throws CertificateException {
+ // BouncyCastle, for the "PEM" cert path encoding only it supports
+ SecurityConfig.initSecurityProvider("BC");
Review Comment:
subsequent calls to this takes no time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-17353. Register the BouncyCastle provider only when it is looked up by name [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #11725:
URL: https://github.com/apache/ozone/pull/11725
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org