You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "wankai123 (via GitHub)" <gi...@apache.org> on 2026/08/26 02:00:43 UTC
[PR] Support hot-reloading the BanyanDB credentials and TLS trust CA. [skywalking]
wankai123 opened a new pull request, #14013:
URL: https://github.com/apache/skywalking/pull/14013
Also fix:
- Fix `MultipleFilesChangeMonitor` being able to silently disable every file watch in the OAP. Its registry of monitors was a plain `ArrayList` that `scanChanges()` iterated from the scheduler thread without holding the lock that `start()` / `stop()` take, so starting a monitor while a scan was in flight could raise a `ConcurrentModificationException` from the iterator. That exception escapes past the per-monitor catch, and an uncaught exception cancels a `scheduleAtFixedRate` task permanently — after which no secrets file, keystore, or TLS certificate is ever reloaded again, with nothing in the log to say so. The registry is now copy-on-write. The failure log in the same scan loop also now names the monitor that failed instead of printing an empty `gourp = `.
- Fix `MultipleFilesChangeMonitor` never honouring its watching period. `lastCheckTimestamp` was declared and compared against, but never assigned, so the guard always measured against `0` and passed — every registered monitor re-stat'd its watched files on each 200ms tick of the shared scheduler thread, and the `watchingPeriodInSec` constructor argument had no effect at all. This affects every file watch in the OAP: the ElasticSearch storage secrets / truststore / keystore watch, the BanyanDB credentials and trust CA watches, and the TLS certificate watches behind each OAP HTTP and gRPC server, all of which ask for 10 seconds. Change detection is now paced as configured, which also means it is no longer near-instant: a rotated file is picked up within the requested period rather than within ~200ms.
- [X] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #12863.
- [X] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/master/docs/en/changes/changes.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support hot-reloading the BanyanDB credentials and TLS trust CA. [skywalking]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #14013:
URL: https://github.com/apache/skywalking/pull/14013#discussion_r3859074082
##########
oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java:
##########
@@ -236,6 +243,81 @@ IPprofDataQueryDAO.class, new BanyanDBPprofDataQueryDAO(client)
);
}
+ /**
+ * Load the credentials from the secrets management file, and keep applying them whenever a 3rd party
+ * tool rewrites it. Only the credentials come from this file; the TLS trust CA is watched separately by
+ * {@link #rebuildChannelOnTrustCAChange()}, because it needs a channel rebuild rather than a field swap.
+ *
+ * <p>Whatever the file contains is applied, including a half-populated or empty one. That is deliberate:
+ * a bad file then fails visibly at the next request rather than being quietly refused, which would leave
+ * the working credentials in place and the mistake undiscovered until the next restart.
+ *
+ * <p>Called before {@link #client} is created so that the monitor's initial synchronous check has
+ * already populated the config by the time the client reads it.
+ */
+ private void loadAndWatchSecretsManagementFile() {
+ final String secretsFile = config.getGlobal().getSecretsManagementFile();
+ if (StringUtil.isBlank(secretsFile)) {
+ return;
+ }
+ new MultipleFilesChangeMonitor(10, readableContents -> {
+ final byte[] secretsFileContent = readableContents.get(0);
+ if (secretsFileContent == null) {
+ return;
+ }
+ final Properties secrets = new Properties();
+ secrets.load(new ByteArrayInputStream(secretsFileContent));
+ final String user = secrets.getProperty("user", null);
+ final String password = secrets.getProperty("password", null);
+ config.getGlobal().setUser(user);
+ config.getGlobal().setPassword(password);
+
+ if (client != null) {
+ // AuthInterceptor reads the credentials per RPC, so this applies to the next call without
+ // reconnecting. Before the client exists the config above is all that is needed.
+ client.client.updateCredentials(user, password);
+ if (StringUtil.isNotBlank(user) && StringUtil.isNotBlank(password)) {
+ log.info("Applied the BanyanDB credentials reloaded from {}, user={}", secretsFile, user);
+ } else {
+ // Whatever the file says is applied, so that a mistake in it surfaces instead of being
+ // masked by credentials that silently keep working.
+ log.error(
+ "Applied incomplete credentials from {}: requests are now sent without authentication, "
+ + "because a username and a password are only ever attached together. BanyanDB "
+ + "answers them with UNAUTHENTICATED if it requires authentication.", secretsFile);
+ }
+ }
Review Comment:
The secrets-management watcher applies whatever it reads into `config.getGlobal().setUser/setPassword()` before the `BanyanDBStorageClient` is created (because `start()` runs an initial synchronous check). If the secrets file exists at startup but contains only one half of the pair, `config` ends up with `user != null && password == null`, and `BanyanDBStorageClient` currently fails fast on that state (it throws when only one half is set). This contradicts the intended behavior described here (apply incomplete -> send no auth -> fail at next RPC), and can prevent OAP from starting whenever the secrets file is temporarily incomplete during rotation.
A safer approach is to treat an incomplete pair as “no credentials”: clear both `user` and `password` together (and update the client similarly), while still logging the incomplete-file warning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support hot-reloading the BanyanDB credentials and TLS trust CA. [skywalking]
Posted by "wankai123 (via GitHub)" <gi...@apache.org>.
wankai123 merged PR #14013:
URL: https://github.com/apache/skywalking/pull/14013
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org