You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "kfaraz (via GitHub)" <gi...@apache.org> on 2026/02/20 11:50:28 UTC
[PR] Migrate ITTLSTest to embedded-tests (druid)
kfaraz opened a new pull request, #19037:
URL: https://github.com/apache/druid/pull/19037
### Changes
- Migrate `ITTLSTest` to embedded-tests
- Add `EmbeddedTLSAuthResource`
- Update scripts in `docker/tls` so that they work with `EmbeddedTLSAuthResource`
<hr>
This PR has:
- [ ] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] a release note entry in the PR description.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Migrate `ITTLSTest` to embedded-tests (druid)
Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz commented on code in PR #19037:
URL: https://github.com/apache/druid/pull/19037#discussion_r2839893424
##########
embedded-tests/src/test/java/org/apache/druid/testing/embedded/auth/TestTLSCertificateChecker.java:
##########
@@ -40,7 +40,7 @@ public void checkClient(
) throws CertificateException
{
// only the integration test client with "thisisprobablynottherighthostname" cert is allowed to talk to me
- if (!chain[0].toString().contains("thisisprobablynottherighthostname") || !engine.getPeerHost().contains("172.172.172.1")) {
Review Comment:
No, this is intentional. Thanks for calling it out
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Migrate ITTLSTest to embedded-tests (druid)
Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz closed pull request #19037: Migrate ITTLSTest to embedded-tests
URL: https://github.com/apache/druid/pull/19037
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Migrate `ITTLSTest` to embedded-tests (druid)
Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz commented on PR #19037:
URL: https://github.com/apache/druid/pull/19037#issuecomment-3945718884
Thanks a lot for the reviews, @Akshat-Jain , @Fly-Style !!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Migrate ITTLSTest to embedded-tests (druid)
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #19037:
URL: https://github.com/apache/druid/pull/19037#discussion_r2833669747
##########
embedded-tests/src/test/java/org/apache/druid/testing/embedded/auth/EmbeddedSSLAuthResource.java:
##########
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.testing.embedded.auth;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.druid.https.SSLContextModule;
+import org.apache.druid.java.util.common.ISE;
+import org.apache.druid.testing.embedded.EmbeddedDruidCluster;
+import org.apache.druid.testing.embedded.EmbeddedResource;
+
+import java.io.File;
+import java.io.IOException;
+
+/**
+ * {@link EmbeddedResource} used to enable SSL on Druid services.
+ * This resource generates the certificates and keystores needed by both the
+ * server and client, and then configures the Druid cluster with the appropriate
+ * keystore and truststore paths.
+ */
+public class EmbeddedSSLAuthResource implements EmbeddedResource
+{
+ private EmbeddedDruidCluster cluster;
+
+ @Override
+ public void beforeStart(EmbeddedDruidCluster cluster)
+ {
+ this.cluster = cluster;
+ }
+
+ @Override
+ public void start() throws Exception
+ {
+ final File tlsDir = getTlsTempDirectory();
+
+ final File tlsScripts = new File(tlsDir, "scripts");
+ copyScriptsToDirectory(tlsScripts);
+
+ // Generate client certificates and keystores
+ final ProcessBuilder generateClientCertificates = new ProcessBuilder(
+ "bash",
+ new File(tlsScripts, "generate-client-certs-and-keystores.sh").getAbsolutePath()
+ );
+ generateClientCertificates.directory(tlsDir);
+ generateClientCertificates.redirectErrorStream(true);
+ generateClientCertificates.redirectOutput(ProcessBuilder.Redirect.INHERIT);
+
+ int exitCode = generateClientCertificates.start().waitFor();
+ if (exitCode != 0) {
+ throw new ISE("Client certificate generation failed");
+ }
+
+ // Generate server certificates and keystores
+ final ProcessBuilder generateServerCertificates = new ProcessBuilder(
+ "bash",
Review Comment:
## Executing a command with a relative path
Command with a relative path 'bash' is executed.
[Show more details](https://github.com/apache/druid/security/code-scanning/10820)
##########
embedded-tests/src/test/java/org/apache/druid/testing/embedded/auth/EmbeddedSSLAuthResource.java:
##########
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.testing.embedded.auth;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.druid.https.SSLContextModule;
+import org.apache.druid.java.util.common.ISE;
+import org.apache.druid.testing.embedded.EmbeddedDruidCluster;
+import org.apache.druid.testing.embedded.EmbeddedResource;
+
+import java.io.File;
+import java.io.IOException;
+
+/**
+ * {@link EmbeddedResource} used to enable SSL on Druid services.
+ * This resource generates the certificates and keystores needed by both the
+ * server and client, and then configures the Druid cluster with the appropriate
+ * keystore and truststore paths.
+ */
+public class EmbeddedSSLAuthResource implements EmbeddedResource
+{
+ private EmbeddedDruidCluster cluster;
+
+ @Override
+ public void beforeStart(EmbeddedDruidCluster cluster)
+ {
+ this.cluster = cluster;
+ }
+
+ @Override
+ public void start() throws Exception
+ {
+ final File tlsDir = getTlsTempDirectory();
+
+ final File tlsScripts = new File(tlsDir, "scripts");
+ copyScriptsToDirectory(tlsScripts);
+
+ // Generate client certificates and keystores
+ final ProcessBuilder generateClientCertificates = new ProcessBuilder(
+ "bash",
Review Comment:
## Executing a command with a relative path
Command with a relative path 'bash' is executed.
[Show more details](https://github.com/apache/druid/security/code-scanning/10819)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Migrate `ITTLSTest` to embedded-tests (druid)
Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz commented on code in PR #19037:
URL: https://github.com/apache/druid/pull/19037#discussion_r2839561078
##########
embedded-tests/src/test/java/org/apache/druid/testing/embedded/auth/TLSTest.java:
##########
@@ -0,0 +1,480 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.testing.embedded.auth;
+
+import com.google.common.base.Throwables;
+import org.apache.druid.guice.http.DruidHttpClientConfig;
+import org.apache.druid.java.util.common.ISE;
+import org.apache.druid.java.util.common.RetryUtils;
+import org.apache.druid.java.util.common.StringUtils;
+import org.apache.druid.java.util.common.lifecycle.Lifecycle;
+import org.apache.druid.java.util.http.client.CredentialedHttpClient;
+import org.apache.druid.java.util.http.client.HttpClient;
+import org.apache.druid.java.util.http.client.HttpClientConfig;
+import org.apache.druid.java.util.http.client.HttpClientInit;
+import org.apache.druid.java.util.http.client.Request;
+import org.apache.druid.java.util.http.client.auth.BasicCredentials;
+import org.apache.druid.java.util.http.client.response.StatusResponseHandler;
+import org.apache.druid.java.util.http.client.response.StatusResponseHolder;
+import org.apache.druid.metadata.DefaultPasswordProvider;
+import org.apache.druid.server.security.TLSCertificateChecker;
+import org.apache.druid.server.security.TLSUtils;
+import org.apache.druid.testing.embedded.EmbeddedBroker;
+import org.apache.druid.testing.embedded.EmbeddedCoordinator;
+import org.apache.druid.testing.embedded.EmbeddedDruidCluster;
+import org.apache.druid.testing.embedded.EmbeddedDruidServer;
+import org.apache.druid.testing.embedded.EmbeddedHistorical;
+import org.apache.druid.testing.embedded.EmbeddedOverlord;
+import org.apache.druid.testing.embedded.EmbeddedRouter;
+import org.apache.druid.testing.embedded.junit5.EmbeddedClusterTestBase;
+import org.jboss.netty.handler.codec.http.HttpMethod;
+import org.jboss.netty.handler.codec.http.HttpResponseStatus;
+import org.joda.time.Duration;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLException;
+import java.io.IOException;
+import java.net.URL;
+import java.nio.channels.ClosedChannelException;
+
+/**
+ * Embedded test to verify communication with Druid services over TLS with
+ * various certificate states - expired, revoked, invalid hostname, etc.
+ */
+public class TLSTest extends EmbeddedClusterTestBase
Review Comment:
The assertions being made in this test are the same as the ones originally done in `ITTLSTest`.
Several utility methods are also the same. But since the file has changed so much, it shows up as an addition rather than as a modification.
##########
embedded-tests/src/test/java/org/apache/druid/testing/embedded/auth/TLSTest.java:
##########
@@ -0,0 +1,480 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.testing.embedded.auth;
+
+import com.google.common.base.Throwables;
+import org.apache.druid.guice.http.DruidHttpClientConfig;
+import org.apache.druid.java.util.common.ISE;
+import org.apache.druid.java.util.common.RetryUtils;
+import org.apache.druid.java.util.common.StringUtils;
+import org.apache.druid.java.util.common.lifecycle.Lifecycle;
+import org.apache.druid.java.util.http.client.CredentialedHttpClient;
+import org.apache.druid.java.util.http.client.HttpClient;
+import org.apache.druid.java.util.http.client.HttpClientConfig;
+import org.apache.druid.java.util.http.client.HttpClientInit;
+import org.apache.druid.java.util.http.client.Request;
+import org.apache.druid.java.util.http.client.auth.BasicCredentials;
+import org.apache.druid.java.util.http.client.response.StatusResponseHandler;
+import org.apache.druid.java.util.http.client.response.StatusResponseHolder;
+import org.apache.druid.metadata.DefaultPasswordProvider;
+import org.apache.druid.server.security.TLSCertificateChecker;
+import org.apache.druid.server.security.TLSUtils;
+import org.apache.druid.testing.embedded.EmbeddedBroker;
+import org.apache.druid.testing.embedded.EmbeddedCoordinator;
+import org.apache.druid.testing.embedded.EmbeddedDruidCluster;
+import org.apache.druid.testing.embedded.EmbeddedDruidServer;
+import org.apache.druid.testing.embedded.EmbeddedHistorical;
+import org.apache.druid.testing.embedded.EmbeddedOverlord;
+import org.apache.druid.testing.embedded.EmbeddedRouter;
+import org.apache.druid.testing.embedded.junit5.EmbeddedClusterTestBase;
+import org.jboss.netty.handler.codec.http.HttpMethod;
+import org.jboss.netty.handler.codec.http.HttpResponseStatus;
+import org.joda.time.Duration;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLException;
+import java.io.IOException;
+import java.net.URL;
+import java.nio.channels.ClosedChannelException;
+
+/**
+ * Embedded test to verify communication with Druid services over TLS with
+ * various certificate states - expired, revoked, invalid hostname, etc.
+ */
+public class TLSTest extends EmbeddedClusterTestBase
Review Comment:
**Note for reviewer:**
The assertions being made in this test are the same as the ones originally done in `ITTLSTest`.
Several utility methods are also the same. But since the file has changed so much, it shows up as an addition rather than as a modification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Migrate `ITTLSTest` to embedded-tests (druid)
Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz merged PR #19037:
URL: https://github.com/apache/druid/pull/19037
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Migrate `ITTLSTest` to embedded-tests (druid)
Posted by "Fly-Style (via GitHub)" <gi...@apache.org>.
Fly-Style commented on code in PR #19037:
URL: https://github.com/apache/druid/pull/19037#discussion_r2839884924
##########
embedded-tests/src/test/java/org/apache/druid/testing/embedded/auth/TestTLSCertificateChecker.java:
##########
@@ -40,7 +40,7 @@ public void checkClient(
) throws CertificateException
{
// only the integration test client with "thisisprobablynottherighthostname" cert is allowed to talk to me
- if (!chain[0].toString().contains("thisisprobablynottherighthostname") || !engine.getPeerHost().contains("172.172.172.1")) {
Review Comment:
Just for attention focus: hopefully it is not an accident removal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Migrate `ITTLSTest` to embedded-tests (druid)
Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz commented on PR #19037:
URL: https://github.com/apache/druid/pull/19037#issuecomment-3945912249
Merged since failure was unrelated
```
java.lang.RuntimeException: Failed to execute helm command
Caused by: java.lang.RuntimeException: Helm command failed: repo add datainfra https://charts.datainfra.io/
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org