You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/09/15 09:17:15 UTC
[GitHub] [druid] maytasm opened a new pull request #11711: Add new config to filter internal Druid-related messages from Query API response
maytasm opened a new pull request #11711:
URL: https://github.com/apache/druid/pull/11711
Add new config to filter internal Druid-related messages from Query API response
### Description
When this new config is enabled, Druid Query API responses will hiding internal information (class name, stack trace, internal concept, thread name / servlet name, code, line/column number, host/ip). This is useful for when we have external users using Druid query API.
For the initial functionality of this feature, we will handle the following Query API responses:
- Jetty layer / Jetty filter (authentication filter, cannot route/connect to downstream services, limit request, http allow method, etc.)
- In AsyncQueryForwardingServlet of the Router
- SqlResource (/druid/v2/sql/ POST endpoint)
The effect of this feature for each layer is described below:
- Without this feature enabled, any error in the Jetty layer / Jetty filter will have `"servlet"`, `"message"`, "url"`, `"status"` and `"cause"` fields in the response. With this feature enabled, the response will only contains `"message"`, "url"`, and `"status"`. Note that the value of these fields remain unchange.
-
This PR has:
- [ ] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713062030
##########
File path: core/src/main/java/org/apache/druid/common/exception/SanitizableException.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import java.util.function.Function;
+
+public interface SanitizableException
Review comment:
This shouldn't effect/change the UI/UX or configs/settings so I think if we want, we can easily rename it later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r709995248
##########
File path: sql/src/main/java/org/apache/druid/sql/http/SqlResource.java
##########
@@ -271,4 +332,14 @@ public Response cancelQuery(
return Response.status(Status.FORBIDDEN).build();
}
}
+
+ @VisibleForTesting
+ String applyErrorMessageFilter(final String errorMessage, final List<Pattern> whitelistRegex)
Review comment:
Moved to a common class so that we can reuse in multiple places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713728271
##########
File path: core/src/main/java/org/apache/druid/query/QueryException.java
##########
@@ -96,4 +98,10 @@ protected static String resolveHostname()
return null;
}
}
+
+ @Override
+ public QueryException transform(Function<String, String> errorMessageTransformFunction)
+ {
+ return new QueryException(errorCode, errorMessageTransformFunction.apply(getMessage()), null, null);
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713752402
##########
File path: server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java
##########
@@ -235,14 +257,15 @@ public boolean equals(Object o)
queueSize == that.queueSize &&
enableRequestLimit == that.enableRequestLimit &&
defaultQueryTimeout == that.defaultQueryTimeout &&
- maxScatterGatherBytes.equals(that.maxScatterGatherBytes) &&
maxSubqueryRows == that.maxSubqueryRows &&
maxQueryTimeout == that.maxQueryTimeout &&
maxRequestHeaderSize == that.maxRequestHeaderSize &&
inflateBufferSize == that.inflateBufferSize &&
compressionLevel == that.compressionLevel &&
enableForwardedRequestCustomizer == that.enableForwardedRequestCustomizer &&
+ sanitizeJettyErrorResponse == that.sanitizeJettyErrorResponse &&
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713705170
##########
File path: core/src/main/java/org/apache/druid/common/exception/SanitizableException.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import java.util.function.Function;
+
+public interface SanitizableException
+{
+ /**
+ * Apply the function for transforming the error message then
+ * return new Exception with sanitized fields and transformed message.
+ */
+ Exception transform(
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713704162
##########
File path: core/src/main/java/org/apache/druid/common/exception/SanitizableException.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import java.util.function.Function;
+
+public interface SanitizableException
+{
+ /**
+ * Apply the function for transforming the error message then
+ * return new Exception with sanitized fields and transformed message.
+ */
+ Exception transform(
+ Function<String, String> errorMessageTransformFunction
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] suneet-s commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
suneet-s commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r714042149
##########
File path: core/src/main/java/org/apache/druid/common/exception/SanitizableException.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import java.util.function.Function;
+
+public interface SanitizableException
Review comment:
> Instead of calling this an Exception - let's make this a Factory class since the interface can't extend an Exception
Please ignore this suggestion. It makes the `ErrorResponseTransformStrategy` clunky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
clintropolis commented on pull request #11711:
URL: https://github.com/apache/druid/pull/11711#issuecomment-920696596
marking design review because it both adds new config and said config changes API responses 😅
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] suneet-s commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
suneet-s commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713585330
##########
File path: core/src/main/java/org/apache/druid/common/exception/AllowedRegexErrorResponseTransformStrategy.java
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import com.fasterxml.jackson.annotation.JsonCreator;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.google.common.collect.ImmutableList;
+
+import java.util.List;
+import java.util.function.Function;
+import java.util.regex.Pattern;
+
+public class AllowedRegexErrorResponseTransformStrategy implements ErrorResponseTransformStrategy
+{
+ @JsonProperty
+ private final List<Pattern> allowedRegex;
+
+ @JsonCreator
+ public AllowedRegexErrorResponseTransformStrategy(
+ @JsonProperty("allowedRegex") List<Pattern> allowedRegex
+ )
+ {
+ this.allowedRegex = allowedRegex == null ? ImmutableList.of() : allowedRegex;
+ }
+
+ @Override
+ public Function<String, String> getErrorMessageTransformFunction()
+ {
+ return (String errorMessage) -> {
+ if (allowedRegex.stream().noneMatch(pattern -> pattern.matcher(errorMessage).matches())) {
Review comment:
nit: Is it better to use `anyMatch` to return the errorMessage so the stream can short circuit and exit early?
##########
File path: core/src/main/java/org/apache/druid/common/exception/SanitizableException.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import java.util.function.Function;
+
+public interface SanitizableException
+{
+ /**
+ * Apply the function for transforming the error message then
+ * return new Exception with sanitized fields and transformed message.
+ */
+ Exception transform(
Review comment:
nit: rename to `sanitize` to be inline with `SanitizableException`
##########
File path: core/src/main/java/org/apache/druid/common/exception/SanitizableException.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import java.util.function.Function;
+
+public interface SanitizableException
Review comment:
javadocs please. Explanation about how this is expected to be used and why this class exists will be helpful.
##########
File path: core/src/main/java/org/apache/druid/common/exception/SanitizableException.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import java.util.function.Function;
+
+public interface SanitizableException
+{
+ /**
+ * Apply the function for transforming the error message then
+ * return new Exception with sanitized fields and transformed message.
+ */
+ Exception transform(
+ Function<String, String> errorMessageTransformFunction
Review comment:
Can you add details on why only the errorMessage is expected to be transformed in this interface to the javadoc please
##########
File path: server/src/main/java/org/apache/druid/server/security/ForbiddenException.java
##########
@@ -44,4 +48,14 @@ public String getErrorMessage()
{
return super.getMessage();
}
+
+ @Override
+ public ForbiddenException transform(Function<String, String> errorMessageTransformFunction)
+ {
+ if (Strings.isNullOrEmpty(errorMessageTransformFunction.apply(getMessage()))) {
+ return new ForbiddenException();
+ } else {
+ return this;
Review comment:
appears to be a logic bug here
unit tests please
##########
File path: core/src/main/java/org/apache/druid/query/QueryException.java
##########
@@ -96,4 +98,10 @@ protected static String resolveHostname()
return null;
}
}
+
+ @Override
+ public QueryException transform(Function<String, String> errorMessageTransformFunction)
+ {
+ return new QueryException(errorCode, errorMessageTransformFunction.apply(getMessage()), null, null);
Review comment:
can we add unit tests for this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
clintropolis commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713766086
##########
File path: server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java
##########
@@ -61,7 +63,9 @@ public ServerConfig(
int inflateBufferSize,
int compressionLevel,
boolean enableForwardedRequestCustomizer,
- @NotNull List<String> allowedHttpMethods
+ @NotNull List<String> allowedHttpMethods,
+ boolean sanitizeJettyErrorResponse,
Review comment:
sorry i got confused about `sanitizeJettyErrorResponse`, it shouldn't be renamed to `enableErrorResponseTransform`, though `sanitizeDruidErrorResponse` should still be renamed `errorResponseTransformStrategy`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713053940
##########
File path: sql/src/main/java/org/apache/druid/sql/http/SqlResource.java
##########
@@ -170,39 +174,50 @@ public Response doPost(
}
catch (QueryCapacityExceededException cap) {
endLifecycle(sqlQueryId, lifecycle, cap, remoteAddr, -1);
- return buildNonOkResponse(QueryCapacityExceededException.STATUS_CODE, cap);
+ return buildNonOkResponse(
+ QueryCapacityExceededException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(cap)
+ );
}
catch (QueryUnsupportedException unsupported) {
endLifecycle(sqlQueryId, lifecycle, unsupported, remoteAddr, -1);
- return buildNonOkResponse(QueryUnsupportedException.STATUS_CODE, unsupported);
+ return buildNonOkResponse(
+ QueryUnsupportedException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(unsupported)
+ );
}
catch (QueryTimeoutException timeout) {
endLifecycle(sqlQueryId, lifecycle, timeout, remoteAddr, -1);
- return buildNonOkResponse(QueryTimeoutException.STATUS_CODE, timeout);
+ return buildNonOkResponse(
+ QueryTimeoutException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(timeout)
+ );
}
catch (SqlPlanningException | ResourceLimitExceededException e) {
endLifecycle(sqlQueryId, lifecycle, e, remoteAddr, -1);
- return buildNonOkResponse(BadQueryException.STATUS_CODE, e);
+ return buildNonOkResponse(
+ BadQueryException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(e)
+ );
}
catch (ForbiddenException e) {
endLifecycleWithoutEmittingMetrics(sqlQueryId, lifecycle);
- throw e; // let ForbiddenExceptionMapper handle this
+ throw (ForbiddenException) serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(e); // let ForbiddenExceptionMapper handle this
}
catch (Exception e) {
log.warn(e, "Failed to handle query: %s", sqlQuery);
endLifecycle(sqlQueryId, lifecycle, e, remoteAddr, -1);
- final Exception exceptionToReport;
+ QueryInterruptedException exceptionToReport;
if (e instanceof RelOptPlanner.CannotPlanException) {
- exceptionToReport = new ISE("Cannot build plan for query: %s", sqlQuery.getQuery());
+ exceptionToReport = QueryInterruptedException.wrapIfNeeded((new ISE("Cannot build plan for query: %s", sqlQuery.getQuery())));
Review comment:
Actually now that I look at it...I didn't have to change/refactor those and can just call `serverConfig.getErrorResponseTransformStrategy().transformIfNeeded` at the end
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713064799
##########
File path: core/src/main/java/org/apache/druid/common/exception/SanitizableException.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import java.util.function.Function;
+
+public interface SanitizableException
+{
+ /**
+ * Apply the function for transforming the error message then
+ * return new Exception with sanitized fields and transformed message.
+ */
+ Exception applyErrorMessageTransformAndSanitizeFields(
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
clintropolis commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r712588136
##########
File path: sql/src/main/java/org/apache/druid/sql/http/SqlResource.java
##########
@@ -170,39 +174,50 @@ public Response doPost(
}
catch (QueryCapacityExceededException cap) {
endLifecycle(sqlQueryId, lifecycle, cap, remoteAddr, -1);
- return buildNonOkResponse(QueryCapacityExceededException.STATUS_CODE, cap);
+ return buildNonOkResponse(
+ QueryCapacityExceededException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(cap)
+ );
}
catch (QueryUnsupportedException unsupported) {
endLifecycle(sqlQueryId, lifecycle, unsupported, remoteAddr, -1);
- return buildNonOkResponse(QueryUnsupportedException.STATUS_CODE, unsupported);
+ return buildNonOkResponse(
+ QueryUnsupportedException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(unsupported)
+ );
}
catch (QueryTimeoutException timeout) {
endLifecycle(sqlQueryId, lifecycle, timeout, remoteAddr, -1);
- return buildNonOkResponse(QueryTimeoutException.STATUS_CODE, timeout);
+ return buildNonOkResponse(
+ QueryTimeoutException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(timeout)
+ );
}
catch (SqlPlanningException | ResourceLimitExceededException e) {
endLifecycle(sqlQueryId, lifecycle, e, remoteAddr, -1);
- return buildNonOkResponse(BadQueryException.STATUS_CODE, e);
+ return buildNonOkResponse(
+ BadQueryException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(e)
+ );
}
catch (ForbiddenException e) {
endLifecycleWithoutEmittingMetrics(sqlQueryId, lifecycle);
- throw e; // let ForbiddenExceptionMapper handle this
+ throw (ForbiddenException) serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(e); // let ForbiddenExceptionMapper handle this
}
catch (Exception e) {
log.warn(e, "Failed to handle query: %s", sqlQuery);
endLifecycle(sqlQueryId, lifecycle, e, remoteAddr, -1);
- final Exception exceptionToReport;
+ QueryInterruptedException exceptionToReport;
if (e instanceof RelOptPlanner.CannotPlanException) {
- exceptionToReport = new ISE("Cannot build plan for query: %s", sqlQuery.getQuery());
+ exceptionToReport = QueryInterruptedException.wrapIfNeeded((new ISE("Cannot build plan for query: %s", sqlQuery.getQuery())));
Review comment:
nit: this isn't new or your change, but I think it is semi confusing that there exists a`SqlPlanningException` that isn't being used here. The reason for this is because I think that this is a different kind of planning exception - one that occurs for some query which we couldn't translate to a native Druid query, which we seem to be treating as a server-side error, where the other one is currently used for SQL parsing and validation errors and considered client side errors since they are things the user can fix.
I wonder if we should have some sort of unsupported query exception to give this a bit more prominence, but I think that is a tangential discussion to this PRs changed.
##########
File path: core/src/main/java/org/apache/druid/common/exception/NoErrorResponseTransformStrategy.java
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import org.apache.druid.java.util.common.IAE;
+
+import java.util.function.Function;
+
+/**
+ * Error response transform strategy that does nothing and simply return the same Exception back without any change
+ */
+public class NoErrorResponseTransformStrategy implements ErrorResponseTransformStrategy
+{
+ public static final NoErrorResponseTransformStrategy INSTANCE = new NoErrorResponseTransformStrategy();
+
+ @Override
+ public Exception transformIfNeeded(SanitizableException exception)
+ {
+ if (exception instanceof Exception) {
+ return (Exception) exception;
+ } else {
+ return new IAE(
+ "Cannot sanitize error response as given argument[%s] is not an Exception",
+ exception.getClass().getName()
+ );
+ }
Review comment:
why not just `return exception;` and be a no-op like the javadoc says?
##########
File path: sql/src/main/java/org/apache/druid/sql/http/SqlResource.java
##########
@@ -170,39 +174,50 @@ public Response doPost(
}
catch (QueryCapacityExceededException cap) {
endLifecycle(sqlQueryId, lifecycle, cap, remoteAddr, -1);
- return buildNonOkResponse(QueryCapacityExceededException.STATUS_CODE, cap);
+ return buildNonOkResponse(
+ QueryCapacityExceededException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(cap)
+ );
}
catch (QueryUnsupportedException unsupported) {
endLifecycle(sqlQueryId, lifecycle, unsupported, remoteAddr, -1);
- return buildNonOkResponse(QueryUnsupportedException.STATUS_CODE, unsupported);
+ return buildNonOkResponse(
+ QueryUnsupportedException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(unsupported)
+ );
}
catch (QueryTimeoutException timeout) {
endLifecycle(sqlQueryId, lifecycle, timeout, remoteAddr, -1);
- return buildNonOkResponse(QueryTimeoutException.STATUS_CODE, timeout);
+ return buildNonOkResponse(
+ QueryTimeoutException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(timeout)
+ );
}
catch (SqlPlanningException | ResourceLimitExceededException e) {
endLifecycle(sqlQueryId, lifecycle, e, remoteAddr, -1);
- return buildNonOkResponse(BadQueryException.STATUS_CODE, e);
+ return buildNonOkResponse(
+ BadQueryException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(e)
+ );
}
catch (ForbiddenException e) {
endLifecycleWithoutEmittingMetrics(sqlQueryId, lifecycle);
- throw e; // let ForbiddenExceptionMapper handle this
+ throw (ForbiddenException) serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(e); // let ForbiddenExceptionMapper handle this
Review comment:
at some point in the future, assuming we eventually make this functionality apply to all API response errors, it probably makes sense to push this transformation functionality into the error handlers so that we everything that lets exceptions fall through doesn't need to do this itself. It is probably ok to leave here for now since this is the only thing doing it
##########
File path: sql/src/main/java/org/apache/druid/sql/http/SqlResource.java
##########
@@ -170,39 +174,50 @@ public Response doPost(
}
catch (QueryCapacityExceededException cap) {
endLifecycle(sqlQueryId, lifecycle, cap, remoteAddr, -1);
- return buildNonOkResponse(QueryCapacityExceededException.STATUS_CODE, cap);
+ return buildNonOkResponse(
Review comment:
it looks like all callers of `buildNonOkResponse` are calling `transformIfNeeded`, it makes sense to update the signature to just accept `SanitizableException`
##########
File path: server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java
##########
@@ -235,14 +257,15 @@ public boolean equals(Object o)
queueSize == that.queueSize &&
enableRequestLimit == that.enableRequestLimit &&
defaultQueryTimeout == that.defaultQueryTimeout &&
- maxScatterGatherBytes.equals(that.maxScatterGatherBytes) &&
maxSubqueryRows == that.maxSubqueryRows &&
maxQueryTimeout == that.maxQueryTimeout &&
maxRequestHeaderSize == that.maxRequestHeaderSize &&
inflateBufferSize == that.inflateBufferSize &&
compressionLevel == that.compressionLevel &&
enableForwardedRequestCustomizer == that.enableForwardedRequestCustomizer &&
+ sanitizeJettyErrorResponse == that.sanitizeJettyErrorResponse &&
Review comment:
hm, I wonder if technically `ErrorResponseTransformStrategy` should implement equals and be checked here
##########
File path: core/src/main/java/org/apache/druid/common/exception/SanitizableException.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import java.util.function.Function;
+
+public interface SanitizableException
+{
+ /**
+ * Apply the function for transforming the error message then
+ * return new Exception with sanitized fields and transformed message.
+ */
+ Exception applyErrorMessageTransformAndSanitizeFields(
Review comment:
nit: i think `transform` would be descriptive enough of a method name, between the parameter name and the javadocs.
##########
File path: core/src/main/java/org/apache/druid/common/exception/SanitizableException.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import java.util.function.Function;
+
+public interface SanitizableException
Review comment:
nit: if a longer term goal is to make sure all surface exceptions are wrapped in this to allow operators to tidy up the error responses, i wonder if we should take a stronger name for this interface like `ServiceApiException` or .. idk naming is hard.
##########
File path: sql/src/test/java/org/apache/druid/sql/http/SqlResourceTest.java
##########
@@ -968,6 +973,44 @@ public void testUnsupportedQueryThrowsException() throws Exception
Assert.assertTrue(lifecycleManager.getAll("id").isEmpty());
}
+ @Test
+ public void testUnsupportedQueryThrowsExceptionWithFilterResponse() throws Exception
+ {
+ resource = new SqlResource(
+ JSON_MAPPER,
+ CalciteTests.TEST_AUTHORIZER_MAPPER,
+ sqlLifecycleFactory,
+ lifecycleManager,
+ new ServerConfig() {
+ @Override
+ public boolean isSanitizeJettyErrorResponse()
+ {
+ return true;
+ }
+ @Override
+ public ErrorResponseTransformStrategy getErrorResponseTransformStrategy()
+ {
+ return new AllowedRegexErrorResponseTransformStrategy(ImmutableList.of());
+ }
+ }
+ );
+
+ String errorMessage = "This will be support in Druid 9999";
Review comment:
<img src="https://i.imgur.com/VlP5fgB.gif"></img>
##########
File path: sql/src/main/java/org/apache/druid/sql/http/SqlResource.java
##########
@@ -170,39 +174,50 @@ public Response doPost(
}
catch (QueryCapacityExceededException cap) {
endLifecycle(sqlQueryId, lifecycle, cap, remoteAddr, -1);
- return buildNonOkResponse(QueryCapacityExceededException.STATUS_CODE, cap);
+ return buildNonOkResponse(
+ QueryCapacityExceededException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(cap)
+ );
}
catch (QueryUnsupportedException unsupported) {
endLifecycle(sqlQueryId, lifecycle, unsupported, remoteAddr, -1);
- return buildNonOkResponse(QueryUnsupportedException.STATUS_CODE, unsupported);
+ return buildNonOkResponse(
+ QueryUnsupportedException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(unsupported)
+ );
}
catch (QueryTimeoutException timeout) {
endLifecycle(sqlQueryId, lifecycle, timeout, remoteAddr, -1);
- return buildNonOkResponse(QueryTimeoutException.STATUS_CODE, timeout);
+ return buildNonOkResponse(
+ QueryTimeoutException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(timeout)
+ );
}
catch (SqlPlanningException | ResourceLimitExceededException e) {
endLifecycle(sqlQueryId, lifecycle, e, remoteAddr, -1);
- return buildNonOkResponse(BadQueryException.STATUS_CODE, e);
+ return buildNonOkResponse(
+ BadQueryException.STATUS_CODE,
+ serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(e)
+ );
}
catch (ForbiddenException e) {
endLifecycleWithoutEmittingMetrics(sqlQueryId, lifecycle);
- throw e; // let ForbiddenExceptionMapper handle this
+ throw (ForbiddenException) serverConfig.getErrorResponseTransformStrategy().transformIfNeeded(e); // let ForbiddenExceptionMapper handle this
Review comment:
in fact, whenever in the future we do that, it might also make sense to move `QueryException` handling out to an `ExceptionMapper<QueryException>` and just rethrow all exceptions so that error response handling can be shared between sql and native json queries. it doesn't gain too much since will still need to handle internally so that metrics can be emitted, but it would cover the response writing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
clintropolis commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713631100
##########
File path: server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java
##########
@@ -61,7 +63,9 @@ public ServerConfig(
int inflateBufferSize,
int compressionLevel,
boolean enableForwardedRequestCustomizer,
- @NotNull List<String> allowedHttpMethods
+ @NotNull List<String> allowedHttpMethods,
+ boolean sanitizeJettyErrorResponse,
Review comment:
I think these configs should be called `enableErrorResponseTransform` and `errorResponseTransformStrategy` respectively, or something similar, since it is clearer their purpose (and also `ErrorResponseTransformStrategy` is the type name)
##########
File path: core/src/main/java/org/apache/druid/common/exception/SanitizableException.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import java.util.function.Function;
+
+public interface SanitizableException
+{
+ /**
+ * Apply the function for transforming the error message then
+ * return new Exception with sanitized fields and transformed message.
+ */
+ Exception transform(
Review comment:
heh oops, was my suggestion to rename `transform`, but I guess I also suggested renaming `SanitizableException` to something else, though I feel less strongly about the name of this than the property name in the config, (which I think should be `druid.server.http.enableErrorResponseTransform` and `druid.server.http.errorResponseTransformStrategy`)
##########
File path: server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java
##########
@@ -235,14 +257,15 @@ public boolean equals(Object o)
queueSize == that.queueSize &&
enableRequestLimit == that.enableRequestLimit &&
defaultQueryTimeout == that.defaultQueryTimeout &&
- maxScatterGatherBytes.equals(that.maxScatterGatherBytes) &&
maxSubqueryRows == that.maxSubqueryRows &&
maxQueryTimeout == that.maxQueryTimeout &&
maxRequestHeaderSize == that.maxRequestHeaderSize &&
inflateBufferSize == that.inflateBufferSize &&
compressionLevel == that.compressionLevel &&
enableForwardedRequestCustomizer == that.enableForwardedRequestCustomizer &&
+ sanitizeJettyErrorResponse == that.sanitizeJettyErrorResponse &&
Review comment:
yeah, I guess you would need to deserialize into strings and keep them around alongside a set of patterns to build (which might be worth doing?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r709928370
##########
File path: server/src/test/java/org/apache/druid/initialization/ServerConfigTest.java
##########
@@ -57,13 +58,16 @@ public void testSerde() throws Exception
defaultConfig.getInflateBufferSize(),
defaultConfig.getCompressionLevel(),
true,
- ImmutableList.of(HttpMethod.OPTIONS)
+ ImmutableList.of(HttpMethod.OPTIONS),
+ true,
+ ImmutableList.of(Pattern.compile(".*"))
);
String modifiedConfigJson = OBJECT_MAPPER.writeValueAsString(modifiedConfig);
ServerConfig modifiedConfig2 = OBJECT_MAPPER.readValue(modifiedConfigJson, ServerConfig.class);
Assert.assertEquals(modifiedConfig, modifiedConfig2);
Assert.assertEquals(999, modifiedConfig2.getNumThreads());
- Assert.assertEquals(888, modifiedConfig2.getQueueSize());
Review comment:
Oops. was accidental.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm merged pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm merged pull request #11711:
URL: https://github.com/apache/druid/pull/11711
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm merged pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm merged pull request #11711:
URL: https://github.com/apache/druid/pull/11711
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] suneet-s commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
suneet-s commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r714037773
##########
File path: core/src/main/java/org/apache/druid/common/exception/SanitizableException.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import java.util.function.Function;
+
+public interface SanitizableException
Review comment:
Instead of calling this an Exception - let's make this a Factory class since the interface can't extend an Exception.
```
public interface SanitizingExceptionFactory<T extends Exception>
{
T transform(T exception, Function<String, String> errorMessageTransformFunction);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713688591
##########
File path: core/src/main/java/org/apache/druid/common/exception/NoErrorResponseTransformStrategy.java
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import org.apache.druid.java.util.common.IAE;
+
+import java.util.function.Function;
+
+/**
+ * Error response transform strategy that does nothing and simply return the same Exception back without any change
+ */
+public class NoErrorResponseTransformStrategy implements ErrorResponseTransformStrategy
+{
+ public static final NoErrorResponseTransformStrategy INSTANCE = new NoErrorResponseTransformStrategy();
+
+ @Override
+ public Exception transformIfNeeded(SanitizableException exception)
+ {
+ if (exception instanceof Exception) {
+ return (Exception) exception;
+ } else {
+ return new IAE(
+ "Cannot sanitize error response as given argument[%s] is not an Exception",
+ exception.getClass().getName()
+ );
+ }
Review comment:
Changed to `return (Exception) exception;`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713728189
##########
File path: server/src/main/java/org/apache/druid/server/security/ForbiddenException.java
##########
@@ -44,4 +48,14 @@ public String getErrorMessage()
{
return super.getMessage();
}
+
+ @Override
+ public ForbiddenException transform(Function<String, String> errorMessageTransformFunction)
+ {
+ if (Strings.isNullOrEmpty(errorMessageTransformFunction.apply(getMessage()))) {
+ return new ForbiddenException();
+ } else {
+ return this;
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713064246
##########
File path: core/src/main/java/org/apache/druid/common/exception/NoErrorResponseTransformStrategy.java
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import org.apache.druid.java.util.common.IAE;
+
+import java.util.function.Function;
+
+/**
+ * Error response transform strategy that does nothing and simply return the same Exception back without any change
+ */
+public class NoErrorResponseTransformStrategy implements ErrorResponseTransformStrategy
+{
+ public static final NoErrorResponseTransformStrategy INSTANCE = new NoErrorResponseTransformStrategy();
+
+ @Override
+ public Exception transformIfNeeded(SanitizableException exception)
+ {
+ if (exception instanceof Exception) {
+ return (Exception) exception;
+ } else {
+ return new IAE(
+ "Cannot sanitize error response as given argument[%s] is not an Exception",
+ exception.getClass().getName()
+ );
+ }
Review comment:
The method takes in `SanitizableException` which isn't guarantee to be an `Exception`. If it not, then it would result in ClassCastException and wouldn't be a no-op too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713778696
##########
File path: server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java
##########
@@ -61,7 +63,9 @@ public ServerConfig(
int inflateBufferSize,
int compressionLevel,
boolean enableForwardedRequestCustomizer,
- @NotNull List<String> allowedHttpMethods
+ @NotNull List<String> allowedHttpMethods,
+ boolean sanitizeJettyErrorResponse,
Review comment:
We don't need `enableErrorResponseTransform` as not setting `errorResponseTransformStrategy` or setting `errorResponseTransformStrategy` to `none` would be the same as current behavior (no-op) and leaving the error response unchanged.
I renamed `sanitizeJettyErrorResponse` to `showDetailedJettyErrors` and flip it
also renamed `sanitizeDruidErrorResponse` to `errorResponseTransformStrategy`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r714125465
##########
File path: server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java
##########
@@ -61,7 +63,9 @@ public ServerConfig(
int inflateBufferSize,
int compressionLevel,
boolean enableForwardedRequestCustomizer,
- @NotNull List<String> allowedHttpMethods
+ @NotNull List<String> allowedHttpMethods,
+ boolean sanitizeJettyErrorResponse,
Review comment:
Actually it should be `errorResponseTransform` not `errorResponseTransformStrategy` for the @JsonProperty
so that the configs look something like
```
druid.server.http.showDetailedJettyErrors=false
druid.server.http.errorResponseTransform.strategy=allowedRegex
druid.server.http.errorResponseTransform.allowedRegex=["asd"]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r709914421
##########
File path: services/src/main/java/org/apache/druid/server/AsyncQueryForwardingServlet.java
##########
@@ -350,10 +369,18 @@ private void handleQueryParseException(
// Write to the response
response.setStatus(HttpServletResponse.SC_BAD_REQUEST);
response.setContentType(MediaType.APPLICATION_JSON);
- objectMapper.writeValue(
- response.getOutputStream(),
- ImmutableMap.of("error", errorMessage)
- );
+ if (serverConfig.isFilterResponse()
+ && serverConfig.getResponseWhitelistRegex().stream().noneMatch(pattern -> pattern.matcher(errorMessage).matches())) {
+ objectMapper.writeValue(
+ response.getOutputStream(),
+ ImmutableMap.of("error", DEFAULT_QUERY_PARSE_EXCEPTION_MESSAGE)
Review comment:
The response format here is inconsistent with `QueryException`. I am not sure why this was done this way. However, in this PR my intention is to keep the response format the same as original. Notice, that this PR does not add any new field in the response format, only remove fields that could already be null, and change string message for existing fields.
However, both Exception thrown here and the Exception from further downstream which uses `QueryException` already make the response inconsistent for the same API call. Since, the format of `QueryException` is well documented, I think we should change this to use `QueryException` too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
clintropolis commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713785862
##########
File path: server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java
##########
@@ -61,7 +63,9 @@ public ServerConfig(
int inflateBufferSize,
int compressionLevel,
boolean enableForwardedRequestCustomizer,
- @NotNull List<String> allowedHttpMethods
+ @NotNull List<String> allowedHttpMethods,
+ boolean sanitizeJettyErrorResponse,
Review comment:
yep sorry confused myself, thinking about too many things at once 🙃 , this sgtm :+1:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
clintropolis commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r709885043
##########
File path: services/src/main/java/org/apache/druid/server/AsyncQueryForwardingServlet.java
##########
@@ -350,10 +369,18 @@ private void handleQueryParseException(
// Write to the response
response.setStatus(HttpServletResponse.SC_BAD_REQUEST);
response.setContentType(MediaType.APPLICATION_JSON);
- objectMapper.writeValue(
- response.getOutputStream(),
- ImmutableMap.of("error", errorMessage)
- );
+ if (serverConfig.isFilterResponse()
+ && serverConfig.getResponseWhitelistRegex().stream().noneMatch(pattern -> pattern.matcher(errorMessage).matches())) {
+ objectMapper.writeValue(
+ response.getOutputStream(),
+ ImmutableMap.of("error", DEFAULT_QUERY_PARSE_EXCEPTION_MESSAGE)
Review comment:
Maybe it is finally time to make a basic `ErrorResponse` class to capture this `ImmutableMap.of("error", ...)` pattern, searching the code-base I see it in a lot of places...
On the other hand, it is somewhat inconsistent with `QueryException` though, which uses `error` for the errorCode and has separate message, class, and some additional information. Since this is specifically forwarding queries, I wonder if this should be a type of `QueryException`? (Especially if we bake a method into `QueryException` to make them clean themselves up if configured as such)
##########
File path: server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java
##########
@@ -145,6 +150,13 @@ public ServerConfig()
@NotNull
private List<String> allowedHttpMethods = ImmutableList.of();
+ @JsonProperty
+ @NotNull
+ private List<Pattern> responseWhitelistRegex = ImmutableList.of();
Review comment:
this config applies to the http servers, not queries specifically, so I think if it is going to live here it might need a name that indicates it only applies to queries, unless you imagine that at some point this setting will apply to all HTTP error responses?
Alternatively, if it lived on a more query centric config object this wouldn't be necessary, though i'm not quite sure off the top of my head what is most appropriate.
##########
File path: server/src/test/java/org/apache/druid/initialization/ServerConfigTest.java
##########
@@ -57,13 +58,16 @@ public void testSerde() throws Exception
defaultConfig.getInflateBufferSize(),
defaultConfig.getCompressionLevel(),
true,
- ImmutableList.of(HttpMethod.OPTIONS)
+ ImmutableList.of(HttpMethod.OPTIONS),
+ true,
+ ImmutableList.of(Pattern.compile(".*"))
);
String modifiedConfigJson = OBJECT_MAPPER.writeValueAsString(modifiedConfig);
ServerConfig modifiedConfig2 = OBJECT_MAPPER.readValue(modifiedConfigJson, ServerConfig.class);
Assert.assertEquals(modifiedConfig, modifiedConfig2);
Assert.assertEquals(999, modifiedConfig2.getNumThreads());
- Assert.assertEquals(888, modifiedConfig2.getQueueSize());
Review comment:
nit: why remove?
##########
File path: sql/src/main/java/org/apache/druid/sql/http/SqlResource.java
##########
@@ -271,4 +332,14 @@ public Response cancelQuery(
return Response.status(Status.FORBIDDEN).build();
}
}
+
+ @VisibleForTesting
+ String applyErrorMessageFilter(final String errorMessage, final List<Pattern> whitelistRegex)
Review comment:
Maybe this should be a utility method on `QueryException` or someplace similar so it can be better shared for when JDBC and native JSON queries are added (also a few other places i see this `.stream().noneMatch(pattern -> pattern.matcher(errorMessage).matches()))` pattern in this PR that could probably be shared). If you imagine this will only ever be done to queries, maybe it makes sense to just bake method into `QueryException` so that all implementations just know how to trim the information they contain, then everything just has to make sure they throw some sort of `QueryException` and everything should be good.
Thinking further though, if this functionality only applies to queries, do we really need to use regex? Could we instead just configure a set of exception message by errorCode or maybe the classes of `QueryException` (not errorClass, but like java class name) to allow, and anything that doesn't match would be sanitized? I worry that regex based on the error messages might be a bit tricky/steep learning curve, and if we go this way we should probably update the documents to include several examples of the types of messages that query errors can encounter, and example configs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713703461
##########
File path: core/src/main/java/org/apache/druid/common/exception/AllowedRegexErrorResponseTransformStrategy.java
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.common.exception;
+
+import com.fasterxml.jackson.annotation.JsonCreator;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.google.common.collect.ImmutableList;
+
+import java.util.List;
+import java.util.function.Function;
+import java.util.regex.Pattern;
+
+public class AllowedRegexErrorResponseTransformStrategy implements ErrorResponseTransformStrategy
+{
+ @JsonProperty
+ private final List<Pattern> allowedRegex;
+
+ @JsonCreator
+ public AllowedRegexErrorResponseTransformStrategy(
+ @JsonProperty("allowedRegex") List<Pattern> allowedRegex
+ )
+ {
+ this.allowedRegex = allowedRegex == null ? ImmutableList.of() : allowedRegex;
+ }
+
+ @Override
+ public Function<String, String> getErrorMessageTransformFunction()
+ {
+ return (String errorMessage) -> {
+ if (allowedRegex.stream().noneMatch(pattern -> pattern.matcher(errorMessage).matches())) {
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r709921117
##########
File path: server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java
##########
@@ -145,6 +150,13 @@ public ServerConfig()
@NotNull
private List<String> allowedHttpMethods = ImmutableList.of();
+ @JsonProperty
+ @NotNull
+ private List<Pattern> responseWhitelistRegex = ImmutableList.of();
Review comment:
I imagine the setting for filterResponse will apply to all HTTP error responses. Currently it does apply to API outside of query APIs if the error is thrown in the Jetty filter layer. For example, if you access non-query API with invalid auth or an invalid API endpoint.
The setting for responseWhitelistRegex...I am not sure yet. I think it can apply to all HTTP error responses. Another idea I had was instead of using a whitelist, we can use a blacklist regex. Then it would be simpler to blacklist with regex like `org\.apache\..*` and `java\..*` to filter out stack traces. This would then be easy to apply to all HTTP error repsonses.
##########
File path: server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java
##########
@@ -145,6 +150,13 @@ public ServerConfig()
@NotNull
private List<String> allowedHttpMethods = ImmutableList.of();
+ @JsonProperty
+ @NotNull
+ private List<Pattern> responseWhitelistRegex = ImmutableList.of();
Review comment:
I imagine the setting for filterResponse will apply to all HTTP error responses. Currently it does apply to API outside of query APIs if the error is thrown in the Jetty filter layer. For example, if you access non-query API with invalid auth or an invalid API endpoint.
The setting for responseWhitelistRegex...I am not sure yet. I think it can apply to all HTTP error responses. Another idea I had was instead of using a whitelist, we can use a blacklist regex. Then it would be simpler to blacklist with regex like `org\.apache\..*` and `java\..*` to filter out stack traces. This would then be easy to apply to all HTTP error responses.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713059071
##########
File path: sql/src/main/java/org/apache/druid/sql/http/SqlResource.java
##########
@@ -170,39 +174,50 @@ public Response doPost(
}
catch (QueryCapacityExceededException cap) {
endLifecycle(sqlQueryId, lifecycle, cap, remoteAddr, -1);
- return buildNonOkResponse(QueryCapacityExceededException.STATUS_CODE, cap);
+ return buildNonOkResponse(
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm commented on a change in pull request #11711: Add new config to filter internal Druid-related messages from Query API response
Posted by GitBox <gi...@apache.org>.
maytasm commented on a change in pull request #11711:
URL: https://github.com/apache/druid/pull/11711#discussion_r713060713
##########
File path: server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java
##########
@@ -235,14 +257,15 @@ public boolean equals(Object o)
queueSize == that.queueSize &&
enableRequestLimit == that.enableRequestLimit &&
defaultQueryTimeout == that.defaultQueryTimeout &&
- maxScatterGatherBytes.equals(that.maxScatterGatherBytes) &&
maxSubqueryRows == that.maxSubqueryRows &&
maxQueryTimeout == that.maxQueryTimeout &&
maxRequestHeaderSize == that.maxRequestHeaderSize &&
inflateBufferSize == that.inflateBufferSize &&
compressionLevel == that.compressionLevel &&
enableForwardedRequestCustomizer == that.enableForwardedRequestCustomizer &&
+ sanitizeJettyErrorResponse == that.sanitizeJettyErrorResponse &&
Review comment:
I was having problem with the `AllowedRegexErrorResponseTransformStrategy` as the Pattern class doesn't implement equals method. So i just decided to exclude the ErrorResponseTransformStrategy here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org