You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by "moresandeep (via GitHub)" <gi...@apache.org> on 2023/03/21 20:41:13 UTC
[GitHub] [knox] moresandeep opened a new pull request, #742: KNOX-2890 - Prevent non-idempotent requests from failing over
moresandeep opened a new pull request, #742:
URL: https://github.com/apache/knox/pull/742
## What changes were proposed in this pull request?
Make sure we don't failover for non-idempotent requests such as POST. The request will anyways fail because the input response stream has already been closed so there is no point "retrying" the request.
## How was this patch tested?
Tested locally, added unit tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] moresandeep commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "moresandeep (via GitHub)" <gi...@apache.org>.
moresandeep commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146883382
##########
gateway-service-webhdfs/src/main/java/org/apache/knox/gateway/hdfs/dispatch/AbstractHdfsHaDispatch.java:
##########
@@ -59,33 +59,19 @@ protected void executeRequest(HttpUriRequest outboundRequest, HttpServletRequest
try {
inboundResponse = executeOutboundRequest(outboundRequest);
writeOutboundResponse(outboundRequest, inboundRequest, outboundResponse, inboundResponse);
- } catch (StandbyException e) {
+ } catch (StandbyException | SafeModeException | IOException e) {
/* if non-idempotent requests are not allowed to failover */
- if(!failoverNonIdempotentRequestEnabled && idempotentRequests.stream().anyMatch(outboundRequest.getMethod()::equalsIgnoreCase)) {
- LOG.cannotFailoverToNonIdempotentRequest(outboundRequest.getMethod());
+ if(!failoverNonIdempotentRequestEnabled && nonIdempotentRequests.stream().anyMatch(outboundRequest.getMethod()::equalsIgnoreCase)) {
+ LOG.cannotFailoverNonIdempotentRequest(outboundRequest.getMethod(), e.toString());
throw e;
} else {
- LOG.errorReceivedFromStandbyNode(e);
- failoverRequest(outboundRequest, inboundRequest, outboundResponse,
- inboundResponse, e);
- }
- } catch (SafeModeException e) {
- /* if non-idempotent requests are not allowed to failover */
- if(!failoverNonIdempotentRequestEnabled && idempotentRequests.stream().anyMatch(outboundRequest.getMethod()::equalsIgnoreCase)) {
- LOG.cannotFailoverToNonIdempotentRequest(outboundRequest.getMethod());
- throw e;
- } else {
- LOG.errorReceivedFromSafeModeNode(e);
- failoverRequest(outboundRequest, inboundRequest, outboundResponse,
- inboundResponse, e);
- }
- } catch (IOException e) {
- /* if non-idempotent requests are not allowed to failover */
- if(!failoverNonIdempotentRequestEnabled && idempotentRequests.stream().anyMatch(outboundRequest.getMethod()::equalsIgnoreCase)) {
- LOG.cannotFailoverToNonIdempotentRequest(outboundRequest.getMethod());
- throw e;
- } else {
- LOG.errorConnectingToServer(outboundRequest.getURI().toString(), e);
+ if(e instanceof StandbyException) {
Review Comment:
Nope, it's at the bottom, below the if else statement (new refactored into a method).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] pzampino commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "pzampino (via GitHub)" <gi...@apache.org>.
pzampino commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146404015
##########
gateway-service-webhdfs/src/main/java/org/apache/knox/gateway/hdfs/dispatch/AbstractHdfsHaDispatch.java:
##########
@@ -78,14 +60,35 @@ protected void executeRequest(HttpUriRequest outboundRequest, HttpServletRequest
inboundResponse = executeOutboundRequest(outboundRequest);
writeOutboundResponse(outboundRequest, inboundRequest, outboundResponse, inboundResponse);
} catch (StandbyException e) {
- LOG.errorReceivedFromStandbyNode(e);
- failoverRequest(outboundRequest, inboundRequest, outboundResponse, inboundResponse, e);
+ /* if non-idempotent requests are not allowed to failover */
Review Comment:
Could still be factored out to a common method though, I think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] moresandeep commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "moresandeep (via GitHub)" <gi...@apache.org>.
moresandeep commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146391125
##########
gateway-provider-ha/src/main/java/org/apache/knox/gateway/ha/dispatch/i18n/HaDispatchMessages.java:
##########
@@ -53,4 +53,7 @@ public interface HaDispatchMessages {
@Message(level = MessageLevel.ERROR, text = "Unsupported encoding, cause: {0}")
void unsupportedEncodingException(String cause);
+
+ @Message(level = MessageLevel.ERROR, text = "Request is non-idempotent {0}, failover prevented, to allow non-idempotent requests to failover set 'failoverNonIdempotentRequestEnabled=true' in HA config.")
+ void cannotFailoverToNonIdempotentRequest(String method);
Review Comment:
Ahh, will do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] moresandeep commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "moresandeep (via GitHub)" <gi...@apache.org>.
moresandeep commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146383037
##########
gateway-provider-ha/src/main/java/org/apache/knox/gateway/ha/dispatch/ConfigurableHADispatch.java:
##########
@@ -61,18 +61,20 @@ public class ConfigurableHADispatch extends ConfigurableDispatch {
protected static final HaDispatchMessages LOG = MessagesFactory.get(HaDispatchMessages.class);
- private int maxFailoverAttempts = HaServiceConfigConstants.DEFAULT_MAX_FAILOVER_ATTEMPTS;
+ protected int maxFailoverAttempts = HaServiceConfigConstants.DEFAULT_MAX_FAILOVER_ATTEMPTS;
- private int failoverSleep = HaServiceConfigConstants.DEFAULT_FAILOVER_SLEEP;
+ protected int failoverSleep = HaServiceConfigConstants.DEFAULT_FAILOVER_SLEEP;
- private HaProvider haProvider;
+ protected HaProvider haProvider;
private static final Map<String, String> urlToHashLookup = new HashMap<>();
private static final Map<String, String> hashToUrlLookup = new HashMap<>();
+ protected static final List<String> idempotentRequests = Arrays.asList("POST", "PATCH", "CONNECT");
Review Comment:
darn it, naming, i'll change it. Good Catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] pzampino commented on pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "pzampino (via GitHub)" <gi...@apache.org>.
pzampino commented on PR #742:
URL: https://github.com/apache/knox/pull/742#issuecomment-1481984428
LGTM, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] pzampino commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "pzampino (via GitHub)" <gi...@apache.org>.
pzampino commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146353661
##########
gateway-provider-ha/src/main/java/org/apache/knox/gateway/ha/dispatch/i18n/HaDispatchMessages.java:
##########
@@ -53,4 +53,7 @@ public interface HaDispatchMessages {
@Message(level = MessageLevel.ERROR, text = "Unsupported encoding, cause: {0}")
void unsupportedEncodingException(String cause);
+
+ @Message(level = MessageLevel.ERROR, text = "Request is non-idempotent {0}, failover prevented, to allow non-idempotent requests to failover set 'failoverNonIdempotentRequestEnabled=true' in HA config.")
+ void cannotFailoverToNonIdempotentRequest(String method);
Review Comment:
cannotFailoverNonIdempotentRequest(String method)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] moresandeep merged pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "moresandeep (via GitHub)" <gi...@apache.org>.
moresandeep merged PR #742:
URL: https://github.com/apache/knox/pull/742
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] zeroflag commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "zeroflag (via GitHub)" <gi...@apache.org>.
zeroflag commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146336678
##########
gateway-provider-ha/src/main/java/org/apache/knox/gateway/ha/dispatch/ConfigurableHADispatch.java:
##########
@@ -61,18 +61,20 @@ public class ConfigurableHADispatch extends ConfigurableDispatch {
protected static final HaDispatchMessages LOG = MessagesFactory.get(HaDispatchMessages.class);
- private int maxFailoverAttempts = HaServiceConfigConstants.DEFAULT_MAX_FAILOVER_ATTEMPTS;
+ protected int maxFailoverAttempts = HaServiceConfigConstants.DEFAULT_MAX_FAILOVER_ATTEMPTS;
- private int failoverSleep = HaServiceConfigConstants.DEFAULT_FAILOVER_SLEEP;
+ protected int failoverSleep = HaServiceConfigConstants.DEFAULT_FAILOVER_SLEEP;
- private HaProvider haProvider;
+ protected HaProvider haProvider;
private static final Map<String, String> urlToHashLookup = new HashMap<>();
private static final Map<String, String> hashToUrlLookup = new HashMap<>();
+ protected static final List<String> idempotentRequests = Arrays.asList("POST", "PATCH", "CONNECT");
Review Comment:
Aren't these the `non` idempotent requests?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] pzampino commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "pzampino (via GitHub)" <gi...@apache.org>.
pzampino commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146864881
##########
gateway-service-webhdfs/src/main/java/org/apache/knox/gateway/hdfs/dispatch/AbstractHdfsHaDispatch.java:
##########
@@ -59,33 +59,19 @@ protected void executeRequest(HttpUriRequest outboundRequest, HttpServletRequest
try {
inboundResponse = executeOutboundRequest(outboundRequest);
writeOutboundResponse(outboundRequest, inboundRequest, outboundResponse, inboundResponse);
- } catch (StandbyException e) {
+ } catch (StandbyException | SafeModeException | IOException e) {
/* if non-idempotent requests are not allowed to failover */
- if(!failoverNonIdempotentRequestEnabled && idempotentRequests.stream().anyMatch(outboundRequest.getMethod()::equalsIgnoreCase)) {
- LOG.cannotFailoverToNonIdempotentRequest(outboundRequest.getMethod());
+ if(!failoverNonIdempotentRequestEnabled && nonIdempotentRequests.stream().anyMatch(outboundRequest.getMethod()::equalsIgnoreCase)) {
+ LOG.cannotFailoverNonIdempotentRequest(outboundRequest.getMethod(), e.toString());
throw e;
} else {
- LOG.errorReceivedFromStandbyNode(e);
- failoverRequest(outboundRequest, inboundRequest, outboundResponse,
- inboundResponse, e);
- }
- } catch (SafeModeException e) {
- /* if non-idempotent requests are not allowed to failover */
- if(!failoverNonIdempotentRequestEnabled && idempotentRequests.stream().anyMatch(outboundRequest.getMethod()::equalsIgnoreCase)) {
- LOG.cannotFailoverToNonIdempotentRequest(outboundRequest.getMethod());
- throw e;
- } else {
- LOG.errorReceivedFromSafeModeNode(e);
- failoverRequest(outboundRequest, inboundRequest, outboundResponse,
- inboundResponse, e);
- }
- } catch (IOException e) {
- /* if non-idempotent requests are not allowed to failover */
- if(!failoverNonIdempotentRequestEnabled && idempotentRequests.stream().anyMatch(outboundRequest.getMethod()::equalsIgnoreCase)) {
- LOG.cannotFailoverToNonIdempotentRequest(outboundRequest.getMethod());
- throw e;
- } else {
- LOG.errorConnectingToServer(outboundRequest.getURI().toString(), e);
+ if(e instanceof StandbyException) {
Review Comment:
Did we lose the failoverRequest invocation here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] pzampino commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "pzampino (via GitHub)" <gi...@apache.org>.
pzampino commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146365404
##########
gateway-service-webhdfs/src/main/java/org/apache/knox/gateway/hdfs/dispatch/AbstractHdfsHaDispatch.java:
##########
@@ -78,14 +60,35 @@ protected void executeRequest(HttpUriRequest outboundRequest, HttpServletRequest
inboundResponse = executeOutboundRequest(outboundRequest);
writeOutboundResponse(outboundRequest, inboundRequest, outboundResponse, inboundResponse);
} catch (StandbyException e) {
- LOG.errorReceivedFromStandbyNode(e);
- failoverRequest(outboundRequest, inboundRequest, outboundResponse, inboundResponse, e);
+ /* if non-idempotent requests are not allowed to failover */
Review Comment:
This seems like it could be pulled out to a common method, or a multi-Exception catch block since the handling is all exactly the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] pzampino commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "pzampino (via GitHub)" <gi...@apache.org>.
pzampino commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146931120
##########
gateway-service-webhdfs/src/main/java/org/apache/knox/gateway/hdfs/dispatch/AbstractHdfsHaDispatch.java:
##########
@@ -59,33 +59,19 @@ protected void executeRequest(HttpUriRequest outboundRequest, HttpServletRequest
try {
inboundResponse = executeOutboundRequest(outboundRequest);
writeOutboundResponse(outboundRequest, inboundRequest, outboundResponse, inboundResponse);
- } catch (StandbyException e) {
+ } catch (StandbyException | SafeModeException | IOException e) {
/* if non-idempotent requests are not allowed to failover */
- if(!failoverNonIdempotentRequestEnabled && idempotentRequests.stream().anyMatch(outboundRequest.getMethod()::equalsIgnoreCase)) {
- LOG.cannotFailoverToNonIdempotentRequest(outboundRequest.getMethod());
+ if(!failoverNonIdempotentRequestEnabled && nonIdempotentRequests.stream().anyMatch(outboundRequest.getMethod()::equalsIgnoreCase)) {
+ LOG.cannotFailoverNonIdempotentRequest(outboundRequest.getMethod(), e.toString());
throw e;
} else {
- LOG.errorReceivedFromStandbyNode(e);
- failoverRequest(outboundRequest, inboundRequest, outboundResponse,
- inboundResponse, e);
- }
- } catch (SafeModeException e) {
- /* if non-idempotent requests are not allowed to failover */
- if(!failoverNonIdempotentRequestEnabled && idempotentRequests.stream().anyMatch(outboundRequest.getMethod()::equalsIgnoreCase)) {
- LOG.cannotFailoverToNonIdempotentRequest(outboundRequest.getMethod());
- throw e;
- } else {
- LOG.errorReceivedFromSafeModeNode(e);
- failoverRequest(outboundRequest, inboundRequest, outboundResponse,
- inboundResponse, e);
- }
- } catch (IOException e) {
- /* if non-idempotent requests are not allowed to failover */
- if(!failoverNonIdempotentRequestEnabled && idempotentRequests.stream().anyMatch(outboundRequest.getMethod()::equalsIgnoreCase)) {
- LOG.cannotFailoverToNonIdempotentRequest(outboundRequest.getMethod());
- throw e;
- } else {
- LOG.errorConnectingToServer(outboundRequest.getURI().toString(), e);
+ if(e instanceof StandbyException) {
Review Comment:
ok, must be the multiple commits confusing me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] pzampino commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "pzampino (via GitHub)" <gi...@apache.org>.
pzampino commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146404015
##########
gateway-service-webhdfs/src/main/java/org/apache/knox/gateway/hdfs/dispatch/AbstractHdfsHaDispatch.java:
##########
@@ -78,14 +60,35 @@ protected void executeRequest(HttpUriRequest outboundRequest, HttpServletRequest
inboundResponse = executeOutboundRequest(outboundRequest);
writeOutboundResponse(outboundRequest, inboundRequest, outboundResponse, inboundResponse);
} catch (StandbyException e) {
- LOG.errorReceivedFromStandbyNode(e);
- failoverRequest(outboundRequest, inboundRequest, outboundResponse, inboundResponse, e);
+ /* if non-idempotent requests are not allowed to failover */
Review Comment:
Could still be factored out to a common method though, I think. This is a minor point though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] moresandeep commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "moresandeep (via GitHub)" <gi...@apache.org>.
moresandeep commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146419057
##########
gateway-service-webhdfs/src/main/java/org/apache/knox/gateway/hdfs/dispatch/AbstractHdfsHaDispatch.java:
##########
@@ -78,14 +60,35 @@ protected void executeRequest(HttpUriRequest outboundRequest, HttpServletRequest
inboundResponse = executeOutboundRequest(outboundRequest);
writeOutboundResponse(outboundRequest, inboundRequest, outboundResponse, inboundResponse);
} catch (StandbyException e) {
- LOG.errorReceivedFromStandbyNode(e);
- failoverRequest(outboundRequest, inboundRequest, outboundResponse, inboundResponse, e);
+ /* if non-idempotent requests are not allowed to failover */
Review Comment:
Oh you mean, like make it a method that does the if ..else logic for logging?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] moresandeep commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "moresandeep (via GitHub)" <gi...@apache.org>.
moresandeep commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146404554
##########
gateway-service-webhdfs/src/main/java/org/apache/knox/gateway/hdfs/dispatch/AbstractHdfsHaDispatch.java:
##########
@@ -78,14 +60,35 @@ protected void executeRequest(HttpUriRequest outboundRequest, HttpServletRequest
inboundResponse = executeOutboundRequest(outboundRequest);
writeOutboundResponse(outboundRequest, inboundRequest, outboundResponse, inboundResponse);
} catch (StandbyException e) {
- LOG.errorReceivedFromStandbyNode(e);
- failoverRequest(outboundRequest, inboundRequest, outboundResponse, inboundResponse, e);
+ /* if non-idempotent requests are not allowed to failover */
Review Comment:
Yup, I can squash the code to take out duplicates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] pzampino commented on a diff in pull request #742: KNOX-2890 - Prevent non-idempotent requests from failing over
Posted by "pzampino (via GitHub)" <gi...@apache.org>.
pzampino commented on code in PR #742:
URL: https://github.com/apache/knox/pull/742#discussion_r1146403234
##########
gateway-service-webhdfs/src/main/java/org/apache/knox/gateway/hdfs/dispatch/AbstractHdfsHaDispatch.java:
##########
@@ -78,14 +60,35 @@ protected void executeRequest(HttpUriRequest outboundRequest, HttpServletRequest
inboundResponse = executeOutboundRequest(outboundRequest);
writeOutboundResponse(outboundRequest, inboundRequest, outboundResponse, inboundResponse);
} catch (StandbyException e) {
- LOG.errorReceivedFromStandbyNode(e);
- failoverRequest(outboundRequest, inboundRequest, outboundResponse, inboundResponse, e);
+ /* if non-idempotent requests are not allowed to failover */
Review Comment:
I do now notice that the only difference is the log message in the failover cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org