You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by "hangc0276 (via GitHub)" <gi...@apache.org> on 2023/06/22 05:41:25 UTC
[GitHub] [bookkeeper] hangc0276 opened a new pull request, #3998: Fix trigger GC not work
hangc0276 opened a new pull request, #3998:
URL: https://github.com/apache/bookkeeper/pull/3998
### Motivation
When I use `curl -XPUT http://<ip>:<port>/api/v1/bookie/gc` command to trigger garbage compaction, the bookie server returns `Internal Server Error`. Then I checked the bookie server's log, and nothing was found.
After a deep debugging of the server, I found the following exception.
```
org.apache.bookkeeper.common.util.JsonUtil$ParseJsonException: Failed to deserialize Object from Json string
at org.apache.bookkeeper.common.util.JsonUtil.fromJson(JsonUtil.java:41)
at org.apache.bookkeeper.server.http.service.TriggerGCService.handle(TriggerGCService.java:71)
at org.apache.bookkeeper.http.vertx.VertxAbstractHandler.processRequest(VertxAbstractHandler.java:55)
at org.apache.bookkeeper.http.vertx.VertxHttpHandlerFactory$1.handle(VertxHttpHandlerFactory.java:47)
at org.apache.bookkeeper.http.vertx.VertxHttpHandlerFactory$1.handle(VertxHttpHandlerFactory.java:43)
at io.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1038)
at io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:137)
at io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:132)
at io.vertx.ext.web.handler.impl.BodyHandlerImpl$BHandler.doEnd(BodyHandlerImpl.java:296)
at io.vertx.ext.web.handler.impl.BodyHandlerImpl$BHandler.end(BodyHandlerImpl.java:276)
at io.vertx.ext.web.handler.impl.BodyHandlerImpl.lambda$handle$0(BodyHandlerImpl.java:87)
at io.vertx.core.http.impl.HttpServerRequestImpl.onEnd(HttpServerRequestImpl.java:525)
at io.vertx.core.http.impl.HttpServerRequestImpl.handleEnd(HttpServerRequestImpl.java:511)
at io.vertx.core.http.impl.Http1xServerConnection.handleEnd(Http1xServerConnection.java:176)
at io.vertx.core.http.impl.Http1xServerConnection.handleMessage(Http1xServerConnection.java:138)
at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:366)
at io.vertx.core.impl.EventLoopContext.execute(EventLoopContext.java:43)
```
This bug is introduced by https://github.com/apache/bookkeeper/pull/3205. When we use `curl -XPUT http://<ip>:<port>/api/v1/bookie/gc` command to trigger gc, the request body is empty string, but not `null`.
https://github.com/apache/bookkeeper/blob/ffc8e8bec19e54dc76710c3de133401248f243d6/bookkeeper-server/src/main/java/org/apache/bookkeeper/server/http/service/TriggerGCService.java#L67-L75
It will enter Line#68 but goes into Line#71. It uses `JsonUtil.fromJson` to parse an empty string will throw the above exception and will return `500` to the client but no logs in the server side.
### Changes
- Fix the request body check bug
- Add a unit test to cover this logic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] hangc0276 commented on a diff in pull request #3998: Fix trigger GC not work
Posted by "hangc0276 (via GitHub)" <gi...@apache.org>.
hangc0276 commented on code in PR #3998:
URL: https://github.com/apache/bookkeeper/pull/3998#discussion_r1239259348
##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/server/http/service/TriggerGCService.java:
##########
@@ -61,40 +62,46 @@ public TriggerGCService(ServerConfiguration conf, BookieServer bookieServer) {
@Override
public HttpServiceResponse handle(HttpServiceRequest request) throws Exception {
HttpServiceResponse response = new HttpServiceResponse();
+ try {
+ if (HttpServer.Method.PUT == request.getMethod()) {
+ String requestBody = request.getBody();
+ if (StringUtils.isBlank(requestBody)) {
+ bookieServer.getBookie().getLedgerStorage().forceGC();
+ } else {
+ @SuppressWarnings("unchecked")
+ Map<String, Object> configMap = JsonUtil.fromJson(requestBody, HashMap.class);
+ Boolean forceMajor = (Boolean) configMap.getOrDefault("forceMajor", null);
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] hangc0276 commented on a diff in pull request #3998: Fix trigger GC not work
Posted by "hangc0276 (via GitHub)" <gi...@apache.org>.
hangc0276 commented on code in PR #3998:
URL: https://github.com/apache/bookkeeper/pull/3998#discussion_r1239259610
##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/server/http/service/TriggerGCService.java:
##########
@@ -61,40 +62,46 @@ public TriggerGCService(ServerConfiguration conf, BookieServer bookieServer) {
@Override
public HttpServiceResponse handle(HttpServiceRequest request) throws Exception {
HttpServiceResponse response = new HttpServiceResponse();
+ try {
+ if (HttpServer.Method.PUT == request.getMethod()) {
+ String requestBody = request.getBody();
+ if (StringUtils.isBlank(requestBody)) {
+ bookieServer.getBookie().getLedgerStorage().forceGC();
+ } else {
+ @SuppressWarnings("unchecked")
+ Map<String, Object> configMap = JsonUtil.fromJson(requestBody, HashMap.class);
+ Boolean forceMajor = (Boolean) configMap.getOrDefault("forceMajor", null);
+ Boolean forceMinor = (Boolean) configMap.getOrDefault("forceMinor", null);
+ bookieServer.getBookie().getLedgerStorage().forceGC(forceMajor, forceMinor);
+ }
- if (HttpServer.Method.PUT == request.getMethod()) {
- String requestBody = request.getBody();
- if (null == requestBody) {
- bookieServer.getBookie().getLedgerStorage().forceGC();
+ String output = "Triggered GC on BookieServer: " + bookieServer.getBookieId();
+ String jsonResponse = JsonUtil.toJson(output);
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("output body:" + jsonResponse);
+ }
+ response.setBody(jsonResponse);
+ response.setCode(HttpServer.StatusCode.OK);
+ return response;
+ } else if (HttpServer.Method.GET == request.getMethod()) {
+ Boolean isInForceGC = bookieServer.getBookie().getLedgerStorage().isInForceGC();
+ Pair<String, String> output = Pair.of("is_in_force_gc", isInForceGC.toString());
+ String jsonResponse = JsonUtil.toJson(output);
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("output body:" + jsonResponse);
+ }
+ response.setBody(jsonResponse);
+ response.setCode(HttpServer.StatusCode.OK);
+ return response;
} else {
- @SuppressWarnings("unchecked")
- Map<String, Object> configMap = JsonUtil.fromJson(requestBody, HashMap.class);
- Boolean forceMajor = (Boolean) configMap.getOrDefault("forceMajor", null);
- Boolean forceMinor = (Boolean) configMap.getOrDefault("forceMinor", null);
- bookieServer.getBookie().getLedgerStorage().forceGC(forceMajor, forceMinor);
+ response.setCode(HttpServer.StatusCode.NOT_FOUND);
Review Comment:
Updated. It will change the current status code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] zymap merged pull request #3998: Fix trigger GC not work
Posted by "zymap (via GitHub)" <gi...@apache.org>.
zymap merged PR #3998:
URL: https://github.com/apache/bookkeeper/pull/3998
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] eolivelli commented on a diff in pull request #3998: Fix trigger GC not work
Posted by "eolivelli (via GitHub)" <gi...@apache.org>.
eolivelli commented on code in PR #3998:
URL: https://github.com/apache/bookkeeper/pull/3998#discussion_r1238328962
##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/server/http/service/TriggerGCService.java:
##########
@@ -61,40 +62,46 @@ public TriggerGCService(ServerConfiguration conf, BookieServer bookieServer) {
@Override
public HttpServiceResponse handle(HttpServiceRequest request) throws Exception {
HttpServiceResponse response = new HttpServiceResponse();
+ try {
+ if (HttpServer.Method.PUT == request.getMethod()) {
+ String requestBody = request.getBody();
+ if (StringUtils.isBlank(requestBody)) {
+ bookieServer.getBookie().getLedgerStorage().forceGC();
+ } else {
+ @SuppressWarnings("unchecked")
+ Map<String, Object> configMap = JsonUtil.fromJson(requestBody, HashMap.class);
+ Boolean forceMajor = (Boolean) configMap.getOrDefault("forceMajor", null);
+ Boolean forceMinor = (Boolean) configMap.getOrDefault("forceMinor", null);
+ bookieServer.getBookie().getLedgerStorage().forceGC(forceMajor, forceMinor);
+ }
- if (HttpServer.Method.PUT == request.getMethod()) {
- String requestBody = request.getBody();
- if (null == requestBody) {
- bookieServer.getBookie().getLedgerStorage().forceGC();
+ String output = "Triggered GC on BookieServer: " + bookieServer.getBookieId();
+ String jsonResponse = JsonUtil.toJson(output);
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("output body:" + jsonResponse);
+ }
+ response.setBody(jsonResponse);
+ response.setCode(HttpServer.StatusCode.OK);
+ return response;
+ } else if (HttpServer.Method.GET == request.getMethod()) {
+ Boolean isInForceGC = bookieServer.getBookie().getLedgerStorage().isInForceGC();
+ Pair<String, String> output = Pair.of("is_in_force_gc", isInForceGC.toString());
+ String jsonResponse = JsonUtil.toJson(output);
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("output body:" + jsonResponse);
+ }
+ response.setBody(jsonResponse);
+ response.setCode(HttpServer.StatusCode.OK);
+ return response;
} else {
- @SuppressWarnings("unchecked")
- Map<String, Object> configMap = JsonUtil.fromJson(requestBody, HashMap.class);
- Boolean forceMajor = (Boolean) configMap.getOrDefault("forceMajor", null);
- Boolean forceMinor = (Boolean) configMap.getOrDefault("forceMinor", null);
- bookieServer.getBookie().getLedgerStorage().forceGC(forceMajor, forceMinor);
+ response.setCode(HttpServer.StatusCode.NOT_FOUND);
Review Comment:
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] zymap commented on a diff in pull request #3998: Fix trigger GC not work
Posted by "zymap (via GitHub)" <gi...@apache.org>.
zymap commented on code in PR #3998:
URL: https://github.com/apache/bookkeeper/pull/3998#discussion_r1238260404
##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/server/http/service/TriggerGCService.java:
##########
@@ -61,40 +62,46 @@ public TriggerGCService(ServerConfiguration conf, BookieServer bookieServer) {
@Override
public HttpServiceResponse handle(HttpServiceRequest request) throws Exception {
HttpServiceResponse response = new HttpServiceResponse();
+ try {
+ if (HttpServer.Method.PUT == request.getMethod()) {
+ String requestBody = request.getBody();
+ if (StringUtils.isBlank(requestBody)) {
+ bookieServer.getBookie().getLedgerStorage().forceGC();
+ } else {
+ @SuppressWarnings("unchecked")
+ Map<String, Object> configMap = JsonUtil.fromJson(requestBody, HashMap.class);
+ Boolean forceMajor = (Boolean) configMap.getOrDefault("forceMajor", null);
Review Comment:
Shouldn't the default value be a boolean? If it is null, it would be easy to cause NPE in the inner code. We can use Boolean.parseBoolean() to make sure it is a boolean and avoid a cast error happening.
##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/server/http/service/TriggerGCService.java:
##########
@@ -61,40 +62,46 @@ public TriggerGCService(ServerConfiguration conf, BookieServer bookieServer) {
@Override
public HttpServiceResponse handle(HttpServiceRequest request) throws Exception {
HttpServiceResponse response = new HttpServiceResponse();
+ try {
+ if (HttpServer.Method.PUT == request.getMethod()) {
+ String requestBody = request.getBody();
+ if (StringUtils.isBlank(requestBody)) {
+ bookieServer.getBookie().getLedgerStorage().forceGC();
+ } else {
+ @SuppressWarnings("unchecked")
+ Map<String, Object> configMap = JsonUtil.fromJson(requestBody, HashMap.class);
+ Boolean forceMajor = (Boolean) configMap.getOrDefault("forceMajor", null);
+ Boolean forceMinor = (Boolean) configMap.getOrDefault("forceMinor", null);
+ bookieServer.getBookie().getLedgerStorage().forceGC(forceMajor, forceMinor);
+ }
- if (HttpServer.Method.PUT == request.getMethod()) {
- String requestBody = request.getBody();
- if (null == requestBody) {
- bookieServer.getBookie().getLedgerStorage().forceGC();
+ String output = "Triggered GC on BookieServer: " + bookieServer.getBookieId();
+ String jsonResponse = JsonUtil.toJson(output);
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("output body:" + jsonResponse);
+ }
+ response.setBody(jsonResponse);
+ response.setCode(HttpServer.StatusCode.OK);
+ return response;
+ } else if (HttpServer.Method.GET == request.getMethod()) {
+ Boolean isInForceGC = bookieServer.getBookie().getLedgerStorage().isInForceGC();
+ Pair<String, String> output = Pair.of("is_in_force_gc", isInForceGC.toString());
+ String jsonResponse = JsonUtil.toJson(output);
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("output body:" + jsonResponse);
+ }
+ response.setBody(jsonResponse);
+ response.setCode(HttpServer.StatusCode.OK);
+ return response;
} else {
- @SuppressWarnings("unchecked")
- Map<String, Object> configMap = JsonUtil.fromJson(requestBody, HashMap.class);
- Boolean forceMajor = (Boolean) configMap.getOrDefault("forceMajor", null);
- Boolean forceMinor = (Boolean) configMap.getOrDefault("forceMinor", null);
- bookieServer.getBookie().getLedgerStorage().forceGC(forceMajor, forceMinor);
+ response.setCode(HttpServer.StatusCode.NOT_FOUND);
Review Comment:
The status should be NOT_ALLOWED, see here https://httpwg.org/specs/rfc9110.html#status.405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org