You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2019/03/05 22:47:19 UTC
[GitHub] [hadoop] bharatviswa504 commented on issue #549: HDDS-1213. Support
plain text S3 MPU initialization request
bharatviswa504 commented on issue #549: HDDS-1213. Support plain text S3 MPU initialization request
URL: https://github.com/apache/hadoop/pull/549#issuecomment-469889963
Over all patch LGTM.
Thank You @elek for offline discussion.
Yes, I see that when file size is large, cp is using multipart upload request.
Few minor comments:
1. Remove unncessary space indentation changes
2. Removed platform.system(), as we don't need it now.
I see below error when running test manually with docker ozones3 cluster
```
s3g_1 | 2019-03-05 22:37:40 WARN HttpChannel:499 - //localhost:9878/b12345/mm11?partNumber=3&uploadId=d51657f3-3276-49d0-977a-af9f4bab94bc-19533321172195
s3g_1 | javax.servlet.ServletException: javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError: Java heap space
s3g_1 | at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:139)
s3g_1 | at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
s3g_1 | at org.eclipse.jetty.server.Server.handle(Server.java:539)
s3g_1 | at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
s3g_1 | at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
s3g_1 | at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
s3g_1 | at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
s3g_1 | at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
s3g_1 | at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
s3g_1 | at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
s3g_1 | at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
s3g_1 | at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
s3g_1 | at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
s3g_1 | at java.lang.Thread.run(Thread.java:748)
s3g_1 | Caused by: javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError: Java heap space
s3g_1 | at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:432)
s3g_1 | at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
s3g_1 | at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
s3g_1 | at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
s3g_1 | at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
s3g_1 | at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:840)
s3g_1 | at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
s3g_1 | at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1609)
s3g_1 | at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
s3g_1 | at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
s3g_1 | at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
s3g_1 | at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
s3g_1 | at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
s3g_1 | at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
s3g_1 | at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
s3g_1 | at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
s3g_1 | at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
s3g_1 | at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
s3g_1 | at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
s3g_1 | at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
s3g_1 | at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
s3g_1 | ... 13 more
s3g_1 | Caused by: org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError: Java heap space
```
I see when allocating a buffer, I am getting this error. I see by default we allocate 2 64mb buffer for all buffer lists. Any idea how to resolve this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org