You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@servicecomb.apache.org by GitBox <gi...@apache.org> on 2020/11/23 09:24:53 UTC
[GitHub] [servicecomb-java-chassis] yuzhouzhouba opened a new issue #2072: 您好，在传递大文件的时候，遇到的网络延时问题
yuzhouzhouba opened a new issue #2072:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2072
我提供了一个下载文件的REST API，它的工作流程是这样的：
获取文件流，将文件流以File类型返回，现在遇到的问题就是文件很大时候，会有几秒的延时
这里调用的是spring.core的ResponseEntity，代码如下
```
@GetMapping("/downloadTest/{fileId}")
@ApiResponses({@ApiResponse(code = 200, response = File.class, message = "")})
public ResponseEntity<Resource> downloadV2(@PathVariable String fileId) throws Exception {
try {
DownloadFileVO ret = fileServiceDelegate.downloadFile(fileId);
Resource resource = new InputStreamResource(ret.getObjectContent());
String contentType = "application/octet-stream";
// 以下这段代码，会有一个5~6秒延时
return ResponseEntity.ok()
.contentType(org.springframework.http.MediaType.parseMediaType(contentType))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + ret.getObjectKey() + "\"")
.body(resource);
} catch (Exception e) {
LOGGER.error("downloading file with Exception: " + e.getMessage(), e);
responseVo = new ResponseVo<>(ResponseCode.SERVER_INTERNAL_ERROR, e.getMessage(),null);
return null;
}
}
```
把延时的部分一步一步打开，发现service comb有对于这个流进行操作读写的操作
```
package org.apache.servicecomb.foundation.vertx.stream;
private synchronized void afterReadInEventloop(AsyncResult<InputStreamToReadStream.ReadResult> ar) {
if (ar.failed()) {
this.handleException(ar.cause());
} else {
this.readInProgress = false;
InputStreamToReadStream.ReadResult readResult = (InputStreamToReadStream.ReadResult)ar.result();
if (readResult.readed < 0) {
this.handleEnd();
} else {
this.handleData(readResult.toBuffer());
if (!this.paused && this.dataHandler != null) {
this.doRead();
}
}
}
}
```
想请教下，我的诉求就是消除这个延时。目前我分析，是以上这个read函数把这个大文件作为流全部读了一遍再返回给浏览器。
我希望是浏览器直接请求这个文件流，进行下载
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] liubao68 commented on issue #2072: 您好，在传递大文件的时候，遇到的网络延时问题
Posted by GitBox <gi...@apache.org>.
liubao68 commented on issue #2072:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2072#issuecomment-733407156
看前面的分析不像是
```
return ResponseEntity.ok()
.contentType(org.springframework.http.MediaType.parseMediaType(contentType))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + ret.getObjectKey() + "\"")
.body(resource);
```
这行代码会阻塞这么久时间。
你能够提供一个简单的重现问题 DEMO 示例吗？ 我可以测试验证下。
相关问题 [1](https://github.com/apache/servicecomb-java-chassis/issues/2071)
如何准备 DEMO 示例 [2](https://github.com/apache/servicecomb-java-chassis/wiki)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] yuzhouzhouba edited a comment on issue #2072: 您好，在传递大文件的时候，遇到的网络延时问题
Posted by GitBox <gi...@apache.org>.
yuzhouzhouba edited a comment on issue #2072:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2072#issuecomment-734685132
我理解应该是每次读一点（1M），然后就返回，所以读文件流读第一个1M就应该返回，那么就应该很快
demo如何准备那种带文件的? 其实我的代码就是这样（Spring boot 集成的 service Comb）
Resource resource = new InputStreamResource(ret.getObjectContent()); // 这个就是一个比较大的文件流
return ResponseEntity.ok()
.contentType(org.springframework.http.MediaType.parseMediaType(contentType))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + ret.getObjectKey() + "\"")
.body(resource);
当调用这个的时候，我期待是立马返回，但是实际了隔了比较久的时间才返回。（用浏览器或者postman都有这种问题）
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] yuzhouzhouba edited a comment on issue #2072: 您好，在传递大文件的时候，遇到的网络延时问题
Posted by GitBox <gi...@apache.org>.
yuzhouzhouba edited a comment on issue #2072:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2072#issuecomment-734685132
我理解应该是每次读一点（1M），然后就返回，所以第一个就读了1M就应该返回，那么就应该很快
demo如何准备那种带文件的? 其实我的代码就是这样
Resource resource = new InputStreamResource(ret.getObjectContent()); // 这个就是一个比较大的文件流
return ResponseEntity.ok()
.contentType(org.springframework.http.MediaType.parseMediaType(contentType))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + ret.getObjectKey() + "\"")
.body(resource);
当调用这个的时候，我期待是立马返回，但是实际了隔了比较久的时间才返回。（用浏览器或者postman都有这种问题）
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] yuzhouzhouba commented on issue #2072: 您好，在传递大文件的时候，遇到的网络延时问题
Posted by GitBox <gi...@apache.org>.
yuzhouzhouba commented on issue #2072:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2072#issuecomment-734685132
我理解应该是每次读一点（1M），然后就返回，所以第一个就读了1M就应该返回，那么就应该很快
================================
demo如何准备那种带文件的? 其实我的代码就是这样
Resource resource = new InputStreamResource(ret.getObjectContent()); // 这个就是一个比较大的文件流
return ResponseEntity.ok()
.contentType(org.springframework.http.MediaType.parseMediaType(contentType))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + ret.getObjectKey() + "\"")
.body(resource);
当调用这个的时候，我期待是立马返回，但是实际了隔了比较久的时间才返回。（用浏览器或者postman都有这种问题）
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] liubao68 commented on issue #2072: 您好，在传递大文件的时候，遇到的网络延时问题
Posted by GitBox <gi...@apache.org>.
liubao68 commented on issue #2072:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2072#issuecomment-732533636
感觉你是对网络流和文件流有所误解。网络发送过程存在一个写入的过程，这个需要将文件读取出来，写入到网络流里面。阻塞的过程就是这个过程。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] wujimin commented on issue #2072: 您好，在传递大文件的时候，遇到的网络延时问题
Posted by GitBox <gi...@apache.org>.
wujimin commented on issue #2072:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2072#issuecomment-732535759
这是一个异步的过程
在vertx worker线程，每次从input stream***最多***读取1MB
然后再在网络线程发送数据
我测试过1GB的本地文件下载，没有一个操作卡住的现象
但是看你的场景，是中转一个网络流的样子，可能需要更多的验证
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] yuzhouzhouba edited a comment on issue #2072: 您好，在传递大文件的时候，遇到的网络延时问题
Posted by GitBox <gi...@apache.org>.
yuzhouzhouba edited a comment on issue #2072:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2072#issuecomment-734685132
我理解应该是每次读一点（1M），然后就返回，所以读文件流读第一个1M就应该返回，那么就应该很快
demo如何准备那种带文件的? 其实我的代码就是这样
Resource resource = new InputStreamResource(ret.getObjectContent()); // 这个就是一个比较大的文件流
return ResponseEntity.ok()
.contentType(org.springframework.http.MediaType.parseMediaType(contentType))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + ret.getObjectKey() + "\"")
.body(resource);
当调用这个的时候，我期待是立马返回，但是实际了隔了比较久的时间才返回。（用浏览器或者postman都有这种问题）
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] yuzhouzhouba commented on issue #2072: 您好，在传递大文件的时候，遇到的网络延时问题
Posted by GitBox <gi...@apache.org>.
yuzhouzhouba commented on issue #2072:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2072#issuecomment-738538763
按照开发的文档的示例代码进行编写
```
@GetMapping(path = "/resource")
@ApiResponses({
@ApiResponse(code = 200, response = File.class, message = "")
})
public ResponseEntity<Resource> resource() {
return ResponseEntity
.ok()
.header(HttpHeaders.CONTENT_TYPE, MediaType.TEXT_PLAIN_VALUE)
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment;filename=resource.txt")
.body(resource);
}
```
1、在点击的时候有几秒的延迟
2、在不同的环境下有几率性残缺，
3、浏览器不能显示进度
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [servicecomb-java-chassis] yuzhouzhouba commented on issue #2072: 您好，在传递大文件的时候，遇到的网络延时问题
Posted by GitBox <gi...@apache.org>.
yuzhouzhouba commented on issue #2072:
URL: https://github.com/apache/servicecomb-java-chassis/issues/2072#issuecomment-739731260
感谢@liubao68 支撑
经过测试，确实对于流的读取是一点一点的读的，目前认为这种方式`不适合传递大文件`，有如下三点问题：
1、在点击的时候有几秒的延迟 ----> 这个跟网络环境有关，可能第一个小包过来就产生了较大的延时。
2、在不同的环境下有几率性残缺 ---> 这个在需要通过网络传输复现，本地文件为复现。
3、浏览器不能显示进度 ---> 本地复现，确实浏览器不能显示进度，只能看到在传输。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org