[apache/incubator-teaclave] 远程下载大文件（50） (#387)

[ly137062117 <no...@github.com>]

在execution service 执行任务时，下载的远程文件，通过docker查看，发现sha256sum (tmp文件地址)和sha256sum(我们注册的文件地址)，其结果不一样，runtime.open_input()出现报错Err(Unknown error: 28676 (os error: 28676))。是否是文件下载过程中出现了损坏？

docker中文件进行sha256：
sha256sum Bob_1000.enc
eb11d658c25c2cf0bdf08135ac5af769b76e6adaed60152c1af57885ab66b11f

外部文件进行sha256：
sha256sum Bob_1000.enc
8cc9b09a3d2a0e62a096c592600114f1b1ba667f4d75085df1d5420dbb04bfc8

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-teaclave/issues/387

Re: [apache/incubator-teaclave] 注册文件执行任务时，远程下载大文件（50k），出现文件损坏 (#387)

[Mingshen Sun <no...@github.com>]

好的，具体是什么引起的？

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-teaclave/issues/387#issuecomment-656495478

Re: [apache/incubator-teaclave] 注册文件执行任务时，远程下载大文件（50k），出现文件损坏 (#387)

[ly137062117 <no...@github.com>]

我们通过日志发现，在这里 tokio 库在写入文件的时候，会出现文件不完整的情况（只写入部分），我们独立运行这个库，也是得到相同的结果：
https://github.com/apache/incubator-teaclave/blob/d3a6abd158e2113df89a7de13cad91080a81d027/file_agent/src/agent.rs#L38

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-teaclave/issues/387#issuecomment-656497615

Re: [apache/incubator-teaclave] 注册文件执行任务时，远程下载大文件（50k），出现文件损坏 (#387)

[Bing Duan <no...@github.com>]

write == create（truncate if exists） + write_all.  这里是不是只调用write_all?  

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-teaclave/issues/387#issuecomment-656511332

Re: [apache/incubator-teaclave] 注册文件执行任务时，远程下载大文件（50k），出现文件损坏 (#387)

[Mingshen Sun <no...@github.com>]

Fixed by cc74ed0. Thanks. Closed.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-teaclave/issues/387#issuecomment-656803676

Re: [apache/incubator-teaclave] 注册文件执行任务时，远程下载大文件（50k），出现文件损坏 (#387)

[ly137062117 <no...@github.com>]

每个循环中下载到chunk中的数据是完整的，但是每次 write(chunk) 写入文件时，最多只能写入16384字节长度的数据，导致了多余的数据丢失

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-teaclave/issues/387#issuecomment-656500382

Re: [apache/incubator-teaclave] 注册文件执行任务时，远程下载大文件（50k），出现文件损坏 (#387)

[Mingshen Sun <no...@github.com>]

Closed #387.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-teaclave/issues/387#event-3534854626

Re: [apache/incubator-teaclave] 注册文件执行任务时，远程下载大文件（50k），出现文件损坏 (#387)

[ly137062117 <no...@github.com>]

问题已定位，是用到的reqwest库，在下载远程文件时，会有损失，尝试下载一份500k的数据文件，下载到本地结果只有140k左右

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-teaclave/issues/387#issuecomment-656486800

Re: [apache/incubator-teaclave] 注册文件执行任务时，远程下载大文件（50k），出现文件损坏 (#387)

[Bing Duan <no...@github.com>]

下载过程中有什么异常日志吗？ 类似 Dest local file:  xxx already exists.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-teaclave/issues/387#issuecomment-656467535

Re: [apache/incubator-teaclave] 注册文件执行任务时，远程下载大文件（50k），出现文件损坏 (#387)

[Mingshen Sun <no...@github.com>]

文件大小是多少？请把复现的步骤写清楚，多谢了。

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-teaclave/issues/387#issuecomment-656484495