You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "everpcpc (via GitHub)" <gi...@apache.org> on 2024/06/24 03:07:17 UTC
[I] Multipart upload error on GCS [opendal]
everpcpc opened a new issue, #4794:
URL: https://github.com/apache/opendal/issues/4794
> Save gcs cache failed: Error: Unexpected (permanent) at Writer::write, context: { uri: https://storage.googleapis.com/upload/storage/v1/b/xxx/o?uploadType=resumable&name=cloud/cache/Linux/go/xxx/xxx/cache.tzst&upload_id=ACJd0No-AuMDWKM0kCVnjavxxx686_MltwDqow-J_vbVrZ0mfhyxxxokY0Oql7cjS4yWlqs4CD-_HjTPmasgjbHcZ, response: Parts { status: 400, version: HTTP/1.1, headers: {"content-type": "text/plain; charset=utf-8", "content-length": "261", "date": "Mon, 24 Jun 2024 02:55:25 GMT", "server": "UploadServer"} }, service: gcs, path: Linux/go/xxx/xxx/cache.tzst, write_buf: 65536 } => Invalid request. The number of bytes uploaded is required to be equal or greater than 262144, except for the final request (it's recommended to be the exact multiple of 262144). The received request contained 65536 bytes, which does not meet this requirement.
code:
```typescript
const op = new Operator(provider, { endpoint, bucket, root });
const rs = fs.createReadStream(archivePath);
const w = await op.writer(object);
const ws = w.createWriteStream();
await pipeline(rs, ws);
await finished(rs);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multipart upload error on GCS [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4794:
URL: https://github.com/apache/opendal/issues/4794#issuecomment-2186293811
> Yes, it did fail. Let's address it.
Should be fixed in https://github.com/apache/opendal/pull/4798
It's related to our CI env changes...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multipart upload error on GCS [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4794:
URL: https://github.com/apache/opendal/issues/4794#issuecomment-2205106294
Sorry, nodejs release failed: https://github.com/apache/opendal/actions/runs/9771241606/job/26973848658
I will get it fixed and start a new release now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multipart upload error on GCS [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4794:
URL: https://github.com/apache/opendal/issues/4794#issuecomment-2185920472
> https://www.npmjs.com/search?q=%40opendal
>
> It seems like [@opendal/lib-darwin-x64](https://www.npmjs.com/package/@opendal/lib-darwin-x64) release failed
Yes, it did fail. Let's address it.
According to this issue, services like gcs require to align with specific size to upload successfully. Users either need to set `chunk` or meet the requirements by hand.
AFAIK, nodejs doesn't support setting chunk in 0.45.1, so we have to add a buffer for this writer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multipart upload error on GCS [opendal]
Posted by "everpcpc (via GitHub)" <gi...@apache.org>.
everpcpc commented on issue #4794:
URL: https://github.com/apache/opendal/issues/4794#issuecomment-2185893423
https://www.npmjs.com/search?q=%40opendal
It seems like [@opendal/lib-darwin-x64](https://www.npmjs.com/package/@opendal/lib-darwin-x64) release failed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multipart upload error on GCS [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4794:
URL: https://github.com/apache/opendal/issues/4794#issuecomment-2212316008
Hi, @everpcpc, this issue should have been fixed. Please give it a try. Welcome to report new issues if it still exists.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multipart upload error on GCS [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4794:
URL: https://github.com/apache/opendal/issues/4794#issuecomment-2185859100
Hi, which version are you using? The latest version will set correct chunk for every storage service.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multipart upload error on GCS [opendal]
Posted by "everpcpc (via GitHub)" <gi...@apache.org>.
everpcpc commented on issue #4794:
URL: https://github.com/apache/opendal/issues/4794#issuecomment-2185865050
https://github.com/everpcpc/actions-cache/blob/main/package.json#L12
`0.45.1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multipart upload error on GCS [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo closed issue #4794: Multipart upload error on GCS
URL: https://github.com/apache/opendal/issues/4794
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org