You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Tanvi Penumudy (Jira)" <ji...@apache.org> on 2024/04/18 05:23:00 UTC
[jira] [Assigned] (HDDS-10688) S3 multipart upload failed for Chinese filename with s3 fuse clients
[ https://issues.apache.org/jira/browse/HDDS-10688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tanvi Penumudy reassigned HDDS-10688:
-------------------------------------
Assignee: Tanvi Penumudy
> S3 multipart upload failed for Chinese filename with s3 fuse clients
> --------------------------------------------------------------------
>
> Key: HDDS-10688
> URL: https://issues.apache.org/jira/browse/HDDS-10688
> Project: Apache Ozone
> Issue Type: Bug
> Components: S3, s3gateway
> Affects Versions: 1.4.0, 1.5.0
> Reporter: Soumitra Sulav
> Assignee: Tanvi Penumudy
> Priority: Critical
>
> * Issue is seen only with s3 fuse clients which internally do the copy via multipart upload mechanism. Basically, the client initiates an MPU, creates individual parts uploads, and finally runs the COMPLETE_MULTIPART_UPLOAD using individual parts. The issue is observed at the last layer where it is trying to merge the file and failing to do so.
> * This issue is only seen with Chinese (Non-English charset) characters.
> * Upload-part API misses the non-English character in the response even after setting proper encoding and locale variables.
> * List-Part API response is proper.
> Below are the repro steps :
> # Install goofys fuse client
> [https://github.com/kahing/goofys/releases/download/v0.24.0/goofys]
> # Mount the ozone s3 endpoint via goofys
> {code:java}
> goofys --debug_fuse --debug_s3 --endpoint http://<OzoneS3GHost>:<OzoneS3GPort> <BucketName> <LocalPath>
> {code}
> # Create a file of size > 5MB and a name containing Chinese characters.
> # Copy the file from the local filesystem to the mounted path.
> {code:java}
> cp 测试三.txt /mnt/test-goofys/
> cp: failed to close '/mnt/test-goofys/测试三.txt': Invalid argument
> {code}
> Error stacktrace
> {code:java}
> 2024-04-08 09:29:36,145 | INFO | S3GAudit | user=om@ROOT.COMOPS.SITE | ip=10.129.77.95 | op=INIT_MULTIPART_UPLOAD {bucket=[buckettest], path=[测试三.txt], uploads=[]} | ret=SUCCESS |
> 2024-04-08 09:29:36,288 | INFO | S3GAudit | user=om@ROOT.COMOPS.SITE | ip=10.129.77.95 | op=CREATE_MULTIPART_KEY {bucket=[buckettest], path=[测试三.txt], uploadId=[669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441], partNumber=[1]} | ret=SUCCESS |
> 2024-04-08 09:29:36,432 | INFO | S3GAudit | user=om@ROOT.COMOPS.SITE | ip=10.129.77.95 | op=CREATE_MULTIPART_KEY {bucket=[buckettest], path=[测试三.txt], uploadId=[669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441], partNumber=[2]} | ret=SUCCESS |
> 2024-04-08 09:29:36,455 | ERROR | S3GAudit | user=om@ROOT.COMOPS.SITE | ip=10.129.77.95 | op=COMPLETE_MULTIPART_UPLOAD {bucket=[buckettest], path=[测试三.txt], uploadId=[669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441]} | ret=FAILURE | INVALID_PART org.apache.hadoop.ozone.om.exceptions.OMException: Complete Multipart Upload Failed: volume: s3v bucket: buckettest key: 测试三.txt. Provided Part info is { /s3v/buckettest/ .txt-669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441-1, 1}, whereas OM has partName /s3v/buckettest/测试三.txt-669be17b-6c05-4066-9398-13a3586c65b1-112234894206109441-1
> at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:728)
> at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.completeMultipartUpload(OzoneManagerProtocolClientSideTranslatorPB.java:1587)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org