You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Marton Elek (Jira)" <ji...@apache.org> on 2020/06/10 15:27:00 UTC
[jira] [Commented] (HDDS-3512) s3g multi-upload saved content
incorrect when client uses aws java sdk 1.11.* jar
[ https://issues.apache.org/jira/browse/HDDS-3512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17130792#comment-17130792 ]
Marton Elek commented on HDDS-3512:
-----------------------------------
Is it still a problem?
I tried to reproduce it with freon:
{code:java}
ozone freon s3kg -e http://s3g:9878 -n 10 -s 5242880 {code}
But chunk files created with the same size:
{code:java}
-rw-r--r-- 1 hadoop users 5242880 Jun 10 15:23 ./hdds/hdds/15292b6c-34d6-48d0-bb97-f58043767ade/current/containerDir0/1/chunks/104320375385031656.block
-rw-r--r-- 1 hadoop users 5242880 Jun 10 15:23 ./hdds/hdds/15292b6c-34d6-48d0-bb97-f58043767ade/current/containerDir0/1/chunks/104320375385031657.block
-rw-r--r-- 1 hadoop users 5242880 Jun 10 15:23 ./hdds/hdds/15292b6c-34d6-48d0-bb97-f58043767ade/current/containerDir0/1/chunks/104320375385031658.block
-rw-r--r-- 1 hadoop users 5242880 Jun 10 15:23 ./hdds/hdds/15292b6c-34d6-48d0-bb97-f58043767ade/current/containerDir0/1/chunks/104320375385162731.block
-rw-r--r-- 1 hadoop users 5242880 Jun 10 15:23 ./hdds/hdds/15292b6c-34d6-48d0-bb97-f58043767ade/current/containerDir0/1/chunks/104320375385424876.block
-rw-r--r-- 1 hadoop users 5242880 Jun 10 15:23 ./hdds/hdds/15292b6c-34d6-48d0-bb97-f58043767ade/current/containerDir0/1/chunks/104320375385424877.block
-rw-r--r-- 1 hadoop users 5242880 Jun 10 15:23 ./hdds/hdds/15292b6c-34d6-48d0-bb97-f58043767ade/current/containerDir0/1/chunks/104320375385424878.block
-rw-r--r-- 1 hadoop users 5242880 Jun 10 15:23 ./hdds/hdds/15292b6c-34d6-48d0-bb97-f58043767ade/current/containerDir0/1/chunks/104320375385424879.block
-rw-r--r-- 1 hadoop users 5242880 Jun 10 15:23 ./hdds/hdds/15292b6c-34d6-48d0-bb97-f58043767ade/current/containerDir0/1/chunks/104320375385490416.block
-rw-r--r-- 1 hadoop users 5242880 Jun 10 15:23 ./hdds/hdds/15292b6c-34d6-48d0-bb97-f58043767ade/current/containerDir0/1/chunks/104320375385555953.block {code}
> s3g multi-upload saved content incorrect when client uses aws java sdk 1.11.* jar
> ---------------------------------------------------------------------------------
>
> Key: HDDS-3512
> URL: https://issues.apache.org/jira/browse/HDDS-3512
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: S3
> Reporter: Sammi Chen
> Assignee: Marton Elek
> Priority: Blocker
> Labels: TriagePending
>
> The default multi-part size is 5MB, which is 5242880 byte, while all the chunks saved by s3g is 5246566 byte which is greater than 5MB.
> By looking into the ObjectEndpoint.java, it seems the chunk size is retrieved from the "Content-Length" header.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org