You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "UENISHI Kota (Jira)" <ji...@apache.org> on 2021/06/28 10:15:00 UTC
[jira] [Commented] (HDDS-5393) Return latest version of key
location for client on createKey
[ https://issues.apache.org/jira/browse/HDDS-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370510#comment-17370510 ]
UENISHI Kota commented on HDDS-5393:
------------------------------------
I did a one-day survey on this issue, and it turned out it's not that straightforward like other read requests. Opening a new key on every overwrite leads to potentially infinite increase of KeyLocations with a lot of duplicates on blocks. IMHO it needs some consistent logic to merge those duplicated KeyLocations with addressing concurrent uploads.
> Return latest version of key location for client on createKey
> -------------------------------------------------------------
>
> Key: HDDS-5393
> URL: https://issues.apache.org/jira/browse/HDDS-5393
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Client
> Reporter: UENISHI Kota
> Priority: Major
> Fix For: 1.2.0
>
>
> HDDS-5243 was a patch for omitting unnecessary key locations for clients on reading. But the same warning of large response size observed in our cluster for putting data. The patch can also be ported for putting data, as long as until object versioning is supported.
> My hypothesis is: The large message was originally, and possibly maybe due to this warning and sudden connection close from client side on reading large message in Hadoop IPC layer, from Ozone Manager - which causes hopeless 15 retries from RetryInvocationHandler. The retries create another entry in OpenKeyTable but they never moved to KeyTable because the key never gets commited.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org