You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Bharat Viswanadham (Jira)" <ji...@apache.org> on 2021/07/19 10:25:00 UTC
[jira] [Comment Edited] (HDDS-5393) Return latest version of key
location for client on createKey
[ https://issues.apache.org/jira/browse/HDDS-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17383243#comment-17383243 ]
Bharat Viswanadham edited comment on HDDS-5393 at 7/19/21, 10:24 AM:
---------------------------------------------------------------------
The problem here is the key create does below.
Key1 create - Blocks Version0 B1(0), B2(0), B3(0)
1st time override
Key1 override - Version0 -> B1(0), B2(0), B3(0), Version1 -> B1(0), B2(0), B3(0), New Blocks for this version (B4(1), B5(1), B6(1))
2nd time override
Key1 override - Version0 B1(0), B2(0), B3(0), Version1 -> B1(0), B2(0), B3(0), New Blocks for this version (B4(1), B5(1), B6(1)), Version 2 -> B1(0), B2(0), B3(0), B4(1), B5(1), B6(1), B7(2), B8(2), B9(2)
So, the problem is each time a key is override, each version has its version of blocks along with all older versions.
So, now when key create it returns all this versions. So, the list can become huge in size, depending on number of times key is override. In this patch I will fix to return latest version for create key. (I will open a new Jira to discuss about having each version including all its previous version)
was (Author: bharatviswa):
The problem here is the key create does below.
Key1 create - Blocks Version1 B1(0), B2(0), B3(0)
1st time override
Key1 override - Version1 -> B1(0), B2(0), B3(0), Version2 -> B1(0), B2(0), B3(0), New Blocks for this version (B4(1), B5(1), B6(1))
2nd time override
Key1 override - Version1 B1(0), B2(0), B3(0), Version2 -> B1(0), B2(0), B3(0), New Blocks for this version (B4(1), B5(1), B6(1)), Version 3 -> B1(0), B2(0), B3(0), B4(1), B5(1), B6(1), B7(2), B8(2), B9(2)
So, the problem is each time a key is override, each version has its version of blocks along with all older versions.
So, now when key create it returns all this versions. So, the list can become huge in size, depending on number of times key is override. In this patch I will fix to return latest version for create key. (I will open a new Jira to discuss about having each version including all its previous version)
> Return latest version of key location for client on createKey
> -------------------------------------------------------------
>
> Key: HDDS-5393
> URL: https://issues.apache.org/jira/browse/HDDS-5393
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Client
> Reporter: UENISHI Kota
> Assignee: Bharat Viswanadham
> Priority: Major
> Fix For: 1.2.0
>
>
> HDDS-5243 was a patch for omitting unnecessary key locations for clients on reading. But the same warning of large response size observed in our cluster for putting data. The patch can also be ported for putting data, as long as until object versioning is supported.
> My hypothesis is: The large message was originally, and possibly maybe due to this warning and sudden connection close from client side on reading large message in Hadoop IPC layer, from Ozone Manager - which causes hopeless 15 retries from RetryInvocationHandler. The retries create another entry in OpenKeyTable but they never moved to KeyTable because the key never gets commited.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org