[jira] [Commented] (HDDS-4813) [FSO]S3Multipart: Implement UploadCompleteRequest

["Rakesh Radhakrishnan (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDDS-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17285988#comment-17285988 ] 

Rakesh Radhakrishnan commented on HDDS-4813:
--------------------------------------------

Adding few details about the DB table level operations on S3 multipart request with an example for future reference.

 KeyName = "a/b/c/file1". 
Assume there are three parts, which will be having part number assigned like, 1, 2 and 3.

*User Operation-1)* S3InitiateMultipartUpload : KeyName="a/b/c/file1"
{code}
        1) UploadID will be generated by OM : UUID.randomUUID().toString() + "-" + UniqueId.next()
	2) Add "a/b/c" parent dirs into DirectoryTable
	3) Add entry to openFileTable:
		TableKey = multipartKey, which is nothing but <parentId/fileName/uploadId>
		TableValue = omKeyInfo, which will store fileName and parentID.
	4) Add entry to multipartFileInfoTable:		
		TableKey = multipartKey, which is nothing but <parentId/fileName/uploadId>
		TableValue = omKeyInfo, which will store uploadID and parentID. Since there is no real upload started, partKeyInfoList will be empty.
	
	Say, uploadID = 12345-0000. This is the unique identifier for uploading parts for the corresponding keyname.
{code}

*User Operation-2)* S3CreateMultipartKey : KeyName="a/b/c/file1", PartNumber=1, uploadID="12345-0000"
{code}
	1) ClientID will be generated by OM : UniqueId.next()
	2) Since it has uploadID, it will check OmKeyInfo in openFileTable.
	2) Add entry to OpenFileTable:
		TableKey = openFileKey, which is nothing but <parentId/fileName/clientID>
		TableValue = omKeyInfo, which will store fileName and parentID.
	
	This returns KeyOutputStream with unique ClientID. Say, ClientID=9777 and user streams data to it.
	Finally during close, will invoke commitMultipartUploadPart.
{code}

*User Operation-3)* S3CommitMultipartUploadPart: KeyName="a/b/c/file1", PartNumber=1, uploadID="12345-0000", ClientID=9777.
{code}
	1) Get <parentId/fileName/clientID> entry from openFileTable. Say, we got OmKeyInfo1 object from DB.
	2) Create 'PartKeyInfo' using the OmKeyInfo1 object.
		PartNumber = 1
		PartName   = file1 + 9777; // fileName + ClientID;
		keyInfo    = OmKeyInfo1    // here it stores only the fileName and parentID.
	3) Get <parentId/fileName/uploadId> entry from multipartFileInfoTable. Say, we got multipartKeyInfo1 object from DB.
	4) Add 'PartKeyInfo' to multipartKeyInfo1 object like,
			multipartKeyInfo.addPartKeyInfo(partNumber, partKeyInfo.build());
	5) Add the above multipartKeyInfo into the multipartFileInfoTable.
	6) Delete <parentId/fileName/clientID> entry from openFileTable.
{code}

*User Operation-4)* 
{code}
        Repeat S3CreateMultipartKey and S3CommitMultipartUploadPart for PartNumber=2 and PartNumber=3.
	After this operation, now the 'multipartKeyInfo' entry in multipartFileInfoTable will have three 'PartKeyInfo' items in the list.
{code}

*User Operation-5)* S3CompleteMultipartUpload: KeyName="a/b/c/file1", partsMap=[1,2,3], uploadID="12345-0000".
{code}
	1) Get <parentId/fileName/uploadId> entry from multipartFileInfoTable. Say, we got multipartKeyInfo1 object from DB.
	2) Compare all the user given part numbers and data size calculation etc using all the parts. I'm not exploring those info now as it is existing logic.
	3) Prepare OmKeyInfo using <parentId/fileName> and store entry into FileTable. This is now the reference to the KeyName="a/b/c/file1".
	4) Delete <parentId/fileName/uploadId> entry from openFileTable.
	5) Delete <parentId/fileName/uploadId> entry from multipartFileInfoTable.
{code}

 

> [FSO]S3Multipart: Implement UploadCompleteRequest
> -------------------------------------------------
>
>                 Key: HDDS-4813
>                 URL: https://issues.apache.org/jira/browse/HDDS-4813
>             Project: Apache Ozone
>          Issue Type: Sub-task
>            Reporter: Rakesh Radhakrishnan
>            Assignee: Rakesh Radhakrishnan
>            Priority: Major
>              Labels: pull-request-available
>
> This task is to implement prefix based FSO for the {{S3MultipartUploadCompleteRequest}}. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org