You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/04/05 15:05:00 UTC
[jira] [Updated] (HDDS-6434) EC: Overwriting an EC key with a Ratis key fails
[ https://issues.apache.org/jira/browse/HDDS-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated HDDS-6434:
---------------------------------
Labels: pull-request-available (was: )
> EC: Overwriting an EC key with a Ratis key fails
> ------------------------------------------------
>
> Key: HDDS-6434
> URL: https://issues.apache.org/jira/browse/HDDS-6434
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Stephen O'Donnell
> Assignee: cchenaxchen
> Priority: Major
> Labels: pull-request-available
>
> Creating a key with Replication Config EC, and then overwiting the key with a Ratis key fails with the error:
> {code}
> bash-4.2$ ozone sh key put -t EC -r rs-3-2-1024k /vol1/bucket1/key11 /etc/hosts
> bash-4.2$ ozone sh key put -t RATIS -r THREE /vol1/bucket1/key11 /etc/hosts
> Required instance of class org.apache.hadoop.hdds.client.ECReplicationConfig but object.getClass() is class org.apache.hadoop.hdds.client.RatisReplicationConfig
> {code}
> If you repeat this test with a RATIS/ONE key and overwrite with RATIS/THREE, the key info continues to have a replication config of ONE, but the first version goes into a ONE container. Then the next version goes into a THREE container.
> {code}
> bash-4.2$ ozone sh key put -t RATIS -r ONE /vol1/bucket1/key40 3mb
> bash-4.2$ ozone sh key put -t RATIS -r THREE /vol1/bucket1/key40 3mb
> bash-4.2$ ozone sh key info /vol1/bucket1/key40
> {
> "volumeName" : "vol1",
> "bucketName" : "bucket1",
> "name" : "key40",
> "dataSize" : 3145728,
> "creationTime" : "2022-03-09T21:53:29.492Z",
> "modificationTime" : "2022-03-09T21:53:39.109Z",
> "replicationConfig" : {
> "replicationFactor" : "ONE",
> "requiredNodes" : 1,
> "replicationType" : "RATIS"
> },
> "ozoneKeyLocations" : [ {
> "containerID" : 20,
> "localID" : 109611004723200035,
> "length" : 3145728,
> "offset" : 0,
> "keyOffset" : 0
> } ],
> "metadata" : { }
> }
> {code}
> Current container is 20:
> {code}
> bash-4.2$ ozone admin container info 20
> Container id: 20
> Pipeline id: 35b541ec-b50f-49ac-ae93-c8bdac3dd659
> Container State: OPEN
> Datanodes: [7465dd7d-6ee7-4c5b-b9f0-5d63b3bdbf09/ozone_datanode_4.ozone_default,
> 71c5c10f-6d38-428e-85aa-7202028e3c1c/ozone_datanode_2.ozone_default,
> 25c43ccd-b027-43f0-b3cc-88e415390e86/ozone_datanode_5.ozone_default]
> Replicas: [State: OPEN; ReplicaIndex: 0; Origin: 25c43ccd-b027-43f0-b3cc-88e415390e86; Location: 25c43ccd-b027-43f0-b3cc-88e415390e86/ozone_datanode_5.ozone_default,
> State: OPEN; ReplicaIndex: 0; Origin: 71c5c10f-6d38-428e-85aa-7202028e3c1c; Location: 71c5c10f-6d38-428e-85aa-7202028e3c1c/ozone_datanode_2.ozone_default,
> State: OPEN; ReplicaIndex: 0; Origin: 7465dd7d-6ee7-4c5b-b9f0-5d63b3bdbf09; Location: 7465dd7d-6ee7-4c5b-b9f0-5d63b3bdbf09/ozone_datanode_4.ozone_default]
> {code}
> Write an EC key - note container changes to 22, but repConfig is still RATIS/ONE in the key info, and container 22 is RATIS/ONE
> {code}
> ozone sh key put -t EC -r rs-3-2-1024k /vol1/bucket1/key40 3mb
> bash-4.2$ ozone sh key info /vol1/bucket1/key40
> {
> "volumeName" : "vol1",
> "bucketName" : "bucket1",
> "name" : "key40",
> "dataSize" : 3145728,
> "creationTime" : "2022-03-09T21:53:29.492Z",
> "modificationTime" : "2022-03-09T21:57:10.909Z",
> "replicationConfig" : {
> "replicationFactor" : "ONE",
> "requiredNodes" : 1,
> "replicationType" : "RATIS"
> },
> "ozoneKeyLocations" : [ {
> "containerID" : 22,
> "localID" : 109611004723200037,
> "length" : 3145728,
> "offset" : 0,
> "keyOffset" : 0
> } ],
> "metadata" : { }
> }
> {code}
> Even more strangely, there is a container 21 created, which is EC and has 3MB written to the first replica, and hence not correctly stripped across the blocks. Its also not stored in the container:
> {code}
> bash-4.2$ ozone admin container info 21 --json
> ...
> "placeOfBirth" : "25c43ccd-b027-43f0-b3cc-88e415390e86",
> "sequenceId" : 0,
> "keyCount" : 0,
> "bytesUsed" : 3145728,
> "replicaIndex" : 1
> {code}
> Its like it writes a RATIS/ONE and also writes the data to an EC container too
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org