You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Pratyush Bhatt (Jira)" <ji...@apache.org> on 2024/04/10 08:57:00 UTC
[jira] [Created] (HDDS-10675) [Hsync] OverWrite using "sh key cp" shouldn't be allowed on hsynced files.
Pratyush Bhatt created HDDS-10675:
-------------------------------------
Summary: [Hsync] OverWrite using "sh key cp" shouldn't be allowed on hsynced files.
Key: HDDS-10675
URL: https://issues.apache.org/jira/browse/HDDS-10675
Project: Apache Ozone
Issue Type: Bug
Components: OM
Reporter: Pratyush Bhatt
Similar Scenario on the lines of HDDS-10670
*Scenario:* Do a _sh key cp_ on the same key that has been hsynced and is open.
*Observations:*
Open a file, write data and do hsync
{code:java}
Data written on iteration : 1
Data written on iteration : 2
Data written on iteration : 3
Data written on iteration : 4
Data written on iteration : 5
Data written on iteration : 6
Data written on iteration : 7
Data written on iteration : 8
Data written on iteration : 9
Data written on iteration : 10
Hsync completed on file, counter: 10 {code}
--Here the code sleeps of 60 seconds--
Checking the file is Open and hsynced
{code:java}
ozone admin om lof --service-id=ozone1712556312 --prefix=/hsyncvol/hsyncbuck/
1809 total open files (est.). Showing 1 open files (limit 100) under path prefix:
/hsyncvol/hsyncbuck/Client ID Creation time Hsync'ed Open File Path
112246067997904292 1712739074675 Yes /hsyncvol/hsyncbuck/-9223372036845049599/File_new_12.txtReached the end of the list.{code}
Doing a key info till now:
{code:java}
ozone sh key info hsyncvol/hsyncbuck/hsync/File_new_12.txt
24/04/10 08:51:18 INFO protocolPB.OmTransportFactory: Loading OM transport implementation org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified by configuration.
24/04/10 08:51:18 INFO client.ClientTrustManager: Loading certificates for client.
{
"volumeName" : "hsyncvol",
"bucketName" : "hsyncbuck",
"name" : "hsync/File_new_12.txt",
"dataSize" : 51200,
"creationTime" : "2024-04-10T08:51:14.675Z",
"modificationTime" : "2024-04-10T08:51:16.008Z",
"replicationConfig" : {
"replicationFactor" : "THREE",
"requiredNodes" : 3,
"replicationType" : "RATIS"
},
"metadata" : {
"hsyncClientId" : "112246067997904292"
},
"ozoneKeyLocations" : [ {
"containerID" : 6,
"localID" : 113750153625606497,
"length" : 51200,
"offset" : 0,
"keyOffset" : 0
} ],
"file" : true
}{code}
Now OverWrite the same key using _sh key cp_
{code:java}
>> ozone sh key put hsyncvol/hsyncbuck/dummy_file.txt /etc/passwd
24/04/10 08:51:32 INFO protocolPB.OmTransportFactory: Loading OM transport implementation org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified by configuration.
24/04/10 08:51:32 INFO client.ClientTrustManager: Loading certificates for client.
24/04/10 08:51:33 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties
24/04/10 08:51:33 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
24/04/10 08:51:33 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics system started
24/04/10 08:51:34 INFO metrics.MetricRegistries: Loaded MetricRegistries class org.apache.ratis.metrics.dropwizard3.Dm3MetricRegistriesImpl
>> ozone sh key cp /hsyncvol/hsyncbuck/ dummy_file.txt hsync/File_new_12.txt
24/04/10 08:51:48 INFO protocolPB.OmTransportFactory: Loading OM transport implementation org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified by configuration.
24/04/10 08:51:49 INFO client.ClientTrustManager: Loading certificates for client.
24/04/10 08:51:50 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties
24/04/10 08:51:50 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
24/04/10 08:51:50 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics system started
24/04/10 08:51:51 INFO metrics.MetricRegistries: Loaded MetricRegistries class org.apache.ratis.metrics.dropwizard3.Dm3MetricRegistriesImpl{code}
Key info, successfully overwritten:
{code:java}
ozone sh key info hsyncvol/hsyncbuck/hsync/File_new_12.txt
24/04/10 08:51:56 INFO protocolPB.OmTransportFactory: Loading OM transport implementation org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified by configuration.
24/04/10 08:51:57 INFO client.ClientTrustManager: Loading certificates for client.
{
"volumeName" : "hsyncvol",
"bucketName" : "hsyncbuck",
"name" : "hsync/File_new_12.txt",
"dataSize" : 6644,
"creationTime" : "2024-04-10T08:51:14.675Z",
"modificationTime" : "2024-04-10T08:51:51.673Z",
"replicationConfig" : {
"replicationFactor" : "THREE",
"requiredNodes" : 3,
"replicationType" : "RATIS"
},
"metadata" : { },
"ozoneKeyLocations" : [ {
"containerID" : 3,
"localID" : 113750153625606500,
"length" : 6644,
"offset" : 0,
"keyOffset" : 0
} ],
"file" : true
}{code}
But key is still open in the OpenKeyTable:
{code:java}
ozone admin om lof --service-id=ozone1712556312 --prefix=/hsyncvol/hsyncbuck/
1809 total open files (est.). Showing 1 open files (limit 100) under path prefix:
/hsyncvol/hsyncbuck/Client ID Creation time Hsync'ed Open File Path
112246067997904292 1712739074675 Yes /hsyncvol/hsyncbuck/-9223372036845049599/File_new_12.txtReached the end of the list.{code}
Perform LeaseRecovery:
{code:java}
ozone debug recover --path=ofs://ozone1712556312//hsyncvol/hsyncbuck/hsync/File_new_12.txt
24/04/10 08:52:13 INFO protocolPB.OmTransportFactory: Loading OM transport implementation org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified by configuration.
24/04/10 08:52:13 INFO client.ClientTrustManager: Loading certificates for client.
Lease recovery SUCCEEDED on ofs://ozone1712556312//hsyncvol/hsyncbuck/hsync/File_new_12.txt{code}
Still the key is open:
{code:java}
ozone admin om lof --service-id=ozone1712556312 --prefix=/hsyncvol/hsyncbuck/
1809 total open files (est.). Showing 1 open files (limit 100) under path prefix:
/hsyncvol/hsyncbuck/Client ID Creation time Hsync'ed Open File Path
112246067997904292 1712739074675 Yes /hsyncvol/hsyncbuck/-9223372036845049599/File_new_12.txtReached the end of the list.{code}
– 60 seconds of code pause is resumed now--
Still able to write the data into the file and hsync
{code:java}
Data written on iteration : 11
Data written on iteration : 12
Data written on iteration : 13
Data written on iteration : 14
Data written on iteration : 15
Data written on iteration : 16
Data written on iteration : 17
Data written on iteration : 18
Data written on iteration : 19
Data written on iteration : 20
Hsync completed on file, counter: 20 {code}
Basically looks like lease recovery gets broken if we allow such overwriting.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org