You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Pratyush Bhatt (Jira)" <ji...@apache.org> on 2023/10/03 10:19:00 UTC
[jira] [Updated] (HDDS-9378) SCM Certs still listed even after decommissioning.
[ https://issues.apache.org/jira/browse/HDDS-9378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pratyush Bhatt updated HDDS-9378:
---------------------------------
Description:
Steps:
* Decommission a SCM Node
* Stop and do cleanup, delete the SCM role host.
* Recommission a new SCM node.
* Do the cert list using _ozone admin cert list_
Observation:
{code:java}
[root@ozn-decom22-1 ~]# ozone admin cert list
Certificate list:(Type=VALID, BatchSize=20, CertCount=17)
SerialNumber Valid From Expiry Subject Issuer
1 Tue Oct 03 04:02:07 UTC 2023 Fri Nov 10 04:02:07 UTC 2028 CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214809950891256 Tue Oct 03 04:02:07 UTC 2023 Fri Nov 10 04:02:07 UTC 2028 CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214827957269038 Tue Oct 03 04:02:25 UTC 2023 Wed Oct 02 04:02:25 UTC 2024 CN=dn@ozn-decom22-6.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214827993342530 Tue Oct 03 04:02:25 UTC 2023 Wed Oct 02 04:02:25 UTC 2024 CN=dn@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214828015859473 Tue Oct 03 04:02:25 UTC 2023 Wed Oct 02 04:02:25 UTC 2024 CN=recon@ozn-decom22-7.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214829027141451 Tue Oct 03 04:02:26 UTC 2023 Wed Oct 02 04:02:26 UTC 2024 CN=om@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214829895666723 Tue Oct 03 04:02:27 UTC 2023 Fri Nov 10 04:02:27 UTC 2028 CN=scm-sub-60330780159055185@ozn-decom22-7.ozn-decom22.<>,OU=5a3f31eb-c5da-486a-a2bd-9cce5005dfa3,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214830223685408 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-5.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214830235039849 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-7.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214830263313767 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-8.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214830375917531 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-4.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214830506264985 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-3.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214831371665280 Tue Oct 03 04:02:28 UTC 2023 Wed Oct 02 04:02:28 UTC 2024 CN=om@ozn-decom22-7.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214831476909927 Tue Oct 03 04:02:28 UTC 2023 Fri Nov 10 04:02:28 UTC 2028 CN=scm-sub-9033920599087679@ozn-decom22-2.ozn-decom22.<>,OU=c7597503-2306-4726-ba7d-13bc1983deed,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214831793721639 Tue Oct 03 04:02:29 UTC 2023 Wed Oct 02 04:02:29 UTC 2024 CN=dn@ozn-decom22-2.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214835433198668 Tue Oct 03 04:02:32 UTC 2023 Wed Oct 02 04:02:32 UTC 2024 CN=om@ozn-decom22-2.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7228003267391949 Tue Oct 03 07:42:00 UTC 2023 Fri Nov 10 07:42:00 UTC 2028 CN=scm-sub-48184021180367933@ozn-decom22-5.ozn-decom22.<>,OU=5aa12cd4-eaef-4840-a12f-88aa628f0c9d,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
[root@ozn-decom22-1 ~]# {code}
We can see there are still 4 SCM sub-ca certs available.
{code:java}
1. 7214809950891256 Tue Oct 03 04:02:07 UTC 2023 Fri Nov 10 04:02:07 UTC 2028 CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee{code}
{code:java}
2. 7214829895666723 Tue Oct 03 04:02:27 UTC 2023 Fri Nov 10 04:02:27 UTC 2028 CN=scm-sub-60330780159055185@ozn-decom22-7.ozn-decom22.<>,OU=5a3f31eb-c5da-486a-a2bd-9cce5005dfa3,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee{code}
{code:java}
3. 7214831476909927 Tue Oct 03 04:02:28 UTC 2023 Fri Nov 10 04:02:28 UTC 2028 CN=scm-sub-9033920599087679@ozn-decom22-2.ozn-decom22.<>,OU=c7597503-2306-4726-ba7d-13bc1983deed,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee{code}
{code:java}
4. 7228003267391949 Tue Oct 03 07:42:00 UTC 2023 Fri Nov 10 07:42:00 UTC 2028 CN=scm-sub-48184021180367933@ozn-decom22-5.ozn-decom22.<>,OU=5aa12cd4-eaef-4840-a12f-88aa628f0c9d,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee {code}
If we see the SCM Nodes:
{code:java}
[root@ozn-decom22-1 ~]# ozone admin scm roles
ozn-decom22-1.ozn-decom22.<>:<>:LEADER:6457e309-4af0-47c6-9b5e-452f9457dfa5:<>
ozn-decom22-5.ozn-decom22.<>:<>:FOLLOWER:5aa12cd4-eaef-4840-a12f-88aa628f0c9d:<>
ozn-decom22-7.ozn-decom22.<>:<>:FOLLOWER:5a3f31eb-c5da-486a-a2bd-9cce5005dfa3:<>{code}
Node 1, 5 and 7 has SCM Role and Node 2 was the one that got decommissioned earlier. In the Cert list command, the certificate is still visible for Node2(i.e. ozn-decom22-2.ozn-decom22.<>) and still has a expiry of Fri Nov 10 04:02:28 UTC 2028
Expected behavior:
The certificates of Decommissioned node should have expired and not shown in the Cert list command.
was:
Steps:
* Decommission a SCM Node
* Stop and do cleanup, delete the SCM role host.
* Recommission a new SCM node.
* Do the cert list using _ozone admin cert list_
Observation:
{code:java}
[root@ozn-decom22-1 ~]# ozone admin cert list
Certificate list:(Type=VALID, BatchSize=20, CertCount=17)
SerialNumber Valid From Expiry Subject Issuer
1 Tue Oct 03 04:02:07 UTC 2023 Fri Nov 10 04:02:07 UTC 2028 CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214809950891256 Tue Oct 03 04:02:07 UTC 2023 Fri Nov 10 04:02:07 UTC 2028 CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214827957269038 Tue Oct 03 04:02:25 UTC 2023 Wed Oct 02 04:02:25 UTC 2024 CN=dn@ozn-decom22-6.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214827993342530 Tue Oct 03 04:02:25 UTC 2023 Wed Oct 02 04:02:25 UTC 2024 CN=dn@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214828015859473 Tue Oct 03 04:02:25 UTC 2023 Wed Oct 02 04:02:25 UTC 2024 CN=recon@ozn-decom22-7.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214829027141451 Tue Oct 03 04:02:26 UTC 2023 Wed Oct 02 04:02:26 UTC 2024 CN=om@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214829895666723 Tue Oct 03 04:02:27 UTC 2023 Fri Nov 10 04:02:27 UTC 2028 CN=scm-sub-60330780159055185@ozn-decom22-7.ozn-decom22.<>,OU=5a3f31eb-c5da-486a-a2bd-9cce5005dfa3,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214830223685408 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-5.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214830235039849 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-7.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214830263313767 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-8.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214830375917531 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-4.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214830506264985 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-3.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214831371665280 Tue Oct 03 04:02:28 UTC 2023 Wed Oct 02 04:02:28 UTC 2024 CN=om@ozn-decom22-7.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214831476909927 Tue Oct 03 04:02:28 UTC 2023 Fri Nov 10 04:02:28 UTC 2028 CN=scm-sub-9033920599087679@ozn-decom22-2.ozn-decom22.<>,OU=c7597503-2306-4726-ba7d-13bc1983deed,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214831793721639 Tue Oct 03 04:02:29 UTC 2023 Wed Oct 02 04:02:29 UTC 2024 CN=dn@ozn-decom22-2.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7214835433198668 Tue Oct 03 04:02:32 UTC 2023 Wed Oct 02 04:02:32 UTC 2024 CN=om@ozn-decom22-2.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
7228003267391949 Tue Oct 03 07:42:00 UTC 2023 Fri Nov 10 07:42:00 UTC 2028 CN=scm-sub-48184021180367933@ozn-decom22-5.ozn-decom22.<>,OU=5aa12cd4-eaef-4840-a12f-88aa628f0c9d,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
[root@ozn-decom22-1 ~]# {code}
We can see there are still 4 SCM sub-ca certs available.
{code:java}
1. 7214809950891256 Tue Oct 03 04:02:07 UTC 2023 Fri Nov 10 04:02:07 UTC 2028 CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee{code}
{code:java}
2. 7214829895666723 Tue Oct 03 04:02:27 UTC 2023 Fri Nov 10 04:02:27 UTC 2028 CN=scm-sub-60330780159055185@ozn-decom22-7.ozn-decom22.<>,OU=5a3f31eb-c5da-486a-a2bd-9cce5005dfa3,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee{code}
{code:java}
3. 7214831476909927 Tue Oct 03 04:02:28 UTC 2023 Fri Nov 10 04:02:28 UTC 2028 CN=scm-sub-9033920599087679@ozn-decom22-2.ozn-decom22.<>,OU=c7597503-2306-4726-ba7d-13bc1983deed,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee{code}
{code:java}
4. 7228003267391949 Tue Oct 03 07:42:00 UTC 2023 Fri Nov 10 07:42:00 UTC 2028 CN=scm-sub-48184021180367933@ozn-decom22-5.ozn-decom22.<>,OU=5aa12cd4-eaef-4840-a12f-88aa628f0c9d,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee {code}
If we see the SCM Nodes:
{code:java}
[root@ozn-decom22-1 ~]# ozone admin scm roles
ozn-decom22-1.ozn-decom22.<>:9894:LEADER:6457e309-4af0-47c6-9b5e-452f9457dfa5:172.27.212.78
ozn-decom22-5.ozn-decom22.<>:9894:FOLLOWER:5aa12cd4-eaef-4840-a12f-88aa628f0c9d:172.27.169.210
ozn-decom22-7.ozn-decom22.<>:9894:FOLLOWER:5a3f31eb-c5da-486a-a2bd-9cce5005dfa3:172.27.17.73 {code}
Node 1, 5 and 7 has SCM Role and Node 2 was the one that got decommissioned earlier. In the Cert list command, the certificate is still visible for Node2(i.e. ozn-decom22-2.ozn-decom22.<>) and still has a expiry of Fri Nov 10 04:02:28 UTC 2028
Expected behavior:
The certificates of Decommissioned node should have expired and not shown in the Cert list command.
> SCM Certs still listed even after decommissioning.
> --------------------------------------------------
>
> Key: HDDS-9378
> URL: https://issues.apache.org/jira/browse/HDDS-9378
> Project: Apache Ozone
> Issue Type: Bug
> Components: Certificates, SCM
> Reporter: Pratyush Bhatt
> Priority: Major
>
> Steps:
> * Decommission a SCM Node
> * Stop and do cleanup, delete the SCM role host.
> * Recommission a new SCM node.
> * Do the cert list using _ozone admin cert list_
>
> Observation:
> {code:java}
> [root@ozn-decom22-1 ~]# ozone admin cert list
> Certificate list:(Type=VALID, BatchSize=20, CertCount=17)
> SerialNumber Valid From Expiry Subject Issuer
> 1 Tue Oct 03 04:02:07 UTC 2023 Fri Nov 10 04:02:07 UTC 2028 CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214809950891256 Tue Oct 03 04:02:07 UTC 2023 Fri Nov 10 04:02:07 UTC 2028 CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214827957269038 Tue Oct 03 04:02:25 UTC 2023 Wed Oct 02 04:02:25 UTC 2024 CN=dn@ozn-decom22-6.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214827993342530 Tue Oct 03 04:02:25 UTC 2023 Wed Oct 02 04:02:25 UTC 2024 CN=dn@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214828015859473 Tue Oct 03 04:02:25 UTC 2023 Wed Oct 02 04:02:25 UTC 2024 CN=recon@ozn-decom22-7.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214829027141451 Tue Oct 03 04:02:26 UTC 2023 Wed Oct 02 04:02:26 UTC 2024 CN=om@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214829895666723 Tue Oct 03 04:02:27 UTC 2023 Fri Nov 10 04:02:27 UTC 2028 CN=scm-sub-60330780159055185@ozn-decom22-7.ozn-decom22.<>,OU=5a3f31eb-c5da-486a-a2bd-9cce5005dfa3,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214830223685408 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-5.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214830235039849 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-7.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214830263313767 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-8.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214830375917531 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-4.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214830506264985 Tue Oct 03 04:02:27 UTC 2023 Wed Oct 02 04:02:27 UTC 2024 CN=dn@ozn-decom22-3.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214831371665280 Tue Oct 03 04:02:28 UTC 2023 Wed Oct 02 04:02:28 UTC 2024 CN=om@ozn-decom22-7.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214831476909927 Tue Oct 03 04:02:28 UTC 2023 Fri Nov 10 04:02:28 UTC 2028 CN=scm-sub-9033920599087679@ozn-decom22-2.ozn-decom22.<>,OU=c7597503-2306-4726-ba7d-13bc1983deed,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214831793721639 Tue Oct 03 04:02:29 UTC 2023 Wed Oct 02 04:02:29 UTC 2024 CN=dn@ozn-decom22-2.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7214835433198668 Tue Oct 03 04:02:32 UTC 2023 Wed Oct 02 04:02:32 UTC 2024 CN=om@ozn-decom22-2.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> 7228003267391949 Tue Oct 03 07:42:00 UTC 2023 Fri Nov 10 07:42:00 UTC 2028 CN=scm-sub-48184021180367933@ozn-decom22-5.ozn-decom22.<>,OU=5aa12cd4-eaef-4840-a12f-88aa628f0c9d,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee
> [root@ozn-decom22-1 ~]# {code}
>
> We can see there are still 4 SCM sub-ca certs available.
> {code:java}
> 1. 7214809950891256 Tue Oct 03 04:02:07 UTC 2023 Fri Nov 10 04:02:07 UTC 2028 CN=scm-sub-7214809915597529@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee{code}
> {code:java}
> 2. 7214829895666723 Tue Oct 03 04:02:27 UTC 2023 Fri Nov 10 04:02:27 UTC 2028 CN=scm-sub-60330780159055185@ozn-decom22-7.ozn-decom22.<>,OU=5a3f31eb-c5da-486a-a2bd-9cce5005dfa3,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee{code}
> {code:java}
> 3. 7214831476909927 Tue Oct 03 04:02:28 UTC 2023 Fri Nov 10 04:02:28 UTC 2028 CN=scm-sub-9033920599087679@ozn-decom22-2.ozn-decom22.<>,OU=c7597503-2306-4726-ba7d-13bc1983deed,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee{code}
> {code:java}
> 4. 7228003267391949 Tue Oct 03 07:42:00 UTC 2023 Fri Nov 10 07:42:00 UTC 2028 CN=scm-sub-48184021180367933@ozn-decom22-5.ozn-decom22.<>,OU=5aa12cd4-eaef-4840-a12f-88aa628f0c9d,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee CN=scm-1@ozn-decom22-1.ozn-decom22.<>,OU=6457e309-4af0-47c6-9b5e-452f9457dfa5,O=CID-7c55997e-1c1d-42be-9fc3-ec6b60f5e3ee {code}
>
> If we see the SCM Nodes:
> {code:java}
> [root@ozn-decom22-1 ~]# ozone admin scm roles
> ozn-decom22-1.ozn-decom22.<>:<>:LEADER:6457e309-4af0-47c6-9b5e-452f9457dfa5:<>
> ozn-decom22-5.ozn-decom22.<>:<>:FOLLOWER:5aa12cd4-eaef-4840-a12f-88aa628f0c9d:<>
> ozn-decom22-7.ozn-decom22.<>:<>:FOLLOWER:5a3f31eb-c5da-486a-a2bd-9cce5005dfa3:<>{code}
> Node 1, 5 and 7 has SCM Role and Node 2 was the one that got decommissioned earlier. In the Cert list command, the certificate is still visible for Node2(i.e. ozn-decom22-2.ozn-decom22.<>) and still has a expiry of Fri Nov 10 04:02:28 UTC 2028
> Expected behavior:
> The certificates of Decommissioned node should have expired and not shown in the Cert list command.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org