[jira] [Created] (HDDS-16936) SCM issues SCM role certificates without the SCM admin check and copies the CA basic constraint for every role

["Siyao Meng (Jira)" <ji...@apache.org>]

Siyao Meng created HDDS-16936:
---------------------------------

             Summary: SCM issues SCM role certificates without the SCM admin check and copies the CA basic constraint for every role
                 Key: HDDS-16936
                 URL: https://issues.apache.org/jira/browse/HDDS-16936
             Project: Apache Ozone
          Issue Type: Bug
            Reporter: Siyao Meng
         Attachments: CR-3-require-scm-admin-for-scm-role-certificates.patch

h3. Mechanism
{{SCMSecurityProtocolServer.getEncodedCertToString}} signs every CSR for the node type the request names. For {{NodeType.SCM}} it uses the root CA server when the SCM holds one and the sub CA server otherwise, and {{DefaultCAServer.getDuration}} gives that role {{hdds.x509.max.duration}}; the result is an SCM sub CA certificate, stored with role SCM.

Two RPCs reach it with that role. {{SCMSecurityProtocolServer.getSCMCertificate}} compares the cluster id of the request with its own and calls {{ScmUtils.checkIfCertSignRequestAllowed}} (the root CA rotation window). {{SCMSecurityProtocolServer.getCertificate}} (the {{NodeDetailsProto}} form, added for Recon in HDDS-6212) forwards {{nodeDetails.getNodeType()}} and checks nothing else. Neither consults {{StorageContainerManager.checkAdminAccess}}; in this server only {{removeExpiredCertificates}} does, while {{SCMBlockProtocolServer.addSCM}}, the other operation an SCM peer performs when it joins, does.

Separately, the sub CA server that issues OM, datanode and Recon certificates is built with {{DefaultCAProfile}} ({{StorageContainerManager}}, HDDS-5052), whose extension map accepts the basic constraints extension, and {{DefaultApprover.sign}} copies every extension the profile lists from the CSR into the certificate. So a basic constraints extension in a CSR is copied into the certificate for every role, not only SCM. {{DefaultCAServer.requestCertificate}} also completes its future exceptionally when {{DefaultApprover.inspectCSR}} rejects the CSR and then continues into {{signAndStoreCertificate}}, so a rejected CSR is still signed and persisted.

h3. Trigger
# An SCM role signing request ({{GetSCMCertificate}}, or {{GetCertificate}} with node type {{SCM}}) whose remote user is not in the SCM admin set.
# A CSR for a role other than SCM that carries the basic constraints extension, handled by a {{DefaultCAServer}} built with {{DefaultCAProfile}}.
# A CSR that {{DefaultApprover.inspectCSR}} rejects.

By history, not run on older code: {{getSCMCertificate}} and the {{DefaultCAProfile}} sub CA arrived with HDDS-4861 and HDDS-5052 (1.2.0), the {{NodeDetailsProto}} form with HDDS-6212 (1.3.0). The three core files are byte identical between the pin and the 2.2.1 tag.

h3. Impact
* The SCM admin check that gates the other SCM peer operation ({{addSCM}}) does not gate the issuance of SCM role certificates, which are CA certificates under the cluster root with the maximum duration, and the CA basic constraint is not limited to the SCM role.
* A CSR the approver rejected is still persisted in the SCM certificate store and returned by {{listCertificate}}.

h3. Reproduction
The regression tests in the attached patch cover it and fail on unmodified source. There is no standalone reproduction.
* {{TestDefaultCAServer.testBasicConstraintsAllowedForScmRoleOnly}} ({{hadoop-hdds/framework}}): a basic constraints CSR for the OM role is rejected and not stored, the same CSR for the SCM role is issued with the constraint and stored, and a CSR the profile rejects is neither signed nor stored ({{MockCAStore}} now records what is stored). Without the change it fails with {{Expected java.util.concurrent.ExecutionException to be thrown, but nothing was thrown}}.
* {{TestSecureOzoneCluster.assertScmSecurityProtocolRequiresAdminForScmRoleCertificate}} ({{hadoop-ozone/integration-test}}, inside {{testSecureScmAndOmStartupAndAccessControl}} next to the existing SCM admin protocol checks): both SCM role signing RPCs answer {{Access denied}} for a remote user outside the SCM admin set. Without the change it fails with {{Expected java.io.IOException to be thrown, but nothing was thrown}}.

h3. Patch
[^CR-3-require-scm-admin-for-scm-role-certificates.patch], against b9b33851b20fa11baf4b07982de9afb8ea44f85a. It also applies to master at 084c60d4fb9.

{{SCMSecurityProtocolServer.getEncodedCertToString}} calls {{checkAdminAccess}} with the RPC remote user when the role is SCM, so both RPCs are covered in one place. {{DefaultCAServer.requestCertificate}} rejects a CSR that carries the basic constraints extension for any role other than SCM before signing, and returns without signing when the approver rejects the CSR. {{MockCAStore}} records stored certificates for the test.

No wire or protocol change and no configuration change. A bootstrapping or renewing SCM requests its certificate under its own login user ({{SCMCertificateClient}}), and the SCM admin set always contains the starter user plus {{ozone.administrators}}, the same membership {{addSCM}} already requires of it, so no new deployment requirement. OM, datanode and Recon CSRs never carry the basic constraints extension (only {{SCMCertificateClient}} sets it), so the second change rejects no legitimate CSR. In a mixed version cluster an older SCM against a patched leader passes the same check it already passes for {{addSCM}}, and a patched SCM against an older primary meets no new check. Outside RPC, with no remote user, the check is a no op, as for {{removeExpiredCertificates}}; with {{ozone.authorization.enabled}} false it returns early, as every SCM admin check does.

With the patch {{TestDefaultCAServer}}, {{TestDefaultProfile}} ({{hadoop-hdds/framework}}) and {{TestRootCARotationManager}} ({{hadoop-hdds/server-scm}}) pass, the SCM part of {{TestSecureOzoneCluster.testSecureScmAndOmStartupAndAccessControl}} passes, and checkstyle on {{hadoop-hdds/framework}}, {{hadoop-hdds/server-scm}} and {{hadoop-ozone/integration-test}} is clean. Not run: a secure multi SCM bootstrap or certificate renewal with the check in place ({{TestSecretKeysApi}} did not start on the test machine for a host name resolution reason unrelated to the change), and the OM part of {{testSecureScmAndOmStartupAndAccessControl}} times out on that machine with and without the patch for the same reason. That bootstrap and renewal keep working follows from the shared login user by code reading and should be confirmed on CI before merge.

Found by code review of the SCM root CA rotation paths, as part of the TLA+ verification effort under HDDS-15926, on commit b9b33851b20fa11baf4b07982de9afb8ea44f85a. Checked against HDDS issues and apache/ozone pull requests for duplicates before filing. The attached patch is a proposal for review. Generated with Specula (Claude Fable 5.1).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org