You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "z-kovacs (via GitHub)" <gi...@apache.org> on 2023/06/08 12:52:44 UTC

[GitHub] [pulsar] z-kovacs opened a new pull request, #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

z-kovacs opened a new pull request, #20542:
URL: https://github.com/apache/pulsar/pull/20542

   Fixes #20373
   
   ### Motivation
   
   <!-- Explain here the context, and why you're making that change. What is the problem you're trying to solve. -->
   If we configure pulsar following the doc to use BouncyCastle FIPS library instead of the default non-FIPS version, we receive a `ClassNotFoundException: org.bouncycastle.jce.provider.BouncyCastleProvider` - as the MessageCryptoBc implementation is not prepared to use the FIPS library API (way more restrictive).
   
   BC FIPS has some restrictions:
   - ECDSA keys cannot be used neither for encryption nor for key wrapping only RSA keys
   - RSA keys can only be used for key wrapping, not encryption
   - different API to manipulate ECDSA key loading/saving
   
   ### Modifications
   
   <!-- Describe the modifications you've done. -->
   I have lifted out the BC FIPS and non-FIPS related code from `MessageCryptoBc` into the already existing `bouncy-castle-bc` and `bouncy-castle-bcfips` module, and created a `bouncy-castle-common` module with an interface definition and a dynamic loader of the implementation based on the security provider in classpath.
   `MessageCryptoBc` now accessing the BC version specific crypto functionality via the dynamically loaded delegate.
   
   in non-FIPS mode we use encrypt mode of cyphers to encrypt data key (probably should be wrap, but ECIES cypher does not support wrap).
   In FIPS mode you have to use wrap. So as of now it is not possible to unify code to wrap the data key.
   
   
   ### Verifying this change
   
   - [ ] Make sure that the change passes the CI checks.
   
   This change added tests and can be verified as follows:
   
     - *Added unittest for making sure key wrapping and encrypting are compatible for RSA keys*
     - *Added unittest for making sure that encryption works end-to-end when only bc FIPS libraries are loaded*
   
   ### Does this pull request potentially affect one of the following parts:
   
   <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
   
   *If the box was checked, please highlight the changes*
   
   - [ ] Dependencies (add or upgrade a dependency)
   - [ ] The public API
   - [ ] The schema
   - [ ] The default values of configurations
   - [ ] The threading model
   - [ ] The binary protocol
   - [ ] The REST endpoints
   - [ ] The admin CLI options
   - [ ] The metrics
   - [ ] Anything that affects deployment
   
   Note: 
   we have to warn users if they want to change the client to use BC FIPS client lib, the messages produced with non FIPS setup and where the master key was ECDSA key - won't be readable! (FIPS has way less enabled cyphers.)
   we have to add to website to BC instructions, that you can mix fips ad non-fips producers and consumers as long as you use RSA master keys, since ECDSA keys cannot be used to wrap keys in BC FIPS.
   
   ### Documentation
   
   <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
   
   - [ ] `doc` <!-- Your PR contains doc changes. -->
   - [x] `doc-required` <!-- Your PR changes impact docs and you will update later -->
   - [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
   - [ ] `doc-complete` <!-- Docs have been already added -->
   
   ### Matching PR in forked repository
   
   PR in forked repository: <!-- ENTER URL HERE -->
   
   <!--
   After opening this PR, the build in apache/pulsar will fail and instructions will
   be provided for opening a PR in the PR author's forked repository.
   
   apache/pulsar pull requests should be first tested in your own fork since the 
   apache/pulsar CI based on GitHub Actions has constrained resources and quota.
   GitHub Actions provides separate quota for pull requests that are executed in 
   a forked repository.
   
   The tests will be run in the forked repository until all PR review comments have
   been handled, the tests pass and the PR is approved by a reviewer.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] nodece commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1230644867


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;

Review Comment:
   ```suggestion
       private BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1233746028


##########
tiered-storage/jcloud/pom.xml:
##########
@@ -103,7 +103,12 @@
       <version>${jclouds.version}</version>
       <scope>provided</scope>
     </dependency>
-
+    <dependency>

Review Comment:
   @nodece - could you pls opine on this? shall I remove it?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1230742217


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;

Review Comment:
   very good spot - thank you - added `final` as well



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] nodece commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1229923721


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   > do we have somebody BC expert we can ask to opine?
   
   Sorry, I don't know that.
   
   Could we use the `NONCEANDIV`  for generating ASE and sending messages? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#issuecomment-1586823806

   It seems I cannot edit labels, so I cannot add `ready-to-test` to trigger tests in this repo.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1233745347


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   Please let me know if you need help on the PIP and what are the next steps.
   
   thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1228445478


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   Could you please elaborate on >if "NONCEANDIV" does not work<? 
   https://downloads.bouncycastle.org/fips-java/BC-FJA-UserGuide-1.0.2.pdf section 13 explicitly mentions these 2 securerandom generators in BC FIPS.
   DEFAULT is the prediction resistant one - so the safest (I think to generate AES keys), and NONCEANDIV is for nonce and IV generation, exactly what we use that random generator when sending messages. But the original implementation was using the on-blocking pseudorandom for both AES key gen and IV generation.
   
   Overall DEFAULT seems to be the stronger one - so depending more on a good entropy source, so DEFAULT may be more prone to blocking if there is not enough entropy.
   
   Maybe what you mean that >if the user cannot give access to a good entropy source, and DEFAULT securerandom is blocking?< - that is definitely worth drilling into and perhaps document that how the default is configured in BC FIPS and what are the assumptions.
   
   Thanks for the deep review!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] nodece commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1228211888


##########
bouncy-castle/bc-common/src/test/java/org/apache/pulsar/client/impl/crypto/WrappingVersusEncryptionCrossCompatibilityTestBase.java:
##########
@@ -0,0 +1,236 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl.crypto;
+
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.testng.AssertJUnit.assertEquals;
+import com.google.common.collect.ImmutableMap;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+import java.security.NoSuchAlgorithmException;
+import java.security.NoSuchProviderException;
+import java.security.PrivateKey;
+import java.security.PublicKey;
+import java.security.spec.InvalidKeySpecException;
+import java.util.Optional;
+import javax.crypto.KeyGenerator;
+import javax.crypto.SecretKey;
+import javax.crypto.SecretKeyFactory;
+import javax.crypto.spec.SecretKeySpec;
+import org.apache.commons.io.FileUtils;
+import org.apache.pulsar.client.api.PulsarClientException;
+import org.apache.pulsar.common.util.SecurityUtility;
+import org.hamcrest.Matcher;
+import org.testng.annotations.DataProvider;
+import org.testng.annotations.Test;
+
+public abstract class WrappingVersusEncryptionCrossCompatibilityTestBase {
+
+    //from https://stackoverflow.com/a/140861
+    //TODO use java.format.HexFormat after we move to java17 language features
+    protected static byte[] hexStringToByteArray(String s/* s must be an even-length string. */) {

Review Comment:
   You can use `Hex.decodeHex(s)` instead.



##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   IMO, if the "NONCEANDIV" doesn't work, we should consider the "DEFAULT" because we will end up using the "DEFAULT".
   
   And then you can remove the following code:
   ```
               SecureRandom secureRandomForKeygen = null;
   
               switch (providerName) {
                   case SecurityUtility.BC: {
                       //TODO is this prediction resistant for generating keys?
                       secureRandomForKeygen = secureRandom;
                       break;
                   }
                   case SecurityUtility.BC_FIPS: {
                       try {
                           //prediction resistant
                           secureRandomForKeygen = SecureRandom.getInstance("DEFAULT", providerName);
                       } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
                           throw new RuntimeException(
                                   "In BC FIPS mode, we expect the specific Random generators to be available!");
                       }
                       break;
                   }
                   default:
                       throw new IllegalStateException("Provider not handled for encryption: " + providerName);
               }
   ```



##########
bouncy-castle/bc-common/src/test/java/org/apache/pulsar/client/impl/crypto/WrappingVersusEncryptionCrossCompatibilityTestBase.java:
##########
@@ -0,0 +1,236 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl.crypto;
+
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.testng.AssertJUnit.assertEquals;
+import com.google.common.collect.ImmutableMap;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+import java.security.NoSuchAlgorithmException;
+import java.security.NoSuchProviderException;
+import java.security.PrivateKey;
+import java.security.PublicKey;
+import java.security.spec.InvalidKeySpecException;
+import java.util.Optional;
+import javax.crypto.KeyGenerator;
+import javax.crypto.SecretKey;
+import javax.crypto.SecretKeyFactory;
+import javax.crypto.spec.SecretKeySpec;
+import org.apache.commons.io.FileUtils;
+import org.apache.pulsar.client.api.PulsarClientException;
+import org.apache.pulsar.common.util.SecurityUtility;
+import org.hamcrest.Matcher;
+import org.testng.annotations.DataProvider;
+import org.testng.annotations.Test;
+
+public abstract class WrappingVersusEncryptionCrossCompatibilityTestBase {
+
+    //from https://stackoverflow.com/a/140861
+    //TODO use java.format.HexFormat after we move to java17 language features
+    protected static byte[] hexStringToByteArray(String s/* s must be an even-length string. */) {
+        int len = s.length();
+        byte[] data = new byte[len / 2];
+        for (int i = 0; i < len; i += 2) {
+            data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
+                    + Character.digit(s.charAt(i + 1), 16));
+        }
+        return data;
+    }
+
+    //from https://stackoverflow.com/a/9855338
+    private static final byte[] HEX_ARRAY = "0123456789ABCDEF".getBytes(StandardCharsets.US_ASCII);
+
+    //TODO use java.format.HexFormat after we move to java17 language features
+    protected static String bytesToHex(byte[] bytes) {
+        byte[] hexChars = new byte[bytes.length * 2];

Review Comment:
   Yan can use `Hex.encodeHexString(bytes)` instead.



##########
bouncy-castle/bcfips/pom.xml:
##########
@@ -51,6 +58,12 @@
       <artifactId>bcpkix-fips</artifactId>
       <version>${bouncycastle.bcpkix-fips.version}</version>
     </dependency>
+    <dependency>
+      <groupId>${project.groupId}</groupId>

Review Comment:
   Duplicated. See line 43-48.



##########
bouncy-castle/bc-common/src/main/java/org/apache/pulsar/client/impl/crypto/BcVersionSpecificCryptoUtilityFactory.java:
##########
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl.crypto;
+
+import java.lang.reflect.InvocationTargetException;
+import org.apache.pulsar.common.util.SecurityUtility;
+
+class BcVersionSpecificCryptoUtilityFactory {
+
+    static BcVersionSpecificCryptoUtility createNewInstance() {
+        String providerName = SecurityUtility.getProvider().getName();
+        try {
+            switch (providerName) {
+                case SecurityUtility.BC:
+                    return (BcVersionSpecificCryptoUtility) BcVersionSpecificCryptoUtility.class.getClassLoader()
+                            .loadClass("org.apache.pulsar.client.impl.crypto.bc.BCNonFipsSpecificUtility")
+                            .getConstructor()
+                            .newInstance();
+                case SecurityUtility.BC_FIPS:
+                    return (BcVersionSpecificCryptoUtility) BcVersionSpecificCryptoUtility.class.getClassLoader()
+                            .loadClass("org.apache.pulsar.client.impl.bcfips.BCFipsSpecificUtility")
+                            .getConstructor().newInstance();
+                default:
+                    throw new IllegalStateException(
+                            "Provider not handled for BC Specific delegate creation: " + providerName);
+            }
+        } catch (InstantiationException | IllegalAccessException | InvocationTargetException | NoSuchMethodException

Review Comment:
   I suggest using the same package name, so like:
   - org.apache.pulsar.client.impl.crypto.bc.BCNonFipsSpecificUtility
   - org.apache.pulsar.client.impl.crypto.bcfips.BCFipsSpecificUtility



##########
bouncy-castle/bc-common/src/main/java/org/apache/pulsar/client/impl/crypto/BcVersionSpecificCryptoUtility.java:
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl.crypto;
+
+import static org.apache.pulsar.client.impl.crypto.BcVersionSpecificCryptoUtilityFactory.createNewInstance;
+import java.security.PrivateKey;
+import java.security.PublicKey;
+import java.util.Optional;
+import javax.crypto.SecretKey;
+import org.apache.pulsar.client.api.PulsarClientException;
+
+/**
+ * Facade to various crypto functions implemented by BouncyCastle FIPS and Non-fips module.
+ */
+public interface BcVersionSpecificCryptoUtility {
+
+    BcVersionSpecificCryptoUtility INSTANCE = createNewInstance();
+
+    String RSA = "RSA";
+
+    // Ideally the transformation should also be part of the message property. This will prevent client
+    // from assuming hardcoded value. However, it will increase the size of the message even further.
+    String RSA_TRANS = "RSA/NONE/OAEPWithSHA1AndMGF1Padding";
+
+
+    byte[] encryptDataKey(String logCtx, String keyName, PublicKey publicKey, SecretKey dataKey)
+            throws PulsarClientException.CryptoException;
+
+    Optional<SecretKey> deCryptDataKey(String datakeyAlgorithm, String logCtx, String keyName, PrivateKey privateKey,
+                                       byte[] encryotedDataKey);

Review Comment:
   ```suggestion
                                          byte[] encryptedDataKey);
   ```
   



##########
bouncy-castle/bc-common/src/test/java/org/apache/pulsar/client/impl/crypto/EncKeyReader.java:
##########
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl.crypto;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Map;
+import java.util.Optional;
+import org.apache.pulsar.client.api.CryptoKeyReader;
+import org.apache.pulsar.client.api.EncryptionKeyInfo;
+import org.testng.Assert;
+
+public class EncKeyReader implements CryptoKeyReader {
+
+    public enum KeyType {
+        PUBLIC("public"), PRIVATE("private");
+        private final String id;
+
+        KeyType(String id) {
+            this.id = id;
+        }
+    }
+
+    public static byte[] getKeyAsPEM(KeyType keyType, String keyName, Map<String, String> keyMeta) {
+        String CERT_FILE_PATH = "../src/test/resources/certificate/" + keyType.id + "-key." + keyName;
+        if (Files.isReadable(Paths.get(CERT_FILE_PATH))) {

Review Comment:
   It seems that you can directly return `Files.readAllBytes(Paths.get(CERT_FILE_PATH))`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1230736727


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   But you are right: if somebody messes up the default secure random config all bets are off, but if somebody attempt that they better know what they are doing.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1229306313


##########
bouncy-castle/bc-common/src/test/java/org/apache/pulsar/client/impl/crypto/EncKeyReader.java:
##########
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl.crypto;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Map;
+import java.util.Optional;
+import org.apache.pulsar.client.api.CryptoKeyReader;
+import org.apache.pulsar.client.api.EncryptionKeyInfo;
+import org.testng.Assert;
+
+public class EncKeyReader implements CryptoKeyReader {
+
+    public enum KeyType {
+        PUBLIC("public"), PRIVATE("private");
+        private final String id;
+
+        KeyType(String id) {
+            this.id = id;
+        }
+    }
+
+    public static byte[] getKeyAsPEM(KeyType keyType, String keyName, Map<String, String> keyMeta) {
+        String CERT_FILE_PATH = "../src/test/resources/certificate/" + keyType.id + "-key." + keyName;
+        if (Files.isReadable(Paths.get(CERT_FILE_PATH))) {

Review Comment:
   hope that is what you meant



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] nodece commented on pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#issuecomment-1587645300

   I will review this PR tomorrow, thank you for your contribution!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] github-actions[bot] commented on pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#issuecomment-1718622211

   The pr had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1230735213


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   `DEFAULT` secure random's default configuration is prediction resistant, so better than `NONCEANDIV`- what is not prediction resistant (I have debugged it: the prediction resistance property is reported by the objects.)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] nodece commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1230644164


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   > Could you please elaborate on what would we gain from using `NONCEANDIV` for everything?
   
   Just to use the same mode.
   
   > On the other hand `DEFAULT` is configurable, so I assume if someone wants to use BC FIPS (as it's a very specific industry-related use-case), they should know about how to configure it in case they experience issues.
   see: https://www.bouncycastle.org/fips-java/BCFipsIn100.pdf page 13
   
   As you said, is using `DEFAULT` better? Because it is configurable.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] nodece commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1230766854


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   For the current implementation, it is correct and has a better performance and security.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1229277165


##########
bouncy-castle/bcfips/pom.xml:
##########
@@ -51,6 +58,12 @@
       <artifactId>bcpkix-fips</artifactId>
       <version>${bouncycastle.bcpkix-fips.version}</version>
     </dependency>
+    <dependency>
+      <groupId>${project.groupId}</groupId>

Review Comment:
   stupid mistake, fixing



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1229305412


##########
bouncy-castle/bc-common/src/test/java/org/apache/pulsar/client/impl/crypto/WrappingVersusEncryptionCrossCompatibilityTestBase.java:
##########
@@ -0,0 +1,236 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl.crypto;
+
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.testng.AssertJUnit.assertEquals;
+import com.google.common.collect.ImmutableMap;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+import java.security.NoSuchAlgorithmException;
+import java.security.NoSuchProviderException;
+import java.security.PrivateKey;
+import java.security.PublicKey;
+import java.security.spec.InvalidKeySpecException;
+import java.util.Optional;
+import javax.crypto.KeyGenerator;
+import javax.crypto.SecretKey;
+import javax.crypto.SecretKeyFactory;
+import javax.crypto.spec.SecretKeySpec;
+import org.apache.commons.io.FileUtils;
+import org.apache.pulsar.client.api.PulsarClientException;
+import org.apache.pulsar.common.util.SecurityUtility;
+import org.hamcrest.Matcher;
+import org.testng.annotations.DataProvider;
+import org.testng.annotations.Test;
+
+public abstract class WrappingVersusEncryptionCrossCompatibilityTestBase {
+
+    //from https://stackoverflow.com/a/140861
+    //TODO use java.format.HexFormat after we move to java17 language features
+    protected static byte[] hexStringToByteArray(String s/* s must be an even-length string. */) {

Review Comment:
   good shout, fixed



##########
bouncy-castle/bc-common/src/main/java/org/apache/pulsar/client/impl/crypto/BcVersionSpecificCryptoUtilityFactory.java:
##########
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl.crypto;
+
+import java.lang.reflect.InvocationTargetException;
+import org.apache.pulsar.common.util.SecurityUtility;
+
+class BcVersionSpecificCryptoUtilityFactory {
+
+    static BcVersionSpecificCryptoUtility createNewInstance() {
+        String providerName = SecurityUtility.getProvider().getName();
+        try {
+            switch (providerName) {
+                case SecurityUtility.BC:
+                    return (BcVersionSpecificCryptoUtility) BcVersionSpecificCryptoUtility.class.getClassLoader()
+                            .loadClass("org.apache.pulsar.client.impl.crypto.bc.BCNonFipsSpecificUtility")
+                            .getConstructor()
+                            .newInstance();
+                case SecurityUtility.BC_FIPS:
+                    return (BcVersionSpecificCryptoUtility) BcVersionSpecificCryptoUtility.class.getClassLoader()
+                            .loadClass("org.apache.pulsar.client.impl.bcfips.BCFipsSpecificUtility")
+                            .getConstructor().newInstance();
+                default:
+                    throw new IllegalStateException(
+                            "Provider not handled for BC Specific delegate creation: " + providerName);
+            }
+        } catch (InstantiationException | IllegalAccessException | InvocationTargetException | NoSuchMethodException

Review Comment:
   good shout, fixed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#issuecomment-1584152628

   @david-streamlio @jiazhai @sijie @nodece @michaeljmarshall (apologies if I missed anyone, I tried to add everyone on various BC FIPS threads) - please have a look on the proposed change, let me know if you have any suggestions
   
   the CI ran in my fork successfully apart from code coverge upload steps - let me know if that is OK - this is my fist PR
   
   many thanks in advance!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1230761098


##########
tiered-storage/jcloud/pom.xml:
##########
@@ -103,7 +103,12 @@
       <version>${jclouds.version}</version>
       <scope>provided</scope>
     </dependency>
-
+    <dependency>

Review Comment:
   I had to add this some some tests are running - pls confirm this is needed!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1230637152


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   > Could we use the NONCEANDIV for generating ASE and sending messages?
   
   We can, but I would advise against it unless an expert says otherwise, as it is not for that based on the documentation. Generally for keys you want a higher-grade random generator (thus `DEFAULT` is prediction resistant). 
   A comment referring to this:
   https://github.com/indrabasak/bouncycastle-fips-examples/blob/master/src/main/java/com/basaki/bc/fips/random/SecureRandomNumberExample.java#L86-L87
   
   Could you please elaborate on what would we gain from using `NONCEANDIV` for everything?
   
   On the other hand `DEFAULT` is configurable, so I assume if someone wants to use BC FIPS (as it's a very specific industry-related use-case), they should know about how to configure it in case they experience issues.
   see: https://www.bouncycastle.org/fips-java/BCFipsIn100.pdf page 13



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] nodece commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1228906227


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   > Could you please elaborate on >if "NONCEANDIV" does not work<?
   
   This is an assumption. When "NONCEANDIV" is not available, we can fallback to "DEFAULT", If it is still not available we throw an error that BC-FIPS is not available.
   
   > DEFAULT is the prediction resistant one - so the safest (I think to generate AES keys), and NONCEANDIV is for nonce and IV generation, exactly what we use that random generator when sending messages. 
   
   Sounds good, you use the different secure-random for generating ASE and sending messages.
   
   I'm not an expert in BC, so I'm not sure which works better, when I don't know which one to use, I prefer to use "DEFAULT".
   
   
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1229313987


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   do we have somebody BC expert we can ask to opine? Would be interesting to know if that is possible that `NONCEANDIV` is not available in BC FIPS, and if so is `DEFAULT` a good fallback (i.e. would not be a good chance that that would not be available either).
   
   btw I think your approach absolutely makes sense regarding the choice of securerandoms and I believe that is what I implemented in spirit: for IV generation we use `NONCEANDIV` - as in the doc and the name suggests it's for IV and Nonce generation, and DEFAULT for everything else (AES key generation)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] tisonkun commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
tisonkun commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1227770733


##########
bouncy-castle/bc-common/LICENSE:
##########


Review Comment:
   We can need a technical review before.
   
   But I'm wondering why we need to carry bc's Apache License. IIRC we distribute it already when packaging the distribution. And if we don't use its source code we don't need to mention it in source form. Also copy the content of Apache License is redundant.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1229276849


##########
bouncy-castle/bc-common/src/test/java/org/apache/pulsar/client/impl/crypto/EncKeyReader.java:
##########
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl.crypto;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Map;
+import java.util.Optional;
+import org.apache.pulsar.client.api.CryptoKeyReader;
+import org.apache.pulsar.client.api.EncryptionKeyInfo;
+import org.testng.Assert;
+
+public class EncKeyReader implements CryptoKeyReader {
+
+    public enum KeyType {
+        PUBLIC("public"), PRIVATE("private");
+        private final String id;
+
+        KeyType(String id) {
+            this.id = id;
+        }
+    }
+
+    public static byte[] getKeyAsPEM(KeyType keyType, String keyName, Map<String, String> keyMeta) {
+        String CERT_FILE_PATH = "../src/test/resources/certificate/" + keyType.id + "-key." + keyName;
+        if (Files.isReadable(Paths.get(CERT_FILE_PATH))) {

Review Comment:
   indeed I can drop the `keyInfo` creation



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1228232007


##########
bouncy-castle/bc-common/LICENSE:
##########


Review Comment:
   I just copied it because I saw it in `bouncy-castle/bc/` I am happy to remove it if you think that is the right way.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1229554464


##########
bouncy-castle/bc-common/LICENSE:
##########


Review Comment:
   removed!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1229305769


##########
bouncy-castle/bc-common/src/main/java/org/apache/pulsar/client/impl/crypto/BcVersionSpecificCryptoUtility.java:
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl.crypto;
+
+import static org.apache.pulsar.client.impl.crypto.BcVersionSpecificCryptoUtilityFactory.createNewInstance;
+import java.security.PrivateKey;
+import java.security.PublicKey;
+import java.util.Optional;
+import javax.crypto.SecretKey;
+import org.apache.pulsar.client.api.PulsarClientException;
+
+/**
+ * Facade to various crypto functions implemented by BouncyCastle FIPS and Non-fips module.
+ */
+public interface BcVersionSpecificCryptoUtility {
+
+    BcVersionSpecificCryptoUtility INSTANCE = createNewInstance();
+
+    String RSA = "RSA";
+
+    // Ideally the transformation should also be part of the message property. This will prevent client
+    // from assuming hardcoded value. However, it will increase the size of the message even further.
+    String RSA_TRANS = "RSA/NONE/OAEPWithSHA1AndMGF1Padding";
+
+
+    byte[] encryptDataKey(String logCtx, String keyName, PublicKey publicKey, SecretKey dataKey)
+            throws PulsarClientException.CryptoException;
+
+    Optional<SecretKey> deCryptDataKey(String datakeyAlgorithm, String logCtx, String keyName, PrivateKey privateKey,
+                                       byte[] encryotedDataKey);

Review Comment:
   fixed



##########
bouncy-castle/bc-common/src/test/java/org/apache/pulsar/client/impl/crypto/WrappingVersusEncryptionCrossCompatibilityTestBase.java:
##########
@@ -0,0 +1,236 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl.crypto;
+
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.testng.AssertJUnit.assertEquals;
+import com.google.common.collect.ImmutableMap;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+import java.security.NoSuchAlgorithmException;
+import java.security.NoSuchProviderException;
+import java.security.PrivateKey;
+import java.security.PublicKey;
+import java.security.spec.InvalidKeySpecException;
+import java.util.Optional;
+import javax.crypto.KeyGenerator;
+import javax.crypto.SecretKey;
+import javax.crypto.SecretKeyFactory;
+import javax.crypto.spec.SecretKeySpec;
+import org.apache.commons.io.FileUtils;
+import org.apache.pulsar.client.api.PulsarClientException;
+import org.apache.pulsar.common.util.SecurityUtility;
+import org.hamcrest.Matcher;
+import org.testng.annotations.DataProvider;
+import org.testng.annotations.Test;
+
+public abstract class WrappingVersusEncryptionCrossCompatibilityTestBase {
+
+    //from https://stackoverflow.com/a/140861
+    //TODO use java.format.HexFormat after we move to java17 language features
+    protected static byte[] hexStringToByteArray(String s/* s must be an even-length string. */) {
+        int len = s.length();
+        byte[] data = new byte[len / 2];
+        for (int i = 0; i < len; i += 2) {
+            data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
+                    + Character.digit(s.charAt(i + 1), 16));
+        }
+        return data;
+    }
+
+    //from https://stackoverflow.com/a/9855338
+    private static final byte[] HEX_ARRAY = "0123456789ABCDEF".getBytes(StandardCharsets.US_ASCII);
+
+    //TODO use java.format.HexFormat after we move to java17 language features
+    protected static String bytesToHex(byte[] bytes) {
+        byte[] hexChars = new byte[bytes.length * 2];

Review Comment:
   good shout, fixed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1254139528


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   @nodece - I was thinking and this PR is more about fixing a feature what should be there, not sure if it warrants a PIP, and I can just move the new module`bouncy-castle-common` 's content into `pulsar-common` and thus no new module will be required



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] z-kovacs commented on pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "z-kovacs (via GitHub)" <gi...@apache.org>.
z-kovacs commented on PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#issuecomment-1637927685

   @nodece had a suggestion offline how to refactor this:
   - Remove the `bc-common` library
   - Create a new `messagecrypto-bcfips` library what would use `messagecrypto-bc` as dependency and implement another `MessageCrypto` implementation purely using BC FIPS inside, in the dependency we would exclude the bc-non-fips libs (shared code would stay in `messagecrypto-bc`)
   - Clients have to explicitly modify their client code to use the `MessageCryptoBcFips` implementation
   
   my argument is this is suboptimal because:
   - people have to change their code if they want to use bc fips - it might not be possible (3rd party application), also it would complicate their client configuration
   - if you reference the module `messagecrypto-bc` as dependency from `messagecrypto-bcfips`,  it might contain code what was compile-able in `messagecrypto-bc` but not runnable from `messagecrypto-bcfips` and thus would cause runtime issues, this is what happened with the bcfips iplementation (`ClassNotFoundException`)
   
   An another idea is of removing the `bc-common` library and moving the crypto api interface and the reflection-based loader into `pulsar-common` this way you don't create a new module (which has other implications), but still you have the isolation of the bcfips and non-fips related implementations, which is giving compile-time and runtime safety.
   Also this way this is entirely transparent for the application, if you upgrade the pulsar libraries, it will became FIPS capable - depending on how the library dependencies re set up - in-line with the current setup.
   
   Please comment if you have preference towards either of the ways, or you are just happy with the way it is currently.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] nodece commented on a diff in pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#discussion_r1230644164


##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   > Could you please elaborate on what would we gain from using `NONCEANDIV` for everything?
   Just to use the same mode.



##########
pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java:
##########
@@ -106,28 +80,64 @@ public class MessageCryptoBc implements MessageCrypto<MessageMetadata, MessageMe
 
 
     private static final SecureRandom secureRandom;
+
+    private static final String providerName;
+    private static BcVersionSpecificCryptoUtility bcVersionSpecificCryptoUtilityDelegate;
+
     static {
-        SecureRandom rand = null;
-        try {
-            rand = SecureRandom.getInstance("NativePRNGNonBlocking");
-        } catch (NoSuchAlgorithmException nsa) {
-            rand = new SecureRandom();
-        }
 
-        secureRandom = rand;
+        providerName = SecurityUtility.getProvider().getName();
+
+        switch (providerName) {
+            case SecurityUtility.BC:
+                SecureRandom rand = null;
+                try {
+                    rand = SecureRandom.getInstance("NativePRNGNonBlocking", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    rand = new SecureRandom();
+                }
+                secureRandom = rand;
+                break;
+            case SecurityUtility.BC_FIPS:
+                try {
+                    secureRandom = SecureRandom.getInstance("NONCEANDIV", providerName);
+                } catch (NoSuchAlgorithmException | NoSuchProviderException nsa) {
+                    throw new RuntimeException(
+                            "In BC FIPS mode, we expect the specific Random generators to be available!");
+                }
+                break;

Review Comment:
   > Could you please elaborate on what would we gain from using `NONCEANDIV` for everything?
   
   Just to use the same mode.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][broker, client] Fixing BC FIPS based payload encryption [pulsar]

Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
tisonkun closed pull request #20542: [improve][broker, client] Fixing BC FIPS based payload encryption
URL: https://github.com/apache/pulsar/pull/20542


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][broker, client] Fixing BC FIPS based payload encryption [pulsar]

Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
tisonkun commented on PR #20542:
URL: https://github.com/apache/pulsar/pull/20542#issuecomment-2100120153

   Closed as stale and conlfict.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org