[jira] [Created] (NIFI-1831) Allow encrypted passwords in configuration files

["Andy LoPresto (JIRA)" <ji...@apache.org>]

Andy LoPresto created NIFI-1831:
-----------------------------------

             Summary: Allow encrypted passwords in configuration files
                 Key: NIFI-1831
                 URL: https://issues.apache.org/jira/browse/NIFI-1831
             Project: Apache NiFi
          Issue Type: New Feature
          Components: Configuration, Core Framework
    Affects Versions: 0.6.1
            Reporter: Andy LoPresto
            Assignee: Andy LoPresto
            Priority: Critical
             Fix For: 1.0.0


Storing passwords in plaintext in configuration files is not a security best practice. While file access can be restricted through OS permissions, these configuration files can be accidentally checked into source control, shared or deployed to multiple instances, etc. 

NiFi should allow a deployer to provide an encrypted password in the configuration file to minimize exposure of the passwords. On application start-up, NiFi should decrypt the passwords in memory. NiFi should also include a utility to encrypt the raw passwords (and optionally populate the configuration files and provide additional metadata in the configuration files). 

I am aware this simply shifts the responsibility/delegation of trust from the passwords in the properties file to a new location on the same system, but mitigating the visibility of the raw passwords in the properties file can be one step in a defense in depth approach and is often mandated by security policies within organizations using NiFi. 

The key used for encryption should not be hard-coded into the application source code, nor should it be universally consistent. The key could be determined by reading static information from the deployed system and feeding it to a key derivation function based on a cryptographically-secure hash function, such as PBKDF2, bcrypt, or scrypt. However, this does introduce upgrade, system migration, and portability issues. These challenges will have to be kept in consideration when determining the key derivation process. 

Manual key entry is a possibility, and then the master key would only be present in memory, but this prevents automatic reboot on loss of power or other recovery scenario. 

This must be backward-compatible to allow systems with plaintext passwords to continue operating. Options for achieving this are to only attempt to decrypt passwords when a sibling property is present, or to match a specific format. 

For these examples, I have used the following default values:

{code}
password: thisIsABadPassword
key: 0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210
iv:  0123456789ABCDEFFEDCBA9876543210
algorithm: AES/CBC 256-bit
{code}

**Note: These values should not be used in production systems -- the key and IV are common test values, and an AEAD cipher is preferable to provide cipher text integrity assurances, however OpenSSL does not support the use of AEAD ciphers for command-line encryption at this time**

Example 1: *here the sibling property indicates the password is encrypted and with which implementation; the absence of the property would default to a raw password*

{code}
hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) alopresto
🔓 0s @ 16:25:56 $ echo "thisIsABadPassword" > password.txt
hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) alopresto
🔓 0s @ 16:26:47 $ ossl aes-256-cbc -e -nosalt -p -K 0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210 -iv 0123456789ABCDEFFEDCBA9876543210 -a -in password.txt -out password.enc
key=0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210
iv =0123456789ABCDEFFEDCBA9876543210
hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) alopresto
🔓 0s @ 16:27:09 $ xxd password.enc
0000000: 5643 5856 6146 6250 4158 364f 5743 7646  VCXVaFbPAX6OWCvF
0000010: 6963 6b76 4a63 7744 3854 6b67 3731 4c76  ickvJcwD8Tkg71Lv
0000020: 4d38 6d32 7952 4776 5739 413d 0a         M8m2yRGvW9A=.
hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) alopresto
🔓 0s @ 16:27:16 $ more password.enc
VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A=
hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) alopresto
🔓 0s @ 16:27:55 $
{code}

In {{nifi.properties}}: 
{code}
nifi.security.keystorePasswd=VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A=
nifi.security.keystorePasswd.encrypted=AES-CBC-256
{code}

Example 2: *here the encrypted password has a header tag indicating both that it is encrypted and the algorithm used*

{code:java}
    @Test
    public void testShouldDecryptPassword() throws Exception {
        // Arrange
        KeyedCipherProvider cipherProvider = new AESKeyedCipherProvider()

        final String PLAINTEXT = "thisIsABadPassword"
        logger.info("Expected: ${Hex.encodeHexString(PLAINTEXT.bytes)}")

        final byte[] IV = Hex.decodeHex("0123456789ABCDEFFEDCBA9876543210" as char[])
        final byte[] LOCAL_KEY = Hex.decodeHex("0123456789ABCDEFFEDCBA9876543210" * 2 as char[])
        // Generated via openssl enc -a
        final String CIPHER_TEXT = "VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A="
        byte[] cipherBytes = Base64.decoder.decode(CIPHER_TEXT)

        SecretKey localKey = new SecretKeySpec(LOCAL_KEY, "AES")

        EncryptionMethod encryptionMethod = EncryptionMethod.AES_CBC
        logger.info("Using algorithm: ${encryptionMethod.getAlgorithm()}")
        logger.info("Cipher text: \$nifipw\$${CIPHER_TEXT} ${cipherBytes.length + 8}")

        // Act
        Cipher cipher = cipherProvider.getCipher(encryptionMethod, localKey, IV, false)
        byte[] recoveredBytes = cipher.doFinal(cipherBytes)
        
        // OpenSSL adds a newline character during encryption
        String recovered = new String(recoveredBytes, "UTF-8").trim()
        logger.info("Recovered: ${recovered} ${Hex.encodeHexString(recoveredBytes)}")

        // Assert
        assert PLAINTEXT.equals(recovered)
    }
{code}

In {{nifi.properties}}: 
{code}
nifi.security.keystorePasswd=$nifipw$VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A=
{code}

Ideally, NiFi would use a pluggable implementation architecture to allow users to integrate with a variety of secret management services. There are both commercial and open source solutions, including CyberArk Enterprise Password Vault [1], Hashicorp Vault [2], and Square Keywhiz [3]. In the future, this could also be extended to Hardware Security Modules (HSM) like SafeNet Luna [4] and Amazon CloudHSM [5]. 

[1] http://www.cyberark.com/products/privileged-account-security-solution/enterprise-password-vault/
[2] https://www.vaultproject.io/
[3] https://square.github.io/keywhiz/
[4] http://www.safenet-inc.com/data-encryption/hardware-security-modules-hsms/luna-hsms-key-management/luna-sa-network-hsm/
[5] https://aws.amazon.com/cloudhsm/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)