Entity Key Store

[David Vediner <ve...@bctonline.com>]

Hi everyone -

Upon reviewing the entity engine encryption code it looks like the keys 
for entity encryption are stored unencrypted in the database.  Because 
of this, I can't really see the purpose of doing any encryption at all, 
other than providing a minor layer of abstraction for a potential 
hacker.  Obviously, if a hacker has gotten access to your database your 
data is compromised, so securing your database is the most important.  
However, it seems to me that the point of encrypting data in the 
database (since it is decrypted the instant it hits the entity engine) 
is to protect that data from unauthorized sources, even if they have 
access to your database (like your DBA, who really shouldn't be able to 
read encrypted data).  I'm having a hard time seeing how the encryption 
as it's implemented provides any security at all (say, of credit card 
numbers) since if someone has gotten access to the encrypted data, they 
can probably get access to the keys in the entity_key_store table (since 
the data is only encrypted in the database, not in the application 
layer).  In other words, if they have a need for the key that means they 
have encrypted data, and if the data they have is encrypted, it seems 
likely that they have somehow gotten access to the the database.

Of course the framework can only do so much - the people implementing 
ofbiz really have a responsibility to secure their installation, and 
anyone can take a very secure application and make it not so by their 
configuration or implementation of it.  However, I think the system as 
it is gives the *illusion* of security through encryption, which seems 
even more dangerous than not providing encryption in the first place.

As I see it there are three options (I'm sure there are more) for 
changing this in the code - 1) either move the key store to the 
application layer, 2) re-encrypt the actual data so it would be 
decrypted/encrypted once using the existing method and once using a new 
key stored in the application layer (just double-encrypt everything), or 
3) encrypt the keys in entity_key_store using a new key stored somewhere 
in the application layer.

#1 has the "benefit" (perhaps from a cleanliness standpoint) of being 
conceptually simple - you have data that is encrypted in the database 
and you have a plaintext keys somewhere (now outside of the data layer) 
to decrypt it.  No double encrypting.  #2 and #3 involve some sort of 
double encryption; #2 just encrypts each piece of data twice using 
different keys and #3 encrypts the encryption key itself.  I have 
implemented #3 for our own code because it meant we only had to 
re-encrypt the handful of encryption keys in the database, rather than 
re-encrypt all the encrypted data in the database.  I didn't like #1 
because I didn't want to think about where the best place to store the 
keys are (and felt, apparently, that it was easier to figure out where 
to stick a single encryption key).

So I changed the entity engine (when it needs to pull a key from the key 
store) to decrypt that key first using a new plaintext key stored in the 
entityengine.xml configuration file.  It uses AES (Rijndael) 256-bit 
encryption using a simple AesCrypt utility class I created that's 
basically a clone of the DesCrypt utility class, and the plaintext key 
is run through a hash so a you can't just use the value in 
entityengine.xml.  It's a really simple addition to the entity engine, 
which was my goal, and an attacker can't compromise the encrypted data 
by just gaining access to the database.  I'd be happy to dot the I's and 
cross the T's in what little code I've written and submit it as a patch.

Also, perhaps someone who has actual experience with cryptanalysis 
(particularly with the 3DES and Rijndael algorithms) can answer this 
question: is there any practically useful extra information for 
brute-forcing a 3DES encryption when you have a Rijndael-encrypted 
version of the key?  That is, is it appreciably easier to crack some 
3DES-encrypted data when you have a the key in encrypted form vs. when 
you don't have any form of the key?  I doubt it, but really don't know.  
If it is easier, than the #3 method above is cryptographically inferior 
to #2 and #1, although it might still be overall more secure than #1.

However, I'm open to the possibility that I haven't thought this through 
completely enough and a) there is a better solution (likely) or b) I'm 
just spinning my wheels here and the system as it is is actually secure 
(although I know that it is not secure enough for our purposes, but I 
admit that this might have to do with how we use our database).

Regards,
David Vediner

Re: Entity Key Store

[David Vediner <ve...@bctonline.com>]

Hi -

Good points.  I don't have to (not yet anyway) consider a server cluster 
so I can be selfish and not worry about multiple machines.  And you're 
right about the DB server being on an isolated network, of course, but 
in reality most small installations can't/don't do that.  That may or 
may not be an issue OFBiz should consider, though, I don't know.  And 
yeah, you've got to trust the people you work with, too, because at some 
point they'll probably need access to sensitive data.  And no matter how 
hard you hide your encryption keys at least your application needs to 
get a hold of them at some point or they're not very useful.  If not, 
that's a pretty good indicator that you should have used a one-way hash, 
or not stored the data in the first place.  :)

I guess my primary concern with the encryption as it is is that I don't 
see what security it provides.  Worse than that, if it doesn't really 
provide security it is at least providing the illusion of security to 
those less diligent than they should be.  If someone gets into your 
database, the encryption will not do anything to stop them, and yet the 
encryption isn't a factor _unless_ you're in the database.. seems like a 
catch-22.  So why encrypt anything?  Maybe I'm missing something 
important here, or maybe the purpose of the encryption wasn't to provide 
security in the first place and I'm reading a purpose into it that 
wasn't ever intended?

Perhaps this isn't really a flaw in the entity engine design, however, 
and it just needs to be emphasized in some future iteration of the setup 
guide that unless you act otherwise the keys are stored in plaintext 
alongside the encrypted data, and that it's a best practice to do 
something like you suggest and use the entity engine to store them 
somewhere else using a different delegator.  Or at least be aware that 
if you don't do something about it, you should not rely on the 
encryption to provide any layer of security.  Perhaps it's already in 
there and I've just missed it.

I'm sure we're agreed that securing your database is much more important 
than encrypting your stored data.  No one can crack the encryption if 
they don't have the data in the first place, and it's not just the 
encrypted data you're trying to protect, but generally all your data.

Good point about using a separate delegator...  I hadn't considered 
that.  That might be a cleaner option and I'll look into it for our own 
purposes (and using different delegators should solve another issue I'm 
having with DB backups, actually), although unless I think that what 
I've done with the keys is actually less secure I'll probably keep 
encrypting the keys in the fashion I'm doing now.  But maybe that's how 
it should be - each user should evaluate this key situation and see if 
it works for their needs, and if not do something like I'm doing.

Thanks,
David Vediner



David E. Jones wrote:
>
> A couple of things to consider:
>
> 1. the initial decision to put the keys in the database was made 
> because it avoids problems with keeping keys updated and consistent on 
> multiple servers, and the database is typically more secure than the 
> application servers (the database should never be accessible from the 
> outside or external network, and so attacks often come through 
> compromised web or application servers
>
> 2. it's tough to stop a resourceful DBA no matter where you put the 
> keys; putting keys on the app server can keep them away from the DBAs 
> if that is your primary concern, but you can also just put them in a 
> separate database that the main DBA group does not have any access to 
> ()this is fairly trivial with the delegator and such; anyone who has 
> access to the app servers of course must necessarily have access to db 
> access information plus the key locations, so it is impossible to 
> avoid having someone in the organization with access to these things
>
> 3. for the database as well as the applications no employee should 
> _ever_ access it with a generic account, they should always have and 
> only use their own username and password and have explicit permissions 
> setup as needed; in fact this is required by the Visa/MasterCard CC 
> security standard group
>
> -David
>
>
>
> David Vediner wrote:
>> Hi everyone -
>>
>> Upon reviewing the entity engine encryption code it looks like the 
>> keys for entity encryption are stored unencrypted in the database.  
>> Because of this, I can't really see the purpose of doing any 
>> encryption at all, other than providing a minor layer of abstraction 
>> for a potential hacker.  Obviously, if a hacker has gotten access to 
>> your database your data is compromised, so securing your database is 
>> the most important.  However, it seems to me that the point of 
>> encrypting data in the database (since it is decrypted the instant it 
>> hits the entity engine) is to protect that data from unauthorized 
>> sources, even if they have access to your database (like your DBA, 
>> who really shouldn't be able to read encrypted data).  I'm having a 
>> hard time seeing how the encryption as it's implemented provides any 
>> security at all (say, of credit card numbers) since if someone has 
>> gotten access to the encrypted data, they can probably get access to 
>> the keys in the entity_key_store table (since the data is only 
>> encrypted in the database, not in the application layer).  In other 
>> words, if they have a need for the key that means they have encrypted 
>> data, and if the data they have is encrypted, it seems likely that 
>> they have somehow gotten access to the the database.
>>
>> Of course the framework can only do so much - the people implementing 
>> ofbiz really have a responsibility to secure their installation, and 
>> anyone can take a very secure application and make it not so by their 
>> configuration or implementation of it.  However, I think the system 
>> as it is gives the *illusion* of security through encryption, which 
>> seems even more dangerous than not providing encryption in the first 
>> place.
>>
>> As I see it there are three options (I'm sure there are more) for 
>> changing this in the code - 1) either move the key store to the 
>> application layer, 2) re-encrypt the actual data so it would be 
>> decrypted/encrypted once using the existing method and once using a 
>> new key stored in the application layer (just double-encrypt 
>> everything), or 3) encrypt the keys in entity_key_store using a new 
>> key stored somewhere in the application layer.
>>
>> #1 has the "benefit" (perhaps from a cleanliness standpoint) of being 
>> conceptually simple - you have data that is encrypted in the database 
>> and you have a plaintext keys somewhere (now outside of the data 
>> layer) to decrypt it.  No double encrypting.  #2 and #3 involve some 
>> sort of double encryption; #2 just encrypts each piece of data twice 
>> using different keys and #3 encrypts the encryption key itself.  I 
>> have implemented #3 for our own code because it meant we only had to 
>> re-encrypt the handful of encryption keys in the database, rather 
>> than re-encrypt all the encrypted data in the database.  I didn't 
>> like #1 because I didn't want to think about where the best place to 
>> store the keys are (and felt, apparently, that it was easier to 
>> figure out where to stick a single encryption key).
>>
>> So I changed the entity engine (when it needs to pull a key from the 
>> key store) to decrypt that key first using a new plaintext key stored 
>> in the entityengine.xml configuration file.  It uses AES (Rijndael) 
>> 256-bit encryption using a simple AesCrypt utility class I created 
>> that's basically a clone of the DesCrypt utility class, and the 
>> plaintext key is run through a hash so a you can't just use the value 
>> in entityengine.xml.  It's a really simple addition to the entity 
>> engine, which was my goal, and an attacker can't compromise the 
>> encrypted data by just gaining access to the database.  I'd be happy 
>> to dot the I's and cross the T's in what little code I've written and 
>> submit it as a patch.
>>
>> Also, perhaps someone who has actual experience with cryptanalysis 
>> (particularly with the 3DES and Rijndael algorithms) can answer this 
>> question: is there any practically useful extra information for 
>> brute-forcing a 3DES encryption when you have a Rijndael-encrypted 
>> version of the key?  That is, is it appreciably easier to crack some 
>> 3DES-encrypted data when you have a the key in encrypted form vs. 
>> when you don't have any form of the key?  I doubt it, but really 
>> don't know.  If it is easier, than the #3 method above is 
>> cryptographically inferior to #2 and #1, although it might still be 
>> overall more secure than #1.
>>
>> However, I'm open to the possibility that I haven't thought this 
>> through completely enough and a) there is a better solution (likely) 
>> or b) I'm just spinning my wheels here and the system as it is is 
>> actually secure (although I know that it is not secure enough for our 
>> purposes, but I admit that this might have to do with how we use our 
>> database).
>>
>> Regards,
>> David Vediner
>>
>>

Re: Entity Key Store

["David E. Jones" <jo...@undersunconsulting.com>]

A couple of things to consider:

1. the initial decision to put the keys in the database was made because it avoids problems with keeping keys updated and consistent on multiple servers, and the database is typically more secure than the application servers (the database should never be accessible from the outside or external network, and so attacks often come through compromised web or application servers

2. it's tough to stop a resourceful DBA no matter where you put the keys; putting keys on the app server can keep them away from the DBAs if that is your primary concern, but you can also just put them in a separate database that the main DBA group does not have any access to ()this is fairly trivial with the delegator and such; anyone who has access to the app servers of course must necessarily have access to db access information plus the key locations, so it is impossible to avoid having someone in the organization with access to these things

3. for the database as well as the applications no employee should _ever_ access it with a generic account, they should always have and only use their own username and password and have explicit permissions setup as needed; in fact this is required by the Visa/MasterCard CC security standard group

-David



David Vediner wrote:
> Hi everyone -
> 
> Upon reviewing the entity engine encryption code it looks like the keys 
> for entity encryption are stored unencrypted in the database.  Because 
> of this, I can't really see the purpose of doing any encryption at all, 
> other than providing a minor layer of abstraction for a potential 
> hacker.  Obviously, if a hacker has gotten access to your database your 
> data is compromised, so securing your database is the most important.  
> However, it seems to me that the point of encrypting data in the 
> database (since it is decrypted the instant it hits the entity engine) 
> is to protect that data from unauthorized sources, even if they have 
> access to your database (like your DBA, who really shouldn't be able to 
> read encrypted data).  I'm having a hard time seeing how the encryption 
> as it's implemented provides any security at all (say, of credit card 
> numbers) since if someone has gotten access to the encrypted data, they 
> can probably get access to the keys in the entity_key_store table (since 
> the data is only encrypted in the database, not in the application 
> layer).  In other words, if they have a need for the key that means they 
> have encrypted data, and if the data they have is encrypted, it seems 
> likely that they have somehow gotten access to the the database.
> 
> Of course the framework can only do so much - the people implementing 
> ofbiz really have a responsibility to secure their installation, and 
> anyone can take a very secure application and make it not so by their 
> configuration or implementation of it.  However, I think the system as 
> it is gives the *illusion* of security through encryption, which seems 
> even more dangerous than not providing encryption in the first place.
> 
> As I see it there are three options (I'm sure there are more) for 
> changing this in the code - 1) either move the key store to the 
> application layer, 2) re-encrypt the actual data so it would be 
> decrypted/encrypted once using the existing method and once using a new 
> key stored in the application layer (just double-encrypt everything), or 
> 3) encrypt the keys in entity_key_store using a new key stored somewhere 
> in the application layer.
> 
> #1 has the "benefit" (perhaps from a cleanliness standpoint) of being 
> conceptually simple - you have data that is encrypted in the database 
> and you have a plaintext keys somewhere (now outside of the data layer) 
> to decrypt it.  No double encrypting.  #2 and #3 involve some sort of 
> double encryption; #2 just encrypts each piece of data twice using 
> different keys and #3 encrypts the encryption key itself.  I have 
> implemented #3 for our own code because it meant we only had to 
> re-encrypt the handful of encryption keys in the database, rather than 
> re-encrypt all the encrypted data in the database.  I didn't like #1 
> because I didn't want to think about where the best place to store the 
> keys are (and felt, apparently, that it was easier to figure out where 
> to stick a single encryption key).
> 
> So I changed the entity engine (when it needs to pull a key from the key 
> store) to decrypt that key first using a new plaintext key stored in the 
> entityengine.xml configuration file.  It uses AES (Rijndael) 256-bit 
> encryption using a simple AesCrypt utility class I created that's 
> basically a clone of the DesCrypt utility class, and the plaintext key 
> is run through a hash so a you can't just use the value in 
> entityengine.xml.  It's a really simple addition to the entity engine, 
> which was my goal, and an attacker can't compromise the encrypted data 
> by just gaining access to the database.  I'd be happy to dot the I's and 
> cross the T's in what little code I've written and submit it as a patch.
> 
> Also, perhaps someone who has actual experience with cryptanalysis 
> (particularly with the 3DES and Rijndael algorithms) can answer this 
> question: is there any practically useful extra information for 
> brute-forcing a 3DES encryption when you have a Rijndael-encrypted 
> version of the key?  That is, is it appreciably easier to crack some 
> 3DES-encrypted data when you have a the key in encrypted form vs. when 
> you don't have any form of the key?  I doubt it, but really don't know.  
> If it is easier, than the #3 method above is cryptographically inferior 
> to #2 and #1, although it might still be overall more secure than #1.
> 
> However, I'm open to the possibility that I haven't thought this through 
> completely enough and a) there is a better solution (likely) or b) I'm 
> just spinning my wheels here and the system as it is is actually secure 
> (although I know that it is not secure enough for our purposes, but I 
> admit that this might have to do with how we use our database).
> 
> Regards,
> David Vediner
> 
>