You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Ahuja, Sakshi" <ah...@upmc.edu> on 2019/06/25 15:32:05 UTC
Encrypting Solr Index
Hi,
I am using solr 6.6 and want to encrypt index for security reasons. I have tried Windows FS encryption option that works but want to know if solr has some inbuilt feature to encrypt index or any good way to encrypt solr index?
Thanks,
Sakshi
Re: Encrypting Solr Index
Posted by Jörn Franke <jo...@gmail.com>.
Maybe in this scenario a Secure Enclave could make sense (eg Intel sgx)?
The scenario that you describes looks like MIT CryptDB, eg https://css.csail.mit.edu/cryptdb/
> Am 25.06.2019 um 21:05 schrieb Tim Casey <tc...@gmail.com>:
>
> My two cents worth of comment,
>
> For our local lucene indexes we use AES encryption. We encrypt the blocks
> on the way out, decrypt on the way in.
> We are using a C version of lucene, not the java version. But, I suspect
> the same methodology could be applied. This assumes the data at rest is
> the attack vector for discovering what is in the invertible index. But
> allows for the indexing/querying to be done in the clear. This would allow
> for stemming and the like.
>
> If you have an attack vector in which the indexing/querying are not
> trusted, then you have a whole different set of problems.
>
> To do stemming, you need a homomorphic encryption scheme which would allow
> per character/byte queries. This is different type of attack vector than
> the on-disk encryption. To me, this implies the query system itself is
> untrusted and you are indexing/querying encrypted content. The first
> "thing" people are going to try is to hash a token into a 256bit value
> which becomes the indexable token value. This leads to the lack of
> stemming from above comments. Depending on how keys are handled and hashes
> are generated you can run out of token space in the various underlying
> lucene indexes because you have more than 2 million tokens.
>
>
>
>> On Tue, Jun 25, 2019 at 10:21 AM Ahuja, Sakshi <ah...@upmc.edu> wrote:
>>
>> I am actually looking for the best option so currently doing research on
>> it.
>> For Window's FS encryption I didn't find a way to use different
>> Username/Password. It by default takes window's username/password to
>> encrypt and decrypt.
>>
>> I tried bitlocker too for creating encrypted virtual directory (Which
>> allows me to use different credentials) and to keep Solr Index in that but
>> somehow Solr Admin was unable to access Index from that encrypted
>> directory. Not sure how that is working.
>>
>> If you have any idea on that- will wok for me. Thanks!
>>
>> -----Original Message-----
>> From: Jörn Franke <jo...@gmail.com>
>> Sent: Tuesday, June 25, 2019 12:47 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Encrypting Solr Index
>>
>> Why does FS encryption does not serve your use case?
>>
>> Can’t you apply it also for backups etc?
>>
>>> Am 25.06.2019 um 17:32 schrieb Ahuja, Sakshi <ah...@upmc.edu>:
>>>
>>> Hi,
>>>
>>> I am using solr 6.6 and want to encrypt index for security reasons. I
>> have tried Windows FS encryption option that works but want to know if solr
>> has some inbuilt feature to encrypt index or any good way to encrypt solr
>> index?
>>>
>>> Thanks,
>>> Sakshi
>>
Re: Encrypting Solr Index
Posted by Tim Casey <tc...@gmail.com>.
My two cents worth of comment,
For our local lucene indexes we use AES encryption. We encrypt the blocks
on the way out, decrypt on the way in.
We are using a C version of lucene, not the java version. But, I suspect
the same methodology could be applied. This assumes the data at rest is
the attack vector for discovering what is in the invertible index. But
allows for the indexing/querying to be done in the clear. This would allow
for stemming and the like.
If you have an attack vector in which the indexing/querying are not
trusted, then you have a whole different set of problems.
To do stemming, you need a homomorphic encryption scheme which would allow
per character/byte queries. This is different type of attack vector than
the on-disk encryption. To me, this implies the query system itself is
untrusted and you are indexing/querying encrypted content. The first
"thing" people are going to try is to hash a token into a 256bit value
which becomes the indexable token value. This leads to the lack of
stemming from above comments. Depending on how keys are handled and hashes
are generated you can run out of token space in the various underlying
lucene indexes because you have more than 2 million tokens.
On Tue, Jun 25, 2019 at 10:21 AM Ahuja, Sakshi <ah...@upmc.edu> wrote:
> I am actually looking for the best option so currently doing research on
> it.
> For Window's FS encryption I didn't find a way to use different
> Username/Password. It by default takes window's username/password to
> encrypt and decrypt.
>
> I tried bitlocker too for creating encrypted virtual directory (Which
> allows me to use different credentials) and to keep Solr Index in that but
> somehow Solr Admin was unable to access Index from that encrypted
> directory. Not sure how that is working.
>
> If you have any idea on that- will wok for me. Thanks!
>
> -----Original Message-----
> From: Jörn Franke <jo...@gmail.com>
> Sent: Tuesday, June 25, 2019 12:47 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Encrypting Solr Index
>
> Why does FS encryption does not serve your use case?
>
> Can’t you apply it also for backups etc?
>
> > Am 25.06.2019 um 17:32 schrieb Ahuja, Sakshi <ah...@upmc.edu>:
> >
> > Hi,
> >
> > I am using solr 6.6 and want to encrypt index for security reasons. I
> have tried Windows FS encryption option that works but want to know if solr
> has some inbuilt feature to encrypt index or any good way to encrypt solr
> index?
> >
> > Thanks,
> > Sakshi
>
RE: Encrypting Solr Index
Posted by "Ahuja, Sakshi" <ah...@upmc.edu>.
I am actually looking for the best option so currently doing research on it.
For Window's FS encryption I didn't find a way to use different Username/Password. It by default takes window's username/password to encrypt and decrypt.
I tried bitlocker too for creating encrypted virtual directory (Which allows me to use different credentials) and to keep Solr Index in that but somehow Solr Admin was unable to access Index from that encrypted directory. Not sure how that is working.
If you have any idea on that- will wok for me. Thanks!
-----Original Message-----
From: Jörn Franke <jo...@gmail.com>
Sent: Tuesday, June 25, 2019 12:47 PM
To: solr-user@lucene.apache.org
Subject: Re: Encrypting Solr Index
Why does FS encryption does not serve your use case?
Can’t you apply it also for backups etc?
> Am 25.06.2019 um 17:32 schrieb Ahuja, Sakshi <ah...@upmc.edu>:
>
> Hi,
>
> I am using solr 6.6 and want to encrypt index for security reasons. I have tried Windows FS encryption option that works but want to know if solr has some inbuilt feature to encrypt index or any good way to encrypt solr index?
>
> Thanks,
> Sakshi
Re: Encrypting Solr Index
Posted by Jörn Franke <jo...@gmail.com>.
Why does FS encryption does not serve your use case?
Can’t you apply it also for backups etc?
> Am 25.06.2019 um 17:32 schrieb Ahuja, Sakshi <ah...@upmc.edu>:
>
> Hi,
>
> I am using solr 6.6 and want to encrypt index for security reasons. I have tried Windows FS encryption option that works but want to know if solr has some inbuilt feature to encrypt index or any good way to encrypt solr index?
>
> Thanks,
> Sakshi
Re: Encrypting Solr Index
Posted by Erick Erickson <er...@gmail.com>.
This is a recurring issue. The Hitachi solution will encrypt individual _tokens_ in the index, even with different keys for different users. However, the price is functionality.
Take wildcards. The Hitachi solution doesn’t solve this, the problem is basically intractable. Consider the words run, running, runner, and runs. A search for run* has to match all those words, and an encryption algorithm that encodes the first three letters identically is trivially breakable.
People do as you are, put the index on an encrypting filesystim if encryption-at-rest is sufficient. My personal take is that if a hacker has unrestricted access to the memory on your Solr servers and could read the unencrypted index, Solr is only one of many problems you have.
Best,
Erick
> On Jun 25, 2019, at 8:40 AM, Alexandre Rafalovitch <ar...@gmail.com> wrote:
>
> No index encryption in the box. I am aware of a commercial solution but no
> details on how good or what the price is:
> https://www.hitachi-solutions.com/securesearch/
>
> Regards,
> Alex
>
> On Tue, Jun 25, 2019, 11:32 AM Ahuja, Sakshi, <ah...@upmc.edu> wrote:
>
>> Hi,
>>
>> I am using solr 6.6 and want to encrypt index for security reasons. I have
>> tried Windows FS encryption option that works but want to know if solr has
>> some inbuilt feature to encrypt index or any good way to encrypt solr index?
>>
>> Thanks,
>> Sakshi
>>
Re: Encrypting Solr Index
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
No index encryption in the box. I am aware of a commercial solution but no
details on how good or what the price is:
https://www.hitachi-solutions.com/securesearch/
Regards,
Alex
On Tue, Jun 25, 2019, 11:32 AM Ahuja, Sakshi, <ah...@upmc.edu> wrote:
> Hi,
>
> I am using solr 6.6 and want to encrypt index for security reasons. I have
> tried Windows FS encryption option that works but want to know if solr has
> some inbuilt feature to encrypt index or any good way to encrypt solr index?
>
> Thanks,
> Sakshi
>