native encryption for couchdb 4.0?

[Robert Samuel Newson <rn...@apache.org>]

Hi All,

Our team at IBM are working on native encryption of document content for the Cloudant service and are wondering if there'd be interest (or objection!) to this landing as a CouchDB feature?

This is only targeted at the (future) CouchDB 4.0 release which introduces FoundationDB as the persistence layer and, as stated above, currently only for document bodies.

This would be a configuration option (and presumably disabled by default).

I'll spare us all the crypto details for now (besides pointing out they've been reviewed by our in-house cryptographers and use only public algorithms and techniques in a straightforward manner).

Thoughts?

B.

Re: native encryption for couchdb 4.0?

[Robert Samuel Newson <rn...@apache.org>]

Another brief note that the encryption approach is predicated on the current approach to document body storage in fdb (briefly: the json body is converted to a binary value which is then broken up into 100KB chunks and then stored in sequential key/value pairs). The alternative strategy where documents were "exploded" or "flattened" into the individual data items within a document would not work nearly as well.

B.

> On 12 Mar 2020, at 17:35, Robert Samuel Newson <rn...@apache.org> wrote:
> 
> Hi,
> 
> Yes, platform independent, it's not custom C work, just calls into the existing crypto module.
> 
> Invisible at the API layer, it's all about the protection of data at rest within FDB.
> 
> I don't know enough about _access to answer but I think not. The whole document will need to be decrypted to access any part of it and this doesn't involve the user.
> 
> B.
> 
>> On 12 Mar 2020, at 17:17, Joan Touzet <wo...@apache.org> wrote:
>> 
>> 
>> 
>> On 2020-03-12 12:29, Robert Samuel Newson wrote:
>>> Hi All,
>>> Our team at IBM are working on native encryption of document content for the Cloudant service and are wondering if there'd be interest (or objection!) to this landing as a CouchDB feature?
>> 
>> Yes!
>> 
>>> This is only targeted at the (future) CouchDB 4.0 release which introduces FoundationDB as the persistence layer and, as stated above, currently only for document bodies.
>>> This would be a configuration option (and presumably disabled by default).
>>> I'll spare us all the crypto details for now (besides pointing out they've been reviewed by our in-house cryptographers and use only public algorithms and techniques in a straightforward manner).
>> 
>> Will the code be platform independent (or at least NIFfed in a way that supports compiling on Mac, FreeBSD, Windows?)
>> 
>> Is there any impact on our CouchDB API surface, other than enabling/disabling document encryption?
>> 
>> Is there any intersection with the _access work Jan is working on?
>> 
>>> Thoughts?
>>> B.
> 

Re: native encryption for couchdb 4.0?

[Robert Samuel Newson <rn...@apache.org>]

Hi,

Good feedback, thanks all.

B.

> On 15 Mar 2020, at 13:25, Dave Cottlehuber <dc...@skunkwerks.at> wrote:
> 
> On Thu, 12 Mar 2020, at 17:35, Robert Samuel Newson wrote:
>> Hi,
>> 
>> Yes, platform independent, it's not custom C work, just calls into the 
>> existing crypto module.
>> 
>> Invisible at the API layer, it's all about the protection of data at 
>> rest within FDB.
> 
> Hey Bob,
> 
> Quietly excited about this - I could use this already - I've had horrible
> work-arounds in the past to have encrypted .couch backups.
> 
> A+
> Dave

Re: native encryption for couchdb 4.0?

[Dave Cottlehuber <dc...@skunkwerks.at>]

On Thu, 12 Mar 2020, at 17:35, Robert Samuel Newson wrote:
> Hi,
> 
> Yes, platform independent, it's not custom C work, just calls into the 
> existing crypto module.
> 
> Invisible at the API layer, it's all about the protection of data at 
> rest within FDB.

Hey Bob,

Quietly excited about this - I could use this already - I've had horrible
work-arounds in the past to have encrypted .couch backups.

A+
Dave

Re: native encryption for couchdb 4.0?

[Robert Samuel Newson <rn...@apache.org>]

Hi,

Yes, platform independent, it's not custom C work, just calls into the existing crypto module.

Invisible at the API layer, it's all about the protection of data at rest within FDB.

I don't know enough about _access to answer but I think not. The whole document will need to be decrypted to access any part of it and this doesn't involve the user.

B.

> On 12 Mar 2020, at 17:17, Joan Touzet <wo...@apache.org> wrote:
> 
> 
> 
> On 2020-03-12 12:29, Robert Samuel Newson wrote:
>> Hi All,
>> Our team at IBM are working on native encryption of document content for the Cloudant service and are wondering if there'd be interest (or objection!) to this landing as a CouchDB feature?
> 
> Yes!
> 
>> This is only targeted at the (future) CouchDB 4.0 release which introduces FoundationDB as the persistence layer and, as stated above, currently only for document bodies.
>> This would be a configuration option (and presumably disabled by default).
>> I'll spare us all the crypto details for now (besides pointing out they've been reviewed by our in-house cryptographers and use only public algorithms and techniques in a straightforward manner).
> 
> Will the code be platform independent (or at least NIFfed in a way that supports compiling on Mac, FreeBSD, Windows?)
> 
> Is there any impact on our CouchDB API surface, other than enabling/disabling document encryption?
> 
> Is there any intersection with the _access work Jan is working on?
> 
>> Thoughts?
>> B.

Re: native encryption for couchdb 4.0?

[Joan Touzet <wo...@apache.org>]

On 2020-03-12 12:29, Robert Samuel Newson wrote:
> Hi All,
> 
> Our team at IBM are working on native encryption of document content for the Cloudant service and are wondering if there'd be interest (or objection!) to this landing as a CouchDB feature?

Yes!

> This is only targeted at the (future) CouchDB 4.0 release which introduces FoundationDB as the persistence layer and, as stated above, currently only for document bodies.
> 
> This would be a configuration option (and presumably disabled by default).
> 
> I'll spare us all the crypto details for now (besides pointing out they've been reviewed by our in-house cryptographers and use only public algorithms and techniques in a straightforward manner).

Will the code be platform independent (or at least NIFfed in a way that 
supports compiling on Mac, FreeBSD, Windows?)

Is there any impact on our CouchDB API surface, other than 
enabling/disabling document encryption?

Is there any intersection with the _access work Jan is working on?

> Thoughts?
> 
> B.
>