You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@hadoop.apache.org by Ted Yu <yu...@gmail.com> on 2010/01/19 20:38:50 UTC

splittable encrypted compressed files

Hi,
I experimented with hadoop-lzo and liked it.

Our requirement is changing and we may need to encrypt data files. Does
anyone know of splittable encrypted compression scheme that map task can use
?

Thanks

Re: splittable encrypted compressed files

Posted by Ted Yu <yu...@gmail.com>.
A bit more background: the encryption is required by third party. Data
coming from third party may come in encrypted .gz format.

Cheers

On Sun, Jan 24, 2010 at 2:34 PM, Gautam Singaraju <
gautam.singaraju@gmail.com> wrote:

> A encryption/decryption mechanism requires the secret key to be
> securely distributed. As far as moving data on the wire is concerned,
> SSH is already being used. I am wondering about the performance hit
> when encryption and then again decryption is applied to the data and
> what value that might add.
>
> ---
> Gautam
>
>
>
> On Tue, Jan 19, 2010 at 2:38 PM, Ted Yu <yu...@gmail.com> wrote:
> > Hi,
> > I experimented with hadoop-lzo and liked it.
> >
> > Our requirement is changing and we may need to encrypt data files. Does
> > anyone know of splittable encrypted compression scheme that map task can
> use
> > ?
> >
> > Thanks
> >
>

Re: splittable encrypted compressed files

Posted by Gautam Singaraju <ga...@gmail.com>.
A encryption/decryption mechanism requires the secret key to be
securely distributed. As far as moving data on the wire is concerned,
SSH is already being used. I am wondering about the performance hit
when encryption and then again decryption is applied to the data and
what value that might add.

---
Gautam



On Tue, Jan 19, 2010 at 2:38 PM, Ted Yu <yu...@gmail.com> wrote:
> Hi,
> I experimented with hadoop-lzo and liked it.
>
> Our requirement is changing and we may need to encrypt data files. Does
> anyone know of splittable encrypted compression scheme that map task can use
> ?
>
> Thanks
>