You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by Kevin Olson <ko...@marinsoftware.com> on 2012/05/05 00:51:23 UTC

Using compression with IndexedStorage

I've just started playing around with IndexedStorage in Pig 0.10, as a 
means to an end to use sparse merge join.  Seems to be working, but is 
there a way to use compression with the IndexedStorage StoreFunc?

I've tried setting output.compression.enabled and 
output.compression.codec to use Snappy, but IndexedStorage seems to be 
ignoring them and just outputs uncompressed data.

Thanks,
Kevin