Re: Question about RLE support in CarbonData

[Ravindra Pesala <ra...@gmail.com>]

Hi,

Here some encodings can be done on each field level and some can be done on
blocklet(batch of column data) level. So DICTIONARY encoding is done on
each field level and this FieldConverter is only encoding data on field
level.
RLE is applied on blocklet level so it is applied while writing the block,
please have a look at IndexStorage class for RLE encoding on blocklet. And
DELTA encoding only happens for measures on blocklet level, have a look at
ValueCompressionUtil class for DELTA encoding.

Regards,
Ravindra

On 1 December 2016 at 01:45, Hao Jiang <ha...@uchicago.edu> wrote:

> Dear Dev Team,
>
> I have a question about run-length encoding (RLE) support in CarbonData.
>
> In Encoding enum type I can see RLE, DELTA and some other encodings.
>
> However while I look at the code (FieldEncoderFactory.createFieldEncoder),
> I notice there is an if / else like the following:
>
> if(dataField.getColumn().hasEncoding(Encoding.DIRECT_DICTIONARY) ….) {
>      // DirectDictionaryConverter
> } else if (… Encoding.DICTIONARY ….)  {
>      // DictionaryConverter
> }
>  …
> else  {
>      // NonDictionaryFieldConverter
> }
>
> And I cannot find any encoding code related to RLE in the
> NonDictionaryFieldConverter.
>
> Please kindly suggest where I can find the FieldConverter for RLE, DELTA
> and other encodings?
>
> Thanks!
>
> Regards,
> Hao




-- 
Thanks & Regards,
Ravi