You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@carbondata.apache.org by Swapnil Shinde <sw...@gmail.com> on 2017/07/20 16:10:08 UTC

Vectorized reader exceptions

Hi All
    I am not sure if this is random exception but this is what I have
observed -
Create and load carbondata table from Spark dataframe - Without
dictionary_include on two INT columns - Works fine. I can "select *" on it.
Create and load carbondata table from same Spark datafrmae - With
dictionary_include on two INT columns - Throws NullPointerException when I
run "select *".

Stacktrace throws error here -
https://github.com/apache/carbondata/blob/branch-1.1/core/src/main/java/org/apache/carbondata/core/datastore/chunk/store/impl/safe/SafeVariableLengthDimensionDataChunkStore.java#L140

I found similar error fixed in this PR but not available in 1.1.0 version -
https://github.com/apache/carbondata/pull/970
I kind of tried adding above PR's changes with additional necessary changes
to build successfully. (Not sure if it was a good idea)
If I load again and run "select *"  it throws different error -
[image: Inline image 1]

I am not much familiar with vectorized reader source code so not sure how
to fix this. However, due to above error I have to load that dataset
without creating dimensions on columns I need for filtration.
As always, your help is very much appreciated!

Thanks
Swapnil