Column isNull[] flag is ignored after the initial batch ?

[Ashika Umanga Umagiliya <um...@gmail.com>]

I will post the link to SO question since it has better formatting :

https://stackoverflow.com/questions/52176439/orc-isnull-flag-doesnt-set-after-initial-batch

Any tips on why this is happening?

Re: Column isNull[] flag is ignored after the initial batch ?

[Ashika Umanga Umagiliya <um...@gmail.com>]

Thank you for the prompt reply..


> Check for the .noNulls=true/false inside the vector.
>

I do check inside the vector when reading .. As shown in the code :

Configuration conf = new Configuration();
Reader reader = OrcFile.createReader(new Path(file),
        OrcFile.readerOptions(conf));
RecordReader rows = reader.rows();
VectorizedRowBatch batch = reader.getSchema().createRowBatch();
BytesColumnVector col1 =(BytesColumnVector)batch.cols[0];
LongColumnVector col2 =(LongColumnVector)batch.cols[1];

int row=0;
while (rows.nextBatch(batch)) {
    for(int r=0; r < batch.size; ++r) {

        byte[] b1 = col1.vector[r];
        long val2 = col2.vector[r];
        System.out.println( ("row #"+row+" col1:"+(col1.isNull[r] ?
"null" : col1.toString(r)))  +" col2:"+(col2.isNull[r] ? "null" :
val2));
        row++;

    }
    //row++;
}
rows.close();




>
> If I'm not wrong the first .reset() sets the .noNulls=true, which causes
> the VRB to ignore the .isNull[] offsets.
>
>  .reset() during the writing ? If I dont reset() the batch , how can I
create a new batch ?

Re: Column isNull[] flag is ignored after the initial batch ?

[Gopal Vijayaraghavan <go...@apache.org>]

> https://stackoverflow.com/questions/52176439/orc-isnull-flag-doesnt-set-after-initial-batch
> 
> Any tips on why this is happening?

Check for the .noNulls=true/false inside the vector.

If I'm not wrong the first .reset() sets the .noNulls=true, which causes the VRB to ignore the .isNull[] offsets.

Cheers,
Gopal