You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Hanifi Gunes <ha...@gmail.com> on 2015/04/10 03:15:24 UTC
Review Request 33052: DRILL-2611: value vectors should report valid
value count
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33052/
-----------------------------------------------------------
Review request for drill, Mehant Baid and Parth Chandra.
Repository: drill-git
Description
-------
DRILL-2611: value vectors should report valid value count
Changes
- unify the behavior of value count interfaces across VVs -- get/setters
- ensure value count reported reflects underlying state of the buffer
- enforce consumers to use getAccessor().get/setValueCount
- ensure metadata created based on getAccessor().getValueCount
Diffs
-----
exec/java-exec/src/main/codegen/templates/ComplexWriters.java 576fd8352197ba950be7d7e661fb52dd92b52f2a
exec/java-exec/src/main/codegen/templates/FixedValueVectors.java f7260e727019b2f38128891e63fdccf7a3e7e032
exec/java-exec/src/main/codegen/templates/NullableValueVectors.java 075316e4f3ac5327e7893688c5e88cfee98e50bc
exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java c7cf8e6fe18f1b9813ae22495ac79a447f61cfff
exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 61a24814fd032e7112f4cf4ca7310517df58cfd3
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReader.java 759327a307aefd51dc69ea4282a7d58d6309e142
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedByteAlignedReader.java c2af964fd606924587fe2093b3ccb1ec1de922af
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedWidthRepeatedReader.java f20d7655c76237fc5d3a95760f00ca2400a8df07
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableColumnReader.java 16519a851a18924fb59753c456a2da4076d5d245
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java 8087118e1de8ef1b043c80ab4fc85215284670dc
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthColumnReaders.java 7464f30179059a728f5c30f37c17adf0f332604c
exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseDataValueVector.java d48ea99237bb822cafc8b835c3af0f4789c6eb29
exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseValueVector.java 81d3a8623fb86068d8c81f08e1d38d37b856e26c
exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java 1558e198accf43a51628bb68b9ae714451d0e436
Diff: https://reviews.apache.org/r/33052/diff/
Testing
-------
unit, reg, sf100
Thanks,
Hanifi Gunes
Re: Review Request 33052: DRILL-2611: value vectors should report
valid value count
Posted by Mehant Baid <ba...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33052/#review80041
-----------------------------------------------------------
Ship it!
Ship It!
- Mehant Baid
On April 13, 2015, 11:01 p.m., Hanifi Gunes wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33052/
> -----------------------------------------------------------
>
> (Updated April 13, 2015, 11:01 p.m.)
>
>
> Review request for drill, Mehant Baid and Parth Chandra.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2611: value vectors should report valid value count
>
> Changes
> - unify the behavior of value count interfaces across VVs -- get/setters
> - ensure value count reported reflects underlying state of the buffer
> - enforce consumers to use getAccessor().get/setValueCount
> - ensure metadata created based on getAccessor().getValueCount
>
>
> Diffs
> -----
>
> exec/java-exec/src/main/codegen/templates/ComplexWriters.java 576fd8352197ba950be7d7e661fb52dd92b52f2a
> exec/java-exec/src/main/codegen/templates/FixedValueVectors.java e9ec220dc653db1e1acb0538bbbc1207fb4ee194
> exec/java-exec/src/main/codegen/templates/NullableValueVectors.java 075316e4f3ac5327e7893688c5e88cfee98e50bc
> exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java c7cf8e6fe18f1b9813ae22495ac79a447f61cfff
> exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java edb851eb10be43d889ce5fd98d9bde036707870a
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReader.java 759327a307aefd51dc69ea4282a7d58d6309e142
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedByteAlignedReader.java c2af964fd606924587fe2093b3ccb1ec1de922af
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedWidthRepeatedReader.java f20d7655c76237fc5d3a95760f00ca2400a8df07
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableColumnReader.java 16519a851a18924fb59753c456a2da4076d5d245
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java 8087118e1de8ef1b043c80ab4fc85215284670dc
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthColumnReaders.java 7464f30179059a728f5c30f37c17adf0f332604c
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseDataValueVector.java d48ea99237bb822cafc8b835c3af0f4789c6eb29
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseValueVector.java 81d3a8623fb86068d8c81f08e1d38d37b856e26c
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java d8bd9723db9f2ecd1466b1144345ca371f68a3bb
>
> Diff: https://reviews.apache.org/r/33052/diff/
>
>
> Testing
> -------
>
> unit, reg, sf100
>
>
> Thanks,
>
> Hanifi Gunes
>
>
Re: Review Request 33052: DRILL-2611: value vectors should report
valid value count
Posted by Hanifi Gunes <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33052/
-----------------------------------------------------------
(Updated April 13, 2015, 11:01 p.m.)
Review request for drill, Mehant Baid and Parth Chandra.
Repository: drill-git
Description
-------
DRILL-2611: value vectors should report valid value count
Changes
- unify the behavior of value count interfaces across VVs -- get/setters
- ensure value count reported reflects underlying state of the buffer
- enforce consumers to use getAccessor().get/setValueCount
- ensure metadata created based on getAccessor().getValueCount
Diffs (updated)
-----
exec/java-exec/src/main/codegen/templates/ComplexWriters.java 576fd8352197ba950be7d7e661fb52dd92b52f2a
exec/java-exec/src/main/codegen/templates/FixedValueVectors.java e9ec220dc653db1e1acb0538bbbc1207fb4ee194
exec/java-exec/src/main/codegen/templates/NullableValueVectors.java 075316e4f3ac5327e7893688c5e88cfee98e50bc
exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java c7cf8e6fe18f1b9813ae22495ac79a447f61cfff
exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java edb851eb10be43d889ce5fd98d9bde036707870a
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReader.java 759327a307aefd51dc69ea4282a7d58d6309e142
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedByteAlignedReader.java c2af964fd606924587fe2093b3ccb1ec1de922af
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedWidthRepeatedReader.java f20d7655c76237fc5d3a95760f00ca2400a8df07
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableColumnReader.java 16519a851a18924fb59753c456a2da4076d5d245
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java 8087118e1de8ef1b043c80ab4fc85215284670dc
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthColumnReaders.java 7464f30179059a728f5c30f37c17adf0f332604c
exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseDataValueVector.java d48ea99237bb822cafc8b835c3af0f4789c6eb29
exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseValueVector.java 81d3a8623fb86068d8c81f08e1d38d37b856e26c
exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java d8bd9723db9f2ecd1466b1144345ca371f68a3bb
Diff: https://reviews.apache.org/r/33052/diff/
Testing
-------
unit, reg, sf100
Thanks,
Hanifi Gunes
Re: Review Request 33052: DRILL-2611: value vectors should report
valid value count
Posted by Hanifi Gunes <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33052/
-----------------------------------------------------------
(Updated April 13, 2015, 10:54 p.m.)
Review request for drill, Mehant Baid and Parth Chandra.
Changes
-------
addressed comments.
Repository: drill-git
Description
-------
DRILL-2611: value vectors should report valid value count
Changes
- unify the behavior of value count interfaces across VVs -- get/setters
- ensure value count reported reflects underlying state of the buffer
- enforce consumers to use getAccessor().get/setValueCount
- ensure metadata created based on getAccessor().getValueCount
Diffs (updated)
-----
exec/java-exec/src/main/codegen/templates/ComplexWriters.java 576fd8352197ba950be7d7e661fb52dd92b52f2a
exec/java-exec/src/main/codegen/templates/FixedValueVectors.java e9ec220dc653db1e1acb0538bbbc1207fb4ee194
exec/java-exec/src/main/codegen/templates/NullableValueVectors.java 075316e4f3ac5327e7893688c5e88cfee98e50bc
exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java c7cf8e6fe18f1b9813ae22495ac79a447f61cfff
exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java edb851eb10be43d889ce5fd98d9bde036707870a
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReader.java 759327a307aefd51dc69ea4282a7d58d6309e142
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedByteAlignedReader.java c2af964fd606924587fe2093b3ccb1ec1de922af
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedWidthRepeatedReader.java f20d7655c76237fc5d3a95760f00ca2400a8df07
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableColumnReader.java 16519a851a18924fb59753c456a2da4076d5d245
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java 8087118e1de8ef1b043c80ab4fc85215284670dc
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthColumnReaders.java 7464f30179059a728f5c30f37c17adf0f332604c
exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseDataValueVector.java d48ea99237bb822cafc8b835c3af0f4789c6eb29
exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseValueVector.java 81d3a8623fb86068d8c81f08e1d38d37b856e26c
exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java d8bd9723db9f2ecd1466b1144345ca371f68a3bb
exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/AbstractMapVector.java b0783afe57317dcd8dc7a2a8d967dcdb1f305edb
exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedListVector.java c0f529961343145d67f835a95f58c4eaf2fae2a4
exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/TestEmptyPopulator.java 8426a6abbf0c20be6f81bc82521f31ed7cde2557
Diff: https://reviews.apache.org/r/33052/diff/
Testing
-------
unit, reg, sf100
Thanks,
Hanifi Gunes
Re: Review Request 33052: DRILL-2611: value vectors should report
valid value count
Posted by Hanifi Gunes <ha...@gmail.com>.
> On April 10, 2015, 10 p.m., Mehant Baid wrote:
> > exec/java-exec/src/main/codegen/templates/NullableValueVectors.java, line 442
> > <https://reviews.apache.org/r/33052/diff/1/?file=922194#file922194line442>
> >
> > For nullable variable length types we can get the record count from the bits vector? Otherwise we will be performing two hops -> First we will delegate to the value vector which will in turn delegate to the offset vector?
This makes perfect sense.
> On April 10, 2015, 10 p.m., Mehant Baid wrote:
> > exec/java-exec/src/main/codegen/templates/FixedValueVectors.java, line 754
> > <https://reviews.apache.org/r/33052/diff/1/?file=922193#file922193line754>
> >
> > should the writer index be modified here?
I think it is safer not to do so here at the expense of breaking duality of get/set calls. As we don't allow nulls in FixedVVs, it would be reasonable to assume all writes will go through #set methods.
- Hanifi
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33052/#review79755
-----------------------------------------------------------
On April 10, 2015, 1:15 a.m., Hanifi Gunes wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33052/
> -----------------------------------------------------------
>
> (Updated April 10, 2015, 1:15 a.m.)
>
>
> Review request for drill, Mehant Baid and Parth Chandra.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2611: value vectors should report valid value count
>
> Changes
> - unify the behavior of value count interfaces across VVs -- get/setters
> - ensure value count reported reflects underlying state of the buffer
> - enforce consumers to use getAccessor().get/setValueCount
> - ensure metadata created based on getAccessor().getValueCount
>
>
> Diffs
> -----
>
> exec/java-exec/src/main/codegen/templates/ComplexWriters.java 576fd8352197ba950be7d7e661fb52dd92b52f2a
> exec/java-exec/src/main/codegen/templates/FixedValueVectors.java f7260e727019b2f38128891e63fdccf7a3e7e032
> exec/java-exec/src/main/codegen/templates/NullableValueVectors.java 075316e4f3ac5327e7893688c5e88cfee98e50bc
> exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java c7cf8e6fe18f1b9813ae22495ac79a447f61cfff
> exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 61a24814fd032e7112f4cf4ca7310517df58cfd3
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReader.java 759327a307aefd51dc69ea4282a7d58d6309e142
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedByteAlignedReader.java c2af964fd606924587fe2093b3ccb1ec1de922af
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedWidthRepeatedReader.java f20d7655c76237fc5d3a95760f00ca2400a8df07
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableColumnReader.java 16519a851a18924fb59753c456a2da4076d5d245
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java 8087118e1de8ef1b043c80ab4fc85215284670dc
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthColumnReaders.java 7464f30179059a728f5c30f37c17adf0f332604c
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseDataValueVector.java d48ea99237bb822cafc8b835c3af0f4789c6eb29
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseValueVector.java 81d3a8623fb86068d8c81f08e1d38d37b856e26c
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java 1558e198accf43a51628bb68b9ae714451d0e436
>
> Diff: https://reviews.apache.org/r/33052/diff/
>
>
> Testing
> -------
>
> unit, reg, sf100
>
>
> Thanks,
>
> Hanifi Gunes
>
>
Re: Review Request 33052: DRILL-2611: value vectors should report
valid value count
Posted by Mehant Baid <ba...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33052/#review79755
-----------------------------------------------------------
exec/java-exec/src/main/codegen/templates/FixedValueVectors.java
<https://reviews.apache.org/r/33052/#comment129287>
should the writer index be modified here?
exec/java-exec/src/main/codegen/templates/NullableValueVectors.java
<https://reviews.apache.org/r/33052/#comment129288>
For nullable variable length types we can get the record count from the bits vector? Otherwise we will be performing two hops -> First we will delegate to the value vector which will in turn delegate to the offset vector?
- Mehant Baid
On April 10, 2015, 1:15 a.m., Hanifi Gunes wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33052/
> -----------------------------------------------------------
>
> (Updated April 10, 2015, 1:15 a.m.)
>
>
> Review request for drill, Mehant Baid and Parth Chandra.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> DRILL-2611: value vectors should report valid value count
>
> Changes
> - unify the behavior of value count interfaces across VVs -- get/setters
> - ensure value count reported reflects underlying state of the buffer
> - enforce consumers to use getAccessor().get/setValueCount
> - ensure metadata created based on getAccessor().getValueCount
>
>
> Diffs
> -----
>
> exec/java-exec/src/main/codegen/templates/ComplexWriters.java 576fd8352197ba950be7d7e661fb52dd92b52f2a
> exec/java-exec/src/main/codegen/templates/FixedValueVectors.java f7260e727019b2f38128891e63fdccf7a3e7e032
> exec/java-exec/src/main/codegen/templates/NullableValueVectors.java 075316e4f3ac5327e7893688c5e88cfee98e50bc
> exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java c7cf8e6fe18f1b9813ae22495ac79a447f61cfff
> exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 61a24814fd032e7112f4cf4ca7310517df58cfd3
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReader.java 759327a307aefd51dc69ea4282a7d58d6309e142
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedByteAlignedReader.java c2af964fd606924587fe2093b3ccb1ec1de922af
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/FixedWidthRepeatedReader.java f20d7655c76237fc5d3a95760f00ca2400a8df07
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableColumnReader.java 16519a851a18924fb59753c456a2da4076d5d245
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java 8087118e1de8ef1b043c80ab4fc85215284670dc
> exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLengthColumnReaders.java 7464f30179059a728f5c30f37c17adf0f332604c
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseDataValueVector.java d48ea99237bb822cafc8b835c3af0f4789c6eb29
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseValueVector.java 81d3a8623fb86068d8c81f08e1d38d37b856e26c
> exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java 1558e198accf43a51628bb68b9ae714451d0e436
>
> Diff: https://reviews.apache.org/r/33052/diff/
>
>
> Testing
> -------
>
> unit, reg, sf100
>
>
> Thanks,
>
> Hanifi Gunes
>
>