You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/11/12 08:18:44 UTC
[GitHub] [iceberg] nastra commented on a change in pull request #3533: Arrow: Fix vectorized position reader
nastra commented on a change in pull request #3533:
URL: https://github.com/apache/iceberg/pull/3533#discussion_r748055193
##########
File path: arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedArrowReader.java
##########
@@ -407,35 +412,55 @@ public void setBatchSize(int batchSize) {
}
private static final class PositionVectorReader extends VectorizedArrowReader {
+ private final Field arrowField = ArrowSchemaUtil.convert(MetadataColumns.ROW_POSITION);
+ private final BufferAllocator bufferAllocator = ArrowAllocation.rootAllocator();
+ private final boolean setArrowValidityVector;
private long rowStart;
+ private int batchSize;
+ private FieldVector vec;
private NullabilityHolder nulls;
+ PositionVectorReader(boolean setArrowValidityVector) {
+ this.setArrowValidityVector = setArrowValidityVector;
+ }
+
@Override
public VectorHolder read(VectorHolder reuse, int numValsToRead) {
- Field arrowField = ArrowSchemaUtil.convert(MetadataColumns.ROW_POSITION);
- FieldVector vec = arrowField.createVector(ArrowAllocation.rootAllocator());
-
- if (reuse != null) {
- vec.setValueCount(0);
- nulls.reset();
+ if (reuse == null) {
+ this.vec = newVector();
+ this.nulls = newNullabilityHolder();
} else {
- ((BigIntVector) vec).allocateNew(numValsToRead);
- for (int i = 0; i < numValsToRead; i += 1) {
- vec.getDataBuffer().setLong(i * Long.BYTES, rowStart + i);
- }
- for (int i = 0; i < numValsToRead; i += 1) {
- BitVectorHelper.setBit(vec.getValidityBuffer(), i);
+ vec.setValueCount(0);
Review comment:
do we need to call `nulls.reset();` here when reusing? Also do we need to call `nulls.setNotNulls(0, numValsToRead)` for both cases?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org