You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu> on 2026/09/20 22:01:01 UTC
[L] Change in asterixdb[master]: [NO ISSUE][STO] Skip column values in bulk
From Ali Alsuliman <al...@gmail.com>:
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21854?usp=email )
Change subject: [NO ISSUE][STO] Skip column values in bulk
......................................................................
[NO ISSUE][STO] Skip column values in bulk
- user model changes: no
- storage format changes: no
- interface changes: yes
Details: Positioning a mega leaf's columns at tuple k skips the k
tuples before it, and AbstractColumnValuesReader.skip() did so by
calling next() once per entry, decoding every value on the way. For a
repeated column that is one decode per array element: on a leaf of
1400 records holding 384 doubles each, a point lookup that projects
the array spent several milliseconds decoding values it then threw
away, which throttled the previous-record fetch that secondary index
maintenance performs to a few thousand records per second.
Skipping now consumes the level stream and moves the value stream
separately. ParquetRunLengthBitPackingHybridDecoder.skipWhile()
consumes a run of identical levels by adjusting its remaining count,
so a run of array elements costs the same whatever its length, and
the column readers count the values among the entries they skipped
and hand the count to their value reader in one call. skip(int) is
abstract on AbstractValueReader and AbstractParquetValuesReader so
that every reader states how its encoding can move: fixed-width
values skip bytes, delta-packed values skip within a decoded block,
run-length entries skip runs, and prefix-compressed strings walk, the
one encoding that cannot be jumped. Primary key readers refuse, since
the cursor owns their position.
ParquetDeltaBinaryPackingValuesReader.skip() had never been called;
it advanced the value count but not the buffer index, so the read
after it would have returned the skipped value. Fixed while wiring it
in.
On a 1400-tuple leaf of 384-dimension doubles, skipping to the middle
went from 3.8 ms to 40 us.
ColumnSkipTest drives every corpus in the module, plus one of
embedding-sized arrays with integer and null components, through the
real flush path and checks that a skip lands on the same tuple as a
sequential read. ParquetRunLengthBitPackingHybridDecoderSkipTest
covers runs, limits, bit-packed groups and a randomized interleaving
of skips and reads.
Co-Authored-By: Claude Fable 5.1 <no...@anthropic.com>
Change-Id: I370fe0f7ba9b607d6f2a027177c985f0e898da55
---
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/AbstractParquetValuesReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaBinaryPackingValuesReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaByteArrayReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaLengthByteArrayValuesReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetPlainFixedLengthValuesReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetRunLengthBitPackingHybridDecoder.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/AbstractColumnValuesReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/PrimitiveColumnValuesReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/RepeatedPrimitiveColumnValuesReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/AbstractValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/BooleanValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/DoubleValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/FloatValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/IntValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/LongValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/NoOpValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/StringValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/UUIDValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/AbstractFixedLengthColumnKeyValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/LevelKeyReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/StringKeyValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DateTimeValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DateValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DayTimeDurationValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DurationValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/IntervalValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/TimeValueReader.java
M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/YearMonthDurationValueReader.java
A asterixdb/asterix-column/src/test/java/org/apache/asterix/column/bytes/decoder/ParquetRunLengthBitPackingHybridDecoderSkipTest.java
A asterixdb/asterix-column/src/test/java/org/apache/asterix/column/test/skip/ColumnSkipTest.java
30 files changed, 761 insertions(+), 31 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/54/21854/1
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/AbstractParquetValuesReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/AbstractParquetValuesReader.java
index 230da0a..ad102ac 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/AbstractParquetValuesReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/AbstractParquetValuesReader.java
@@ -22,6 +22,7 @@
import org.apache.asterix.column.bytes.stream.in.AbstractBytesInputStream;
import org.apache.hyracks.data.std.api.IValueReference;
+import org.apache.hyracks.util.annotations.AiProvenance;
import org.apache.parquet.column.values.ValuesReader;
/**
@@ -32,6 +33,19 @@
public abstract void skip();
+ /**
+ * Skips {@code count} values. Abstract so that every decoder states whether its layout allows a positional move
+ * or forces a walk; {@link #skipOneByOne(int)} is the walk.
+ */
+ public abstract void skip(int count);
+
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ protected final void skipOneByOne(int count) {
+ for (int i = 0; i < count; i++) {
+ skip();
+ }
+ }
+
public int readInteger() {
throw new UnsupportedOperationException(getClass().getName());
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaBinaryPackingValuesReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaBinaryPackingValuesReader.java
index 9aafa0f..e7cb0d0 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaBinaryPackingValuesReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaBinaryPackingValuesReader.java
@@ -25,6 +25,7 @@
import org.apache.asterix.column.bytes.ParquetDeltaBinaryPackingConfig;
import org.apache.asterix.column.bytes.stream.in.AbstractBytesInputStream;
+import org.apache.hyracks.util.annotations.AiProvenance;
import org.apache.parquet.bytes.BytesUtils;
import org.apache.parquet.column.values.bitpacking.BytePackerForLong;
import org.apache.parquet.column.values.bitpacking.Packer;
@@ -104,9 +105,24 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.REFACTORED)
public void skip() {
checkRead();
valuesRead++;
+ // without this the next read returns the value that was supposed to be skipped
+ valuesBufferedRead++;
+ }
+
+ @Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ while (count > 0) {
+ checkRead();
+ int n = Math.min(count, valuesBuffered - valuesBufferedRead);
+ valuesBufferedRead += n;
+ valuesRead += n;
+ count -= n;
+ }
}
@Override
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaByteArrayReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaByteArrayReader.java
index 70c25b8..65f7930 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaByteArrayReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaByteArrayReader.java
@@ -23,6 +23,7 @@
import org.apache.asterix.column.bytes.stream.in.AbstractBytesInputStream;
import org.apache.hyracks.data.std.api.IValueReference;
import org.apache.hyracks.data.std.util.ArrayBackedValueStorage;
+import org.apache.hyracks.util.annotations.AiProvenance;
import org.apache.hyracks.util.string.UTF8StringUtil;
import org.apache.parquet.bytes.BytesUtils;
import org.apache.parquet.column.values.deltastrings.DeltaByteArrayReader;
@@ -65,6 +66,13 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ // every value is a prefix of its predecessor plus a suffix, so there is no way past a value but through it
+ skipOneByOne(count);
+ }
+
+ @Override
public IValueReference readBytes() {
int prefixLength = prefixLengthReader.readInteger();
// This does not copy bytes
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaLengthByteArrayValuesReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaLengthByteArrayValuesReader.java
index 9913269..357b954 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaLengthByteArrayValuesReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetDeltaLengthByteArrayValuesReader.java
@@ -23,6 +23,7 @@
import org.apache.asterix.column.bytes.stream.in.AbstractBytesInputStream;
import org.apache.hyracks.data.std.api.IValueReference;
import org.apache.hyracks.data.std.primitive.VoidPointable;
+import org.apache.hyracks.util.annotations.AiProvenance;
import org.apache.parquet.bytes.BytesUtils;
import org.apache.parquet.io.ParquetDecodingException;
@@ -55,6 +56,20 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ long bytes = 0;
+ for (int i = 0; i < count; i++) {
+ bytes += lengthReader.readInteger();
+ }
+ try {
+ in.skipFully(bytes);
+ } catch (IOException e) {
+ throw new ParquetDecodingException("Failed to skip " + bytes + " bytes");
+ }
+ }
+
+ @Override
public IValueReference readBytes() {
int length = lengthReader.readInteger();
try {
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetPlainFixedLengthValuesReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetPlainFixedLengthValuesReader.java
index 28e0033..1333f82 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetPlainFixedLengthValuesReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetPlainFixedLengthValuesReader.java
@@ -25,6 +25,7 @@
import org.apache.asterix.column.bytes.stream.in.ValueInputStream;
import org.apache.hyracks.data.std.api.IPointable;
import org.apache.hyracks.data.std.api.IValueReference;
+import org.apache.hyracks.util.annotations.AiProvenance;
import org.apache.parquet.io.ParquetDecodingException;
public class ParquetPlainFixedLengthValuesReader extends AbstractParquetValuesReader {
@@ -59,6 +60,17 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ int bytes = count * valueLength;
+ try {
+ in.skipBytes(bytes);
+ } catch (IOException e) {
+ throw new ParquetDecodingException("could not skip " + bytes + " bytes", e);
+ }
+ }
+
+ @Override
public long readLong() {
try {
return in.readLong();
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetRunLengthBitPackingHybridDecoder.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetRunLengthBitPackingHybridDecoder.java
index 4607dc2..6493856 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetRunLengthBitPackingHybridDecoder.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/bytes/decoder/ParquetRunLengthBitPackingHybridDecoder.java
@@ -24,6 +24,7 @@
import java.util.Arrays;
import org.apache.hyracks.api.exceptions.HyracksDataException;
+import org.apache.hyracks.util.annotations.AiProvenance;
import org.apache.parquet.Preconditions;
import org.apache.parquet.bytes.BytesUtils;
import org.apache.parquet.column.values.bitpacking.BytePacker;
@@ -90,6 +91,74 @@
return result;
}
+ /**
+ * Consumes consecutive entries equal to {@code value}, at most {@code limit} of them, and returns how many were
+ * consumed. The first entry that differs is left for {@link #readInt()}. A run-length run is consumed by
+ * adjusting its remaining count, so the cost of skipping an array of identical levels does not depend on the
+ * array's length; a bit-packed group is scanned in its decoded buffer.
+ * <p>
+ * The stream's last bit-packed group is padded with zeros that are indistinguishable from real entries, so the
+ * caller must bound {@code limit} by the number of entries it knows remain, as it already does for
+ * {@link #readInt()}.
+ */
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public int skipWhile(int value, int limit) throws HyracksDataException {
+ int skipped = 0;
+ try {
+ while (skipped < limit) {
+ if (currentCount == 0) {
+ if (in.available() <= 0) {
+ // leave the exhausted stream to readInt(), which reports it the same way it always has
+ break;
+ }
+ readNext();
+ }
+ if (mode == MODE.RLE) {
+ if (currentValue != value) {
+ break;
+ }
+ int n = Math.min(currentCount, limit - skipped);
+ currentCount -= n;
+ skipped += n;
+ } else {
+ int start = currentBufferLength - currentCount;
+ int end = start + Math.min(currentCount, limit - skipped);
+ int i = start;
+ while (i < end && currentBuffer[i] == value) {
+ i++;
+ }
+ currentCount -= i - start;
+ skipped += i - start;
+ if (i < end) {
+ break;
+ }
+ }
+ }
+ } catch (IOException e) {
+ throw HyracksDataException.create(e);
+ }
+ return skipped;
+ }
+
+ /**
+ * Skips {@code count} entries whatever their values, a run at a time.
+ */
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) throws HyracksDataException {
+ try {
+ while (count > 0) {
+ if (currentCount == 0) {
+ readNext();
+ }
+ int n = Math.min(currentCount, count);
+ currentCount -= n;
+ count -= n;
+ }
+ } catch (IOException e) {
+ throw HyracksDataException.create(e);
+ }
+ }
+
private void readNext() throws IOException {
Preconditions.checkArgument(in.available() > 0, "Reading past RLE/BitPacking stream.");
final int header = BytesUtils.readUnsignedVarInt(in);
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/AbstractColumnValuesReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/AbstractColumnValuesReader.java
index 702de17..2141b79 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/AbstractColumnValuesReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/AbstractColumnValuesReader.java
@@ -32,6 +32,7 @@
import org.apache.hyracks.api.exceptions.HyracksDataException;
import org.apache.hyracks.data.std.api.IValueReference;
import org.apache.hyracks.storage.am.lsm.btree.column.error.ColumnarValueException;
+import org.apache.hyracks.util.annotations.AiProvenance;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.parquet.bytes.BytesUtils;
@@ -254,28 +255,45 @@
}
}
- @Override
- public void skip(int count) throws HyracksDataException {
- if (primaryKey) {
- /*
- * Do not modify the position of primary key (PK) reader as it is maintained by the cursor.
- * Previously, we used two separate primary key readers
- * 1- One for the cursor
- * - Its position maintained by the cursor
- * 2- And one for assembler (if the primary key is requested -- like in SELECT *)
- * - Its position maintained by calling this skip method
- * In the previous approach, maintaining the positions of two primary key(s) readers were messy,
- * as we needed to re-sync the assembler reader with the cursor PK reader. The reason is that
- * anti-matters are handled at the cursor level. When anti-matters are processed, they are skipped --
- * making the assembler PK reader out of sync.
- *
- * Additionally, maintaining two readers that are decoding the same values (twice) is unnecessary.
- */
- return;
+ protected final boolean isPrimaryKey() {
+ return primaryKey;
+ }
+
+ /**
+ * Consumes up to {@code limit} consecutive plain values (entries at the max level that are not null) without
+ * decoding them, and returns how many were consumed. Zero means the next entry is a delimiter, a null, a missing,
+ * or the end of the column, and the caller has to step over it with {@link #nextLevel()}.
+ * <p>
+ * Afterwards this reader looks exactly as if {@code next()} had been called that many times, except that the
+ * value reader has not moved: the caller owes it a bulk {@link AbstractValueReader#skip(int)} of the returned
+ * count. Counting values here and skipping them in one move is what makes positioning inside a mega leaf cheap
+ * for arrays of fixed-width values, where {@code next()} would decode every element on the way.
+ */
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ protected final int skipPlainValues(int limit) throws HyracksDataException {
+ if (allMissing || limit <= 0) {
+ return 0;
}
- for (int i = 0; i < count; i++) {
- next();
+ int skipped;
+ try {
+ skipped = currentDefinitionLevels.skipWhile(maxLevel, limit);
+ } catch (Exception e) {
+ ColumnarValueException ex = new ColumnarValueException();
+ ObjectNode readerNode = ex.createNode(getClass().getSimpleName());
+ appendReaderInformation(readerNode);
+ LOGGER.error("error skipping plain values, collected info: {}", ex.getNode());
+ ex.addSuppressed(e);
+ throw ex;
}
+ if (skipped > 0) {
+ valueIndex += skipped;
+ level = maxLevel;
+ nullLevel = false;
+ // the state that the same number of nextLevel() calls would have left behind
+ firstValueForCurrentTuple = skipped == 1 && currentTupleIndex != previousTupleIndex;
+ previousTupleIndex = currentTupleIndex;
+ }
+ return skipped;
}
protected final void writeLevel(IColumnValuesWriter writer) throws HyracksDataException {
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/PrimitiveColumnValuesReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/PrimitiveColumnValuesReader.java
index 07569e6..39ec174 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/PrimitiveColumnValuesReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/PrimitiveColumnValuesReader.java
@@ -26,6 +26,7 @@
import org.apache.hyracks.api.exceptions.HyracksDataException;
import org.apache.hyracks.data.std.api.IValueReference;
import org.apache.hyracks.storage.am.lsm.btree.column.error.ColumnarValueException;
+import org.apache.hyracks.util.annotations.AiProvenance;
import com.fasterxml.jackson.databind.node.ObjectNode;
@@ -63,6 +64,45 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) throws HyracksDataException {
+ if (isPrimaryKey()) {
+ /*
+ * Do not modify the position of primary key (PK) reader as it is maintained by the cursor.
+ * Previously, we used two separate primary key readers
+ * 1- One for the cursor
+ * - Its position maintained by the cursor
+ * 2- And one for assembler (if the primary key is requested -- like in SELECT *)
+ * - Its position maintained by calling this skip method
+ * In the previous approach, maintaining the positions of two primary key(s) readers were messy,
+ * as we needed to re-sync the assembler reader with the cursor PK reader. The reason is that
+ * anti-matters are handled at the cursor level. When anti-matters are processed, they are skipped --
+ * making the assembler PK reader out of sync.
+ *
+ * Additionally, maintaining two readers that are decoding the same values (twice) is unnecessary.
+ */
+ return;
+ }
+ int values = 0;
+ while (count > 0) {
+ int plain = skipPlainValues(count);
+ values += plain;
+ count -= plain;
+ if (count > 0) {
+ if (valueIndex == valueCount) {
+ break;
+ }
+ nextLevel();
+ if (level == maxLevel) {
+ values++;
+ }
+ count--;
+ }
+ }
+ valueReader.skip(values);
+ }
+
+ @Override
public boolean isRepeated() {
return false;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/RepeatedPrimitiveColumnValuesReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/RepeatedPrimitiveColumnValuesReader.java
index 46f3c4a..363390c 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/RepeatedPrimitiveColumnValuesReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/RepeatedPrimitiveColumnValuesReader.java
@@ -26,6 +26,7 @@
import org.apache.hyracks.api.exceptions.HyracksDataException;
import org.apache.hyracks.api.util.InvokeUtil;
import org.apache.hyracks.storage.am.lsm.btree.column.error.ColumnarValueException;
+import org.apache.hyracks.util.annotations.AiProvenance;
import com.fasterxml.jackson.databind.node.ObjectNode;
@@ -124,17 +125,58 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.REFACTORED)
public void skip(int count) throws HyracksDataException {
+ int values = 0;
for (int i = 0; i < count; i++) {
- doNextAndCheck();
- if (isRepeatedValue()) {
- // the delimiter is the sole exit; inconsistent delimiter state turns this into a pure-CPU spin
- while (!isLastDelimiter()) {
- InvokeUtil.failIfInterrupted();
- doNextAndCheck();
- }
+ values += skipTuple();
+ }
+ valueReader.skip(values);
+ }
+
+ /**
+ * Consumes one tuple's entries, up to and including its last delimiter, and returns the number of values among
+ * them. Runs of plain values are consumed in bulk; only the entries that carry structure (delimiters, nulls,
+ * missing) are stepped over one at a time.
+ */
+ private int skipTuple() throws HyracksDataException {
+ int values = skipPlainEntries();
+ if (values == 0) {
+ values = skipEntry();
+ if (!isRepeatedValue()) {
+ return values;
}
}
+ while (!isLastDelimiter()) {
+ // the delimiter is the sole exit; inconsistent delimiter state turns this into a pure-CPU spin
+ InvokeUtil.failIfInterrupted();
+ int plain = skipPlainEntries();
+ values += plain > 0 ? plain : skipEntry();
+ }
+ return values;
+ }
+
+ private int skipPlainEntries() throws HyracksDataException {
+ int plain = skipPlainValues(valueCount - valueIndex);
+ if (plain > 0) {
+ // what setDelimiterIndex() computes for an entry at the max level
+ delimiterIndex = levelToDelimiterMap[maxLevel];
+ }
+ return plain;
+ }
+
+ /**
+ * Steps over a single entry exactly as {@link #next()} does, minus decoding the value; returns 1 if the entry
+ * was a value.
+ */
+ private int skipEntry() throws HyracksDataException {
+ if (valueIndex == valueCount) {
+ throw exhausted();
+ }
+ consumeDelimiterIfAny();
+ nextLevel();
+ setDelimiterIndex();
+ return level == maxLevel ? 1 : 0;
}
private void consumeDelimiterIfAny() {
@@ -161,9 +203,13 @@
private void doNextAndCheck() throws HyracksDataException {
if (!next()) {
- ColumnarValueException e = new ColumnarValueException();
- appendReaderInformation(e.createNode(getClass().getSimpleName()));
- throw e;
+ throw exhausted();
}
}
+
+ private ColumnarValueException exhausted() {
+ ColumnarValueException e = new ColumnarValueException();
+ appendReaderInformation(e.createNode(getClass().getSimpleName()));
+ return e;
+ }
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/AbstractValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/AbstractValueReader.java
index cec8a38..9f33863 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/AbstractValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/AbstractValueReader.java
@@ -33,6 +33,13 @@
public abstract ATypeTag getTypeTag();
+ /**
+ * Advances past {@code count} values without materializing them. Abstract on purpose: a reader over a fixed-width
+ * or block-decoded encoding can move positionally, one over a prefix-compressed encoding has to walk, and a new
+ * reader must say which it is rather than silently inherit a walk.
+ */
+ public abstract void skip(int count) throws HyracksDataException;
+
public boolean getBoolean() {
throw new UnsupportedOperationException(getClass().getName());
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/BooleanValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/BooleanValueReader.java
index 6b5e0d4..0b70905 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/BooleanValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/BooleanValueReader.java
@@ -24,6 +24,7 @@
import org.apache.asterix.column.bytes.stream.in.AbstractBytesInputStream;
import org.apache.asterix.om.types.ATypeTag;
import org.apache.hyracks.api.exceptions.HyracksDataException;
+import org.apache.hyracks.util.annotations.AiProvenance;
public final class BooleanValueReader extends AbstractValueReader {
private final ParquetRunLengthBitPackingHybridDecoder booleanReader;
@@ -54,6 +55,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) throws HyracksDataException {
+ booleanReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.BOOLEAN;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/DoubleValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/DoubleValueReader.java
index faa60d1..b521e46 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/DoubleValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/DoubleValueReader.java
@@ -23,6 +23,7 @@
import org.apache.asterix.column.bytes.decoder.ParquetPlainFixedLengthValuesReader;
import org.apache.asterix.column.bytes.stream.in.AbstractBytesInputStream;
import org.apache.asterix.om.types.ATypeTag;
+import org.apache.hyracks.util.annotations.AiProvenance;
public final class DoubleValueReader extends AbstractValueReader {
private final ParquetPlainFixedLengthValuesReader doubleReader;
@@ -48,6 +49,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ doubleReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.DOUBLE;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/FloatValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/FloatValueReader.java
index 90f76e3..5de747b 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/FloatValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/FloatValueReader.java
@@ -23,6 +23,7 @@
import org.apache.asterix.column.bytes.decoder.ParquetPlainFixedLengthValuesReader;
import org.apache.asterix.column.bytes.stream.in.AbstractBytesInputStream;
import org.apache.asterix.om.types.ATypeTag;
+import org.apache.hyracks.util.annotations.AiProvenance;
public final class FloatValueReader extends AbstractValueReader {
private final ParquetPlainFixedLengthValuesReader floatReader;
@@ -48,6 +49,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ floatReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.FLOAT;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/IntValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/IntValueReader.java
index 1307849..8efea98 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/IntValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/IntValueReader.java
@@ -24,6 +24,7 @@
import org.apache.asterix.column.bytes.stream.in.AbstractBytesInputStream;
import org.apache.asterix.om.types.ATypeTag;
import org.apache.hyracks.api.exceptions.HyracksDataException;
+import org.apache.hyracks.util.annotations.AiProvenance;
public class IntValueReader extends AbstractValueReader {
private final ParquetDeltaBinaryPackingValuesReader intReader;
@@ -51,6 +52,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ intReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return typeTag;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/LongValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/LongValueReader.java
index c22687e..57026df 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/LongValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/LongValueReader.java
@@ -23,6 +23,7 @@
import org.apache.asterix.column.bytes.decoder.ParquetDeltaBinaryPackingValuesReader;
import org.apache.asterix.column.bytes.stream.in.AbstractBytesInputStream;
import org.apache.asterix.om.types.ATypeTag;
+import org.apache.hyracks.util.annotations.AiProvenance;
public final class LongValueReader extends AbstractValueReader {
private final ParquetDeltaBinaryPackingValuesReader longReader;
@@ -48,6 +49,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ longReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.BIGINT;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/NoOpValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/NoOpValueReader.java
index 0d59499..adfd962 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/NoOpValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/NoOpValueReader.java
@@ -23,6 +23,7 @@
import org.apache.asterix.column.bytes.stream.in.AbstractBytesInputStream;
import org.apache.asterix.om.types.ATypeTag;
import org.apache.hyracks.api.exceptions.HyracksDataException;
+import org.apache.hyracks.util.annotations.AiProvenance;
public final class NoOpValueReader extends AbstractValueReader {
public static final AbstractValueReader INSTANCE = new NoOpValueReader();
@@ -41,6 +42,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ throw new UnsupportedOperationException(getClass().getName());
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.MISSING;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/StringValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/StringValueReader.java
index 19da3dd..6535da9 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/StringValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/StringValueReader.java
@@ -25,6 +25,7 @@
import org.apache.asterix.om.types.ATypeTag;
import org.apache.hyracks.data.std.api.IValueReference;
import org.apache.hyracks.data.std.primitive.UTF8StringPointable;
+import org.apache.hyracks.util.annotations.AiProvenance;
public final class StringValueReader extends AbstractValueReader {
private final ParquetDeltaByteArrayReader stringReader;
@@ -50,6 +51,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ stringReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.STRING;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/UUIDValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/UUIDValueReader.java
index a029d6f..13591f8 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/UUIDValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/UUIDValueReader.java
@@ -27,6 +27,7 @@
import org.apache.asterix.om.types.ATypeTag;
import org.apache.hyracks.data.std.api.IValueReference;
import org.apache.hyracks.data.std.util.ArrayBackedValueStorage;
+import org.apache.hyracks.util.annotations.AiProvenance;
public final class UUIDValueReader extends AbstractValueReader {
private final ParquetPlainFixedLengthValuesReader uuidReader;
@@ -54,6 +55,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ uuidReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.UUID;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/AbstractFixedLengthColumnKeyValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/AbstractFixedLengthColumnKeyValueReader.java
index c79437e..3bb23f4 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/AbstractFixedLengthColumnKeyValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/AbstractFixedLengthColumnKeyValueReader.java
@@ -27,6 +27,7 @@
import org.apache.hyracks.data.std.api.IPointable;
import org.apache.hyracks.data.std.api.IValueReference;
import org.apache.hyracks.data.std.primitive.VoidPointable;
+import org.apache.hyracks.util.annotations.AiProvenance;
public abstract class AbstractFixedLengthColumnKeyValueReader extends AbstractValueReader
implements IColumnKeyValueReader {
@@ -70,6 +71,13 @@
value.set(buffer.array(), offset, valueLength);
}
+ @Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ throw new UnsupportedOperationException(
+ "the cursor positions a primary key reader; see PrimitiveColumnValuesReader#skip");
+ }
+
protected abstract int getValueLength();
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/LevelKeyReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/LevelKeyReader.java
index dddd0fa..85613e3 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/LevelKeyReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/LevelKeyReader.java
@@ -26,6 +26,7 @@
import org.apache.asterix.om.types.ATypeTag;
import org.apache.hyracks.api.exceptions.HyracksDataException;
import org.apache.hyracks.data.std.api.IValueReference;
+import org.apache.hyracks.util.annotations.AiProvenance;
/**
* This singleton is used to ignore the key value and provide a way to read the key's level using
@@ -60,6 +61,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ // NoOp, like nextValue()
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.MISSING;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/StringKeyValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/StringKeyValueReader.java
index 17225b9..3edb449 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/StringKeyValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/key/StringKeyValueReader.java
@@ -30,6 +30,7 @@
import org.apache.hyracks.data.std.primitive.IntegerPointable;
import org.apache.hyracks.data.std.primitive.UTF8StringPointable;
import org.apache.hyracks.data.std.primitive.VoidPointable;
+import org.apache.hyracks.util.annotations.AiProvenance;
import org.apache.hyracks.util.string.UTF8StringUtil;
public final class StringKeyValueReader extends AbstractValueReader implements IColumnKeyValueReader {
@@ -85,6 +86,13 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ throw new UnsupportedOperationException(
+ "the cursor positions a primary key reader; see PrimitiveColumnValuesReader#skip");
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.STRING;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DateTimeValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DateTimeValueReader.java
index 737cfc1..3f739da 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DateTimeValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DateTimeValueReader.java
@@ -24,6 +24,7 @@
import org.apache.asterix.column.bytes.stream.in.AbstractBytesInputStream;
import org.apache.asterix.column.values.reader.value.AbstractValueReader;
import org.apache.asterix.om.types.ATypeTag;
+import org.apache.hyracks.util.annotations.AiProvenance;
public class DateTimeValueReader extends AbstractValueReader {
private final ParquetDeltaBinaryPackingValuesReader dateTimeValueReader;
@@ -49,6 +50,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ dateTimeValueReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.DATETIME;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DateValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DateValueReader.java
index 8ef6c29..c7d6383 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DateValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DateValueReader.java
@@ -25,6 +25,7 @@
import org.apache.asterix.column.values.reader.value.AbstractValueReader;
import org.apache.asterix.om.types.ATypeTag;
import org.apache.hyracks.api.exceptions.HyracksDataException;
+import org.apache.hyracks.util.annotations.AiProvenance;
public class DateValueReader extends AbstractValueReader {
private final ParquetDeltaBinaryPackingValuesReader dateReader;
@@ -50,6 +51,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ dateReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.DATE;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DayTimeDurationValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DayTimeDurationValueReader.java
index 698c0a7..0a5e6f7 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DayTimeDurationValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DayTimeDurationValueReader.java
@@ -24,6 +24,7 @@
import org.apache.asterix.column.bytes.stream.in.AbstractBytesInputStream;
import org.apache.asterix.column.values.reader.value.AbstractValueReader;
import org.apache.asterix.om.types.ATypeTag;
+import org.apache.hyracks.util.annotations.AiProvenance;
public class DayTimeDurationValueReader extends AbstractValueReader {
private final ParquetDeltaBinaryPackingValuesReader dayTimeDurationValueReader;
@@ -49,6 +50,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ dayTimeDurationValueReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.DATETIME;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DurationValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DurationValueReader.java
index 7bd5683..82bc92c 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DurationValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/DurationValueReader.java
@@ -29,6 +29,7 @@
import org.apache.hyracks.data.std.primitive.IntegerPointable;
import org.apache.hyracks.data.std.primitive.LongPointable;
import org.apache.hyracks.data.std.util.ArrayBackedValueStorage;
+import org.apache.hyracks.util.annotations.AiProvenance;
import org.apache.parquet.bytes.BytesUtils;
public class DurationValueReader extends AbstractValueReader {
@@ -69,6 +70,13 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ monthsReader.skip(count);
+ millisReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.DURATION;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/IntervalValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/IntervalValueReader.java
index d6e37b6..3d354f3 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/IntervalValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/IntervalValueReader.java
@@ -31,6 +31,7 @@
import org.apache.hyracks.data.std.primitive.IntegerPointable;
import org.apache.hyracks.data.std.primitive.LongPointable;
import org.apache.hyracks.data.std.util.ArrayBackedValueStorage;
+import org.apache.hyracks.util.annotations.AiProvenance;
import org.apache.parquet.bytes.BytesUtils;
public final class IntervalValueReader extends AbstractValueReader {
@@ -94,6 +95,14 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) throws HyracksDataException {
+ typeReader.skip(count);
+ startReader.skip(count);
+ endReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.INTERVAL;
}
@@ -123,4 +132,4 @@
long e2 = AIntervalSerializerDeserializer.getIntervalEnd(b, bo);
return Long.compare(e1, e2);
}
-}
\ No newline at end of file
+}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/TimeValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/TimeValueReader.java
index 94161e0..dbbaabd 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/TimeValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/TimeValueReader.java
@@ -25,6 +25,7 @@
import org.apache.asterix.column.values.reader.value.AbstractValueReader;
import org.apache.asterix.om.types.ATypeTag;
import org.apache.hyracks.api.exceptions.HyracksDataException;
+import org.apache.hyracks.util.annotations.AiProvenance;
public class TimeValueReader extends AbstractValueReader {
private final ParquetDeltaBinaryPackingValuesReader timeReader;
@@ -50,6 +51,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ timeReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.TIME;
}
diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/YearMonthDurationValueReader.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/YearMonthDurationValueReader.java
index 9d13e4e..b11822c 100644
--- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/YearMonthDurationValueReader.java
+++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/values/reader/value/temporal/YearMonthDurationValueReader.java
@@ -25,6 +25,7 @@
import org.apache.asterix.column.values.reader.value.AbstractValueReader;
import org.apache.asterix.om.types.ATypeTag;
import org.apache.hyracks.api.exceptions.HyracksDataException;
+import org.apache.hyracks.util.annotations.AiProvenance;
public class YearMonthDurationValueReader extends AbstractValueReader {
private final ParquetDeltaBinaryPackingValuesReader yearMonthDurationReader;
@@ -50,6 +51,12 @@
}
@Override
+ @AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.GENERATED)
+ public void skip(int count) {
+ yearMonthDurationReader.skip(count);
+ }
+
+ @Override
public ATypeTag getTypeTag() {
return ATypeTag.YEARMONTHDURATION;
}
diff --git a/asterixdb/asterix-column/src/test/java/org/apache/asterix/column/bytes/decoder/ParquetRunLengthBitPackingHybridDecoderSkipTest.java b/asterixdb/asterix-column/src/test/java/org/apache/asterix/column/bytes/decoder/ParquetRunLengthBitPackingHybridDecoderSkipTest.java
new file mode 100644
index 0000000..b196625
--- /dev/null
+++ b/asterixdb/asterix-column/src/test/java/org/apache/asterix/column/bytes/decoder/ParquetRunLengthBitPackingHybridDecoderSkipTest.java
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.column.bytes.decoder;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.util.Random;
+
+import org.apache.asterix.column.bytes.encoder.ParquetRunLengthBitPackingHybridEncoder;
+import org.apache.hyracks.util.annotations.AiProvenance;
+import org.junit.Assert;
+import org.junit.Test;
+
+/**
+ * {@link ParquetRunLengthBitPackingHybridDecoder#skipWhile(int, int)} must consume exactly the entries a sequence of
+ * {@link ParquetRunLengthBitPackingHybridDecoder#readInt()} calls would have returned as equal to the value, and
+ * leave the first differing entry in place, whichever way the encoder happened to lay the entries out.
+ */
+@AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.TEST_GENERATED)
+public class ParquetRunLengthBitPackingHybridDecoderSkipTest {
+ private static final int BIT_WIDTH = 3;
+ private static final int MAX_LEVEL = 3;
+ private static final int DELIMITER = 1;
+
+ @Test
+ public void skipsWholeRunsOfArrayLevels() throws Exception {
+ // the shape an array of 384 doubles per record produces: 384 max-level entries then a delimiter
+ int records = 1000;
+ int[] levels = new int[records * 385];
+ for (int r = 0, i = 0; r < records; r++) {
+ for (int j = 0; j < 384; j++) {
+ levels[i++] = MAX_LEVEL;
+ }
+ levels[i++] = DELIMITER;
+ }
+ ParquetRunLengthBitPackingHybridDecoder decoder = decode(levels);
+ for (int r = 0; r < records; r++) {
+ Assert.assertEquals("record " + r, 384, decoder.skipWhile(MAX_LEVEL, Integer.MAX_VALUE));
+ Assert.assertEquals("delimiter of record " + r, 0, decoder.skipWhile(MAX_LEVEL, Integer.MAX_VALUE));
+ Assert.assertEquals("delimiter of record " + r, DELIMITER, decoder.readInt());
+ }
+ }
+
+ @Test
+ public void honoursTheLimitInsideARun() throws Exception {
+ int[] levels = new int[500];
+ java.util.Arrays.fill(levels, MAX_LEVEL);
+ levels[499] = DELIMITER;
+ ParquetRunLengthBitPackingHybridDecoder decoder = decode(levels);
+ Assert.assertEquals(10, decoder.skipWhile(MAX_LEVEL, 10));
+ Assert.assertEquals(MAX_LEVEL, decoder.readInt());
+ Assert.assertEquals(488, decoder.skipWhile(MAX_LEVEL, 1000));
+ Assert.assertEquals(DELIMITER, decoder.readInt());
+ }
+
+ @Test
+ public void stopsAtTheFirstDifferentEntryInBitPackedGroups() throws Exception {
+ // alternating values never form a run, so the encoder bit-packs everything
+ int[] levels = new int[64];
+ for (int i = 0; i < levels.length; i++) {
+ levels[i] = i % 2 == 0 ? MAX_LEVEL : DELIMITER;
+ }
+ ParquetRunLengthBitPackingHybridDecoder decoder = decode(levels);
+ for (int i = 0; i < levels.length; i += 2) {
+ Assert.assertEquals(1, decoder.skipWhile(MAX_LEVEL, 100));
+ Assert.assertEquals(0, decoder.skipWhile(MAX_LEVEL, 100));
+ Assert.assertEquals(DELIMITER, decoder.readInt());
+ }
+ }
+
+ @Test
+ public void returnsZeroAtTheEnd() throws Exception {
+ ParquetRunLengthBitPackingHybridDecoder decoder = decode(new int[] { MAX_LEVEL, MAX_LEVEL });
+ Assert.assertEquals(2, decoder.skipWhile(MAX_LEVEL, 5));
+ Assert.assertEquals(0, decoder.skipWhile(MAX_LEVEL, 5));
+ }
+
+ @Test
+ public void interleavedSkipsAndReadsAgreeWithSequentialReads() throws Exception {
+ Random random = new Random(20260920);
+ for (int round = 0; round < 200; round++) {
+ int[] levels = randomLevels(random);
+ ParquetRunLengthBitPackingHybridDecoder decoder = decode(levels);
+ int position = 0;
+ while (position < levels.length) {
+ int action = random.nextInt(3);
+ if (action == 0) {
+ int n = random.nextInt(Math.min(50, levels.length - position) + 1);
+ decoder.skip(n);
+ position += n;
+ } else if (action == 1) {
+ int value = random.nextInt(MAX_LEVEL + 1);
+ // bounded by the remaining entries, as readers bound it by their value count
+ int limit = Math.min(random.nextInt(50), levels.length - position);
+ int expected = 0;
+ while (expected < limit && position + expected < levels.length
+ && levels[position + expected] == value) {
+ expected++;
+ }
+ Assert.assertEquals("round " + round + " at " + position, expected,
+ decoder.skipWhile(value, limit));
+ position += expected;
+ } else {
+ Assert.assertEquals("round " + round + " at " + position, levels[position++], decoder.readInt());
+ }
+ }
+ }
+ }
+
+ private static int[] randomLevels(Random random) {
+ // runs of random length, short enough to mix run-length runs with bit-packed groups
+ int size = 1 + random.nextInt(3000);
+ int[] levels = new int[size];
+ int i = 0;
+ while (i < size) {
+ int value = random.nextInt(MAX_LEVEL + 1);
+ int run = 1 + random.nextInt(random.nextBoolean() ? 3 : 60);
+ for (int j = 0; j < run && i < size; j++) {
+ levels[i++] = value;
+ }
+ }
+ return levels;
+ }
+
+ private static ParquetRunLengthBitPackingHybridDecoder decode(int[] levels) throws Exception {
+ ParquetRunLengthBitPackingHybridEncoder encoder = new ParquetRunLengthBitPackingHybridEncoder(BIT_WIDTH);
+ for (int level : levels) {
+ encoder.writeInt(level);
+ }
+ ByteArrayOutputStream bos = new ByteArrayOutputStream();
+ encoder.toBytes().writeAllTo(bos);
+ ParquetRunLengthBitPackingHybridDecoder decoder = new ParquetRunLengthBitPackingHybridDecoder(BIT_WIDTH);
+ decoder.reset(new ByteArrayInputStream(bos.toByteArray()));
+ return decoder;
+ }
+}
diff --git a/asterixdb/asterix-column/src/test/java/org/apache/asterix/column/test/skip/ColumnSkipTest.java b/asterixdb/asterix-column/src/test/java/org/apache/asterix/column/test/skip/ColumnSkipTest.java
new file mode 100644
index 0000000..45613c5
--- /dev/null
+++ b/asterixdb/asterix-column/src/test/java/org/apache/asterix/column/test/skip/ColumnSkipTest.java
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.column.test.skip;
+
+import static org.apache.hyracks.storage.am.lsm.btree.column.impls.btree.AbstractColumnBTreeLeafFrame.TUPLE_COUNT_OFFSET;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.nio.file.Files;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import java.util.Random;
+
+import org.apache.asterix.column.assembler.value.ValueGetterFactory;
+import org.apache.asterix.column.bytes.stream.in.MultiByteBufferInputStream;
+import org.apache.asterix.column.common.buffer.DummyPage;
+import org.apache.asterix.column.common.test.TestCase;
+import org.apache.asterix.column.filter.NoOpColumnFilterEvaluatorFactory;
+import org.apache.asterix.column.operation.lsm.flush.FlushColumnMetadata;
+import org.apache.asterix.column.operation.query.ColumnAssembler;
+import org.apache.asterix.column.operation.query.QueryColumnMetadata;
+import org.apache.asterix.column.test.bytes.AbstractBytesTest;
+import org.apache.asterix.column.test.bytes.components.TestColumnBufferProvider;
+import org.apache.asterix.column.values.reader.ColumnValueReaderFactory;
+import org.apache.asterix.common.exceptions.NoOpWarningCollector;
+import org.apache.asterix.om.utils.ProjectionFiltrationTypeUtil;
+import org.apache.hyracks.api.exceptions.HyracksDataException;
+import org.apache.hyracks.data.std.api.IValueReference;
+import org.apache.hyracks.storage.am.lsm.btree.column.api.projection.ColumnProjectorType;
+import org.apache.hyracks.util.annotations.AiProvenance;
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
+
+/**
+ * Positioning a mega leaf's columns at tuple k by skipping must yield the same tuple as reading the k tuples before
+ * it. Every corpus in the data directory goes through the real flush path, so the readers see the level streams the
+ * writer really produces: arrays of one type, arrays mixing types (one column per type, missing entries for the
+ * others), null and missing items, empty arrays and nested arrays. A synthetic corpus adds embedding-sized arrays of
+ * doubles with integer and null components, the case that motivated the bulk skip.
+ */
+@RunWith(Parameterized.class)
+@AiProvenance(agent = AiProvenance.Agent.CLAUDE_FABLE_5_1, tool = AiProvenance.Tool.CLAUDE_CODE_UI, contributionKind = AiProvenance.ContributionKind.TEST_GENERATED)
+public class ColumnSkipTest extends AbstractBytesTest {
+ private static final int REPEAT = 5;
+ private static final int EMBEDDING_DIMENSIONS = 384;
+ private static final int EMBEDDING_RECORDS = 90;
+
+ public ColumnSkipTest(TestCase testCase) throws HyracksDataException {
+ super(testCase);
+ }
+
+ @Parameters(name = "SkipTest {index}: {0}")
+ public static Collection<Object[]> tests() throws Exception {
+ List<Object[]> cases = new ArrayList<>();
+ for (File file : listFiles(DATA_PATH)) {
+ cases.add(new Object[] { new TestCase(file, file, OUTPUT_PATH) });
+ }
+ File embeddings = writeEmbeddingCorpus();
+ cases.add(new Object[] { new TestCase(embeddings, embeddings, OUTPUT_PATH) });
+ return cases;
+ }
+
+ @Test
+ public void skipLandsOnTheSameTupleAsSequentialReads() throws IOException {
+ int fileId = createFile();
+ FlushColumnMetadata columnMetadata = prepareNewFile(fileId);
+ List<IValueReference> records = getParsedRecords();
+ List<DummyPage> pageZeros = transform(fileId, columnMetadata, records, records.size() * REPEAT);
+ QueryColumnMetadata readMetadata = QueryColumnMetadata.create(columnMetadata.getDatasetType(),
+ columnMetadata.getNumberOfPrimaryKeys(), columnMetadata.serializeColumnsMetadata(),
+ new ColumnValueReaderFactory(), ValueGetterFactory.INSTANCE,
+ ProjectionFiltrationTypeUtil.ALL_FIELDS_TYPE, Collections.emptyMap(),
+ NoOpColumnFilterEvaluatorFactory.INSTANCE, NoOpColumnFilterEvaluatorFactory.INSTANCE,
+ NoOpWarningCollector.INSTANCE, null, ColumnProjectorType.QUERY);
+ ColumnAssembler assembler = readMetadata.getAssembler();
+ int numberOfColumns = assembler.getNumberOfColumns();
+ TestColumnBufferProvider[] providers = new TestColumnBufferProvider[numberOfColumns];
+ MultiByteBufferInputStream[] streams = new MultiByteBufferInputStream[numberOfColumns];
+ for (int i = 0; i < numberOfColumns; i++) {
+ providers[i] = new TestColumnBufferProvider(fileId, assembler.getColumnIndex(i), dummyBufferCache);
+ streams[i] = new MultiByteBufferInputStream();
+ }
+
+ Random random = new Random(20260920);
+ int pageNumber = 0;
+ for (DummyPage pageZero : pageZeros) {
+ int tupleCount = reset(pageZero, assembler, providers, streams);
+ List<byte[]> expected = new ArrayList<>(tupleCount);
+ while (assembler.hasNext()) {
+ expected.add(copy(assembler.nextValue()));
+ }
+ Assert.assertEquals("page " + pageNumber, tupleCount, expected.size());
+
+ for (int k = 0; k < tupleCount; k++) {
+ String where = "page " + pageNumber + " tuple " + k;
+ // one skip straight to k, the way a point lookup positions the cursor
+ reset(pageZero, assembler, providers, streams);
+ assembler.skip(k);
+ Assert.assertArrayEquals(where, expected.get(k), copy(assembler.nextValue()));
+ if (k + 1 < tupleCount) {
+ // the tuple after the target must still assemble correctly
+ Assert.assertArrayEquals(where + " successor", expected.get(k + 1), copy(assembler.nextValue()));
+ }
+ // the same position reached by reading some tuples and skipping the rest
+ reset(pageZero, assembler, providers, streams);
+ int read = random.nextInt(k + 1);
+ for (int i = 0; i < read; i++) {
+ assembler.nextValue();
+ }
+ assembler.skip(k - read);
+ Assert.assertArrayEquals(where + " after reading " + read, expected.get(k),
+ copy(assembler.nextValue()));
+ }
+ pageNumber++;
+ }
+ }
+
+ private static int reset(DummyPage pageZero, ColumnAssembler assembler, TestColumnBufferProvider[] providers,
+ MultiByteBufferInputStream[] streams) throws HyracksDataException {
+ for (int i = 0; i < providers.length; i++) {
+ providers[i].reset(pageZero);
+ streams[i].reset(providers[i]);
+ }
+ int tupleCount = pageZero.getBuffer().getInt(TUPLE_COUNT_OFFSET);
+ assembler.reset(tupleCount);
+ for (int i = 0; i < streams.length; i++) {
+ assembler.resetColumn(streams[i], i);
+ }
+ return tupleCount;
+ }
+
+ private static byte[] copy(IValueReference value) {
+ byte[] bytes = new byte[value.getLength()];
+ System.arraycopy(value.getByteArray(), value.getStartOffset(), bytes, 0, bytes.length);
+ return bytes;
+ }
+
+ /**
+ * Embedding-like records: arrays of 384 doubles where some components print as integers (so the item column is
+ * a union of a double column and an integer column), some are null, plus an empty array, a missing field and a
+ * null field.
+ */
+ private static File writeEmbeddingCorpus() throws IOException {
+ File dir = new File(OUTPUT_PATH, "skip");
+ Files.createDirectories(dir.toPath());
+ File file = new File(dir, "embeddings.json");
+ Random random = new Random(42);
+ try (PrintWriter out = new PrintWriter(file)) {
+ for (int r = 0; r < EMBEDDING_RECORDS; r++) {
+ StringBuilder sb = new StringBuilder();
+ sb.append("{\"id\": ").append(r);
+ if (r % 29 == 7) {
+ // field missing entirely
+ } else if (r % 29 == 15) {
+ sb.append(", \"emb\": null");
+ } else if (r % 29 == 22) {
+ sb.append(", \"emb\": []");
+ } else {
+ sb.append(", \"emb\": [");
+ for (int d = 0; d < EMBEDDING_DIMENSIONS; d++) {
+ if (d > 0) {
+ sb.append(", ");
+ }
+ if (r % 5 == 0 && d % 97 == 3) {
+ sb.append(random.nextInt(3));
+ } else if (r % 7 == 0 && d % 131 == 5) {
+ sb.append("null");
+ } else {
+ sb.append(String.format("%.6f", random.nextGaussian()));
+ }
+ }
+ sb.append(']');
+ }
+ sb.append('}');
+ out.println(sb);
+ }
+ }
+ return file;
+ }
+}
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21854?usp=email
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email
Gerrit-MessageType: newchange
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I370fe0f7ba9b607d6f2a027177c985f0e898da55
Gerrit-Change-Number: 21854
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <al...@gmail.com>