You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2021/02/08 14:46:25 UTC
[spark] branch master updated: [MINOR][SQL][FOLLOW-UP] Add
assertion to FixedLengthRowBasedKeyValueBatch
This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new d1131bc [MINOR][SQL][FOLLOW-UP] Add assertion to FixedLengthRowBasedKeyValueBatch
d1131bc is described below
commit d1131bc85028ea0f78ac9ef73bba731080f1ff6a
Author: yliou <yl...@berkeley.edu>
AuthorDate: Mon Feb 8 08:46:01 2021 -0600
[MINOR][SQL][FOLLOW-UP] Add assertion to FixedLengthRowBasedKeyValueBatch
### What changes were proposed in this pull request?
Adds an assert to `FixedLengthRowBasedKeyValueBatch#appendRow` method to check the incoming vlen and klen by comparing them with the lengths stored as member variables as followup to https://github.com/apache/spark/pull/30788
### Why are the changes needed?
Add assert statement to catch similar bugs in future.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Ran some tests locally, though not easy to test.
Closes #31447 from yliou/SPARK-33726-Assert.
Authored-by: yliou <yl...@berkeley.edu>
Signed-off-by: Sean Owen <sr...@gmail.com>
---
.../sql/catalyst/expressions/FixedLengthRowBasedKeyValueBatch.java | 2 ++
1 file changed, 2 insertions(+)
diff --git a/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/FixedLengthRowBasedKeyValueBatch.java b/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/FixedLengthRowBasedKeyValueBatch.java
index df52f9c..25400be 100644
--- a/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/FixedLengthRowBasedKeyValueBatch.java
+++ b/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/FixedLengthRowBasedKeyValueBatch.java
@@ -46,6 +46,8 @@ public final class FixedLengthRowBasedKeyValueBatch extends RowBasedKeyValueBatc
public UnsafeRow appendRow(Object kbase, long koff, int klen,
Object vbase, long voff, int vlen) {
// if run out of max supported rows or page size, return null
+ assert(vlen == this.vlen);
+ assert(klen == this.klen);
if (numRows >= capacity || page == null || page.size() - pageCursor < recordLength) {
return null;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org