You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "David Mollitor (Jira)" <ji...@apache.org> on 2026/10/03 03:05:00 UTC
[jira] [Created] (SPARK-59969) Skip the redundant bounds check on internal UTF8String byte reads
David Mollitor created SPARK-59969:
--------------------------------------
Summary: Skip the redundant bounds check on internal UTF8String byte reads
Key: SPARK-59969
URL: https://issues.apache.org/jira/browse/SPARK-59969
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.2.0
Reporter: David Mollitor
{{UTF8String.getByte(int)}} checks that the index is in {{[0, numBytes)}} and returns 0 otherwise. {{UTF8String}} also calls it internally from the loops behind {{numChars}}, {{isValid}} / {{makeValid}}, {{isFullAscii}}, {{substring}}, {{charPosToByte}} / {{bytePosToChar}}, {{indexOf}}, {{reverse}}, the {{trim}} family, {{toInt}} / {{toLong}}, ASCII case conversion, {{findInSet}}, {{soundex}} and the code point iterators. In those loops the loop condition already keeps the index in range, so the check is redundant. The JIT can't always remove it, especially in character walks, where each step advances by the width of the current character.
This adds a private {{getByteUnchecked(int)}}, which asserts that the index is in range (Spark tests run with {{-ea}}), and makes {{getByte}} call it after its own check. The 45 internal call sites whose loop provably keeps the index in range switch to {{getByteUnchecked}}. The other 11 stay on {{getByte}} because they can index past the end, or depend on an invariant outside the loop:
* {{contains}}, which SPARK-59967 replaces
* {{getChar}}
* {{trimTrailingSpaces}}
* {{prevCharStart}}
* one read in {{indexOf(pattern, start, occurrence)}} that can run past the end on truncated UTF-8
* the walks in {{levenshteinDistance}}
The behavior of the public {{getByte}} does not change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org