You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/09/11 03:26:00 UTC

[jira] [Updated] (SPARK-59430) Cache UTF8String.hashCode

     [ https://issues.apache.org/jira/browse/SPARK-59430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-59430:
-----------------------------------
    Labels: pull-request-available  (was: )

> Cache UTF8String.hashCode
> -------------------------
>
>                 Key: SPARK-59430
>                 URL: https://issues.apache.org/jira/browse/SPARK-59430
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 4.1.0
>            Reporter: David Mollitor
>            Priority: Minor
>              Labels: pull-request-available
>
> h2. Summary
> {{UTF8String.hashCode()}} recomputes a Murmur3 hash over *all* of the string's bytes on every call:
> {code:java}
> public int hashCode() {
>   return Murmur3_x86_32.hashUnsafeBytes(base, offset, numBytes, 42);
> }
> {code}
> For a {{UTF8String}} that is hashed more than once -- e.g. a key in a Scala {{Map}}/{{Set}}, or the repeated {{contains}}/{{apply}}/{{update}} probes against an aggregation buffer -- that repeats an O(length) scan each time. {{java.lang.String}} avoids this by caching its hash in a field; this does the same for {{UTF8String}}.
> h2. Change
> Cache the hash in a non-volatile {{int}}, using the modern {{java.lang.String}} technique so a
> *legitimately* zero hash is remembered rather than recomputed on every call



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org