You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Rui Fan (Jira)" <ji...@apache.org> on 2025/07/28 10:38:00 UTC
[jira] [Assigned] (FLINK-38137) RocksDB State Backend Null Serialization Causes NPE and Asymmetric (De)Serialization Logic
[ https://issues.apache.org/jira/browse/FLINK-38137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rui Fan reassigned FLINK-38137:
-------------------------------
Assignee: Rui Fan
> RocksDB State Backend Null Serialization Causes NPE and Asymmetric (De)Serialization Logic
> ------------------------------------------------------------------------------------------
>
> Key: FLINK-38137
> URL: https://issues.apache.org/jira/browse/FLINK-38137
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Reporter: Ramin Gharib
> Assignee: Rui Fan
> Priority: Major
> Labels: pull-request-available
>
> The RocksDB state backend has a critical flaw in its handling of null values, which can cause NullPointerExceptions and create unnecessary serialization overhead.
> h3. Problem Description
> In [AbstractRocksDBState, the serializeValueNullSensitive()|https://github.com/raminqaf/flink/blob/main/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/state/rocksdb/AbstractRocksDBState.java#L179-L179] method unconditionally calls the serializer even when the value is null:
> {code:java}
> <T> byte[] serializeValueNullSensitive(T value, TypeSerializer<T> serializer)
> throws IOException {
> dataOutputView.clear();
> dataOutputView.writeBoolean(value == null); // Write null flag
> return serializeValueInternal(value, serializer); // Always serialize, even if null
> }
> private <T> byte[] serializeValueInternal(T value, TypeSerializer<T> serializer)
> throws IOException {
> serializer.serialize(value, dataOutputView); // Can throw NPE if value is null
> return dataOutputView.getCopyOfBuffer();
> } {code}
> This design has two major flaws:
> # NPE Risk: The behavior becomes dependent on the TypeSerializer implementation. Serializers not designed to handle null inputs will throw NullPointerException.
> # Asymmetric Logic: There's a critical mismatch between serialization and deserialization:
> *
> ** Serialization: Writes null flag + always attempts to serialize the value object (even if null)
> ** Deserialization: Reads null flag + immediately returns null without attempting deserialization if flag is true
> h3. Evidence from Deserialization Code
> In [RocksDBMapState|https://github.com/apache/flink/blob/bf1cd860617f7b51ac91516814c0e931e5bba241/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/state/rocksdb/RocksDBMapState.java#L413], the deserialization logic shows this asymmetry:
> {code:java}
> private static <UV> UV deserializeValueNullSensitive(
> byte[] rawValueBytes,
> TypeSerializer<UV> valueSerializer)
> throws IOException {
> dataInputView.setBuffer(rawValueBytes);
> boolean isNull = dataInputView.readBoolean();
> return isNull ? null : valueSerializer.deserialize(dataInputView); // Never deserializes if null
> } {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)