You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Priyesh K (Jira)" <ji...@apache.org> on 2026/07/08 10:20:00 UTC
[jira] [Updated] (HDDS-15783) ScmListCodec fails to deserialize empty lists from Ratis log due to java.lang.Object element type regression
[ https://issues.apache.org/jira/browse/HDDS-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Priyesh K updated HDDS-15783:
-----------------------------
Description:
During an SCM HA cluster upgrade, SCM fails while replaying the Ratis log with the following error:
{code:java}
2026-07-05 10:00:12,699 ERROR [36209121-a632-4067-a36d-a6e0194cd83e@group-65230FEE3C3A-StateMachineUpdater]-org.apache.ratis.statemachine.StateMachine: Terminating with exit status 1: Failed to resolve java.lang.Objectorg.apache.ratis.thirdparty.com.google.protobuf.InvalidProtocolBufferException: Failed to resolve java.lang.Object at org.apache.hadoop.hdds.scm.ha.io.ScmCodecFactory$ClassResolver.get(ScmCodecFactory.java:177) at org.apache.hadoop.hdds.scm.ha.io.ScmCodecFactory$ClassResolver.get(ScmCodecFactory.java:147) at org.apache.hadoop.hdds.scm.ha.io.ScmListCodec.deserialize(ScmListCodec.java:72) at org.apache.hadoop.hdds.scm.ha.SCMRatisRequest.decode(SCMRatisRequest.java:154) at org.apache.hadoop.hdds.scm.ha.SCMStateMachine.applyTransaction(SCMStateMachine.java:146) at org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1888) at org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:255) at org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:194) at java.base/java.lang.Thread.run(Thread.java:833)2026-07-05 10:00:12,702 INFO [shutdown-hook-0]-org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter: SHUTDOWN_MSG: {code}
*Root cause*
An empty List/ArrayList was written into the SCM Ratis log, and the new ScmListCodec cannot deserialize it.On serialize, empty lists are stored with element type java.lang.Object. On deserialize, the codec always tries to resolve that type and get an element codec — even when there are zero elements.This is a regression from the reflection-based codec refactor (HDDS-14740/[HDDS-14846|https://issues.apache.org/jira/browse/HDDS-14846]). The old ListCodec had the same Object.class sentinel on serialize, but on deserialize it only loaded the element codec inside the value loop
was:
During an SCM HA cluster upgrade, SCM fails while replaying the Ratis log with the following error:
{code:java}
2026-07-05 10:00:12,699 ERROR [36209121-a632-4067-a36d-a6e0194cd83e@group-65230FEE3C3A-StateMachineUpdater]-org.apache.ratis.statemachine.StateMachine: Terminating with exit status 1: Failed to resolve java.lang.Objectorg.apache.ratis.thirdparty.com.google.protobuf.InvalidProtocolBufferException: Failed to resolve java.lang.Object at org.apache.hadoop.hdds.scm.ha.io.ScmCodecFactory$ClassResolver.get(ScmCodecFactory.java:177) at org.apache.hadoop.hdds.scm.ha.io.ScmCodecFactory$ClassResolver.get(ScmCodecFactory.java:147) at org.apache.hadoop.hdds.scm.ha.io.ScmListCodec.deserialize(ScmListCodec.java:72) at org.apache.hadoop.hdds.scm.ha.SCMRatisRequest.decode(SCMRatisRequest.java:154) at org.apache.hadoop.hdds.scm.ha.SCMStateMachine.applyTransaction(SCMStateMachine.java:146) at org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1888) at org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:255) at org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:194) at java.base/java.lang.Thread.run(Thread.java:833)2026-07-05 10:00:12,702 INFO [shutdown-hook-0]-org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter: SHUTDOWN_MSG: {code}
*Root cause*
An empty List/ArrayList was written into the SCM Ratis log, and the new ScmListCodec cannot deserialize it.On serialize, empty lists are stored with element type java.lang.Object. On deserialize, the codec always tries to resolve that type and get an element codec — even when there are zero elements.This is a regression from the reflection-based codec refactor ([HDDS-14740 |https://issues.apache.org/jira/browse/HDDS-14740 ]/ [HDDS-14846|
https://issues.apache.org/jira/browse/HDDS-14846
]). The old ListCodec had the same Object.class sentinel on serialize, but on deserialize it only loaded the element codec inside the value loop
> ScmListCodec fails to deserialize empty lists from Ratis log due to java.lang.Object element type regression
> ------------------------------------------------------------------------------------------------------------
>
> Key: HDDS-15783
> URL: https://issues.apache.org/jira/browse/HDDS-15783
> Project: Apache Ozone
> Issue Type: Bug
> Components: SCM, SCM HA
> Reporter: Priyesh K
> Assignee: Priyesh K
> Priority: Major
>
> During an SCM HA cluster upgrade, SCM fails while replaying the Ratis log with the following error:
> {code:java}
> 2026-07-05 10:00:12,699 ERROR [36209121-a632-4067-a36d-a6e0194cd83e@group-65230FEE3C3A-StateMachineUpdater]-org.apache.ratis.statemachine.StateMachine: Terminating with exit status 1: Failed to resolve java.lang.Objectorg.apache.ratis.thirdparty.com.google.protobuf.InvalidProtocolBufferException: Failed to resolve java.lang.Object at org.apache.hadoop.hdds.scm.ha.io.ScmCodecFactory$ClassResolver.get(ScmCodecFactory.java:177) at org.apache.hadoop.hdds.scm.ha.io.ScmCodecFactory$ClassResolver.get(ScmCodecFactory.java:147) at org.apache.hadoop.hdds.scm.ha.io.ScmListCodec.deserialize(ScmListCodec.java:72) at org.apache.hadoop.hdds.scm.ha.SCMRatisRequest.decode(SCMRatisRequest.java:154) at org.apache.hadoop.hdds.scm.ha.SCMStateMachine.applyTransaction(SCMStateMachine.java:146) at org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1888) at org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:255) at org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:194) at java.base/java.lang.Thread.run(Thread.java:833)2026-07-05 10:00:12,702 INFO [shutdown-hook-0]-org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter: SHUTDOWN_MSG: {code}
> *Root cause*
> An empty List/ArrayList was written into the SCM Ratis log, and the new ScmListCodec cannot deserialize it.On serialize, empty lists are stored with element type java.lang.Object. On deserialize, the codec always tries to resolve that type and get an element codec — even when there are zero elements.This is a regression from the reflection-based codec refactor (HDDS-14740/[HDDS-14846|https://issues.apache.org/jira/browse/HDDS-14846]). The old ListCodec had the same Object.class sentinel on serialize, but on deserialize it only loaded the element codec inside the value loop
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org