You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@amoro.apache.org by "lintingbin (via GitHub)" <gi...@apache.org> on 2024/10/16 06:57:34 UTC
[I] [Bug]: Upgrading the iceberg version can cause AMS startup to fail when processes are present. [amoro]
lintingbin opened a new issue, #3261:
URL: https://github.com/apache/amoro/issues/3261
### What happened?
Upgrading the iceberg version can cause AMS startup to fail when processes are present.
### Affects Versions
0.7.0
### What table formats are you seeing the problem on?
Iceberg
### What engines are you seeing the problem on?
AMS
### How to reproduce
Upgrading the iceberg version while retaining the process prior to the upgrade can cause AMS to fail to start when using the higher version of iceberg.
### Relevant log output
```shell
2024-10-16 02:33:18,933 ERROR [main] [org.apache.amoro.server.AmoroServiceContainer] [] - AMS start error
org.apache.amoro.server.exception.PersistenceException: org.apache.ibatis.exceptions.PersistenceException:
### Error querying database. Cause: java.lang.IllegalArgumentException: deserialization error
### The error may exist in org/apache/amoro/server/persistence/mapper/OptimizingMapper.java (best guess)
### The error may involve org.apache.amoro.server.persistence.mapper.OptimizingMapper.selectTaskRuntimes
### The error occurred while handling results
### SQL: SELECT process_id, task_id, retry_num, table_id, partition_data, create_time, start_time, end_time, status, fail_reason, optimizer_token, thread_id, rewrite_output, metrics_summary, properties FROM task_runtime WHERE table_id = ? AND process_id = ?
### Cause: java.lang.IllegalArgumentException: deserialization error
at org.apache.amoro.server.exception.AmoroRuntimeException.wrap(AmoroRuntimeException.java:141) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.persistence.PersistentBase.getAs(PersistentBase.java:92) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.optimizing.OptimizingQueue.access$500(OptimizingQueue.java:73) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.optimizing.OptimizingQueue$TableOptimizingProcess.loadTaskRuntimes(OptimizingQueue.java:674) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.optimizing.OptimizingQueue$TableOptimizingProcess.<init>(OptimizingQueue.java:406) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.optimizing.OptimizingQueue.initTableRuntime(OptimizingQueue.java:116) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at java.util.ArrayList.forEach(ArrayList.java:1259) ~[?:1.8.0_422]
at org.apache.amoro.server.optimizing.OptimizingQueue.<init>(OptimizingQueue.java:109) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.DefaultOptimizingService.lambda$loadOptimizingQueues$0(DefaultOptimizingService.java:142) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at java.util.ArrayList.forEach(ArrayList.java:1259) ~[?:1.8.0_422]
at org.apache.amoro.server.DefaultOptimizingService.loadOptimizingQueues(DefaultOptimizingService.java:132) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.DefaultOptimizingService.access$300(DefaultOptimizingService.java:88) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.DefaultOptimizingService$TableRuntimeHandlerImpl.initHandler(DefaultOptimizingService.java:462) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.table.RuntimeHandlerChain.initialize(RuntimeHandlerChain.java:60) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.table.DefaultTableService.initialize(DefaultTableService.java:324) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.AmoroServiceContainer.startService(AmoroServiceContainer.java:158) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.AmoroServiceContainer.main(AmoroServiceContainer.java:116) [amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
Caused by: org.apache.ibatis.exceptions.PersistenceException:
### Error querying database. Cause: java.lang.IllegalArgumentException: deserialization error
### The error may exist in org/apache/amoro/server/persistence/mapper/OptimizingMapper.java (best guess)
### The error may involve org.apache.amoro.server.persistence.mapper.OptimizingMapper.selectTaskRuntimes
### The error occurred while handling results
### SQL: SELECT process_id, task_id, retry_num, table_id, partition_data, create_time, start_time, end_time, status, fail_reason, optimizer_token, thread_id, rewrite_output, metrics_summary, properties FROM task_runtime WHERE table_id = ? AND process_id = ?
### Cause: java.lang.IllegalArgumentException: deserialization error
at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:156) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:147) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:142) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:147) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:80) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.binding.MapperProxy$PlainMethodInvoker.invoke(MapperProxy.java:141) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:86) ~[mybatis-3.5.15.jar:3.5.15]
at com.sun.proxy.$Proxy48.selectTaskRuntimes(Unknown Source) ~[?:?]
at org.apache.amoro.server.optimizing.OptimizingQueue$TableOptimizingProcess.lambda$loadTaskRuntimes$13(OptimizingQueue.java:677) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.persistence.PersistentBase.getAs(PersistentBase.java:90) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
... 15 more
Caused by: java.lang.IllegalArgumentException: deserialization error
at org.apache.amoro.utils.SerializationUtil.simpleDeserialize(SerializationUtil.java:68) ~[amoro-core-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.persistence.converter.Object2ByteArrayConvert.getResult(Object2ByteArrayConvert.java:61) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.getPropertyMappingValue(DefaultResultSetHandler.java:530) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.applyPropertyMappings(DefaultResultSetHandler.java:495) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.getRowValue(DefaultResultSetHandler.java:418) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.handleRowValuesForSimpleResultMap(DefaultResultSetHandler.java:366) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.handleRowValues(DefaultResultSetHandler.java:337) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.handleResultSet(DefaultResultSetHandler.java:310) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.handleResultSets(DefaultResultSetHandler.java:202) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:66) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:80) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:65) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:336) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:158) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:110) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:90) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:154) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:147) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:142) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:147) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:80) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.binding.MapperProxy$PlainMethodInvoker.invoke(MapperProxy.java:141) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:86) ~[mybatis-3.5.15.jar:3.5.15]
at com.sun.proxy.$Proxy48.selectTaskRuntimes(Unknown Source) ~[?:?]
at org.apache.amoro.server.optimizing.OptimizingQueue$TableOptimizingProcess.lambda$loadTaskRuntimes$13(OptimizingQueue.java:677) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.persistence.PersistentBase.getAs(PersistentBase.java:90) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
... 15 more
Caused by: java.io.InvalidClassException: org.apache.iceberg.BaseFile; local class incompatible: stream classdesc serialVersionUID = -5355849892016662001, local class serialVersionUID = 4493876333706690896
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:699) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2005) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1852) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2005) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1852) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2186) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1669) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readArray(ObjectInputStream.java:2119) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1657) ~[?:1.8.0_422]
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2431) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2355) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2213) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1669) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:503) ~[?:1.8.0_422]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:461) ~[?:1.8.0_422]
at org.apache.amoro.utils.SerializationUtil.simpleDeserialize(SerializationUtil.java:65) ~[amoro-core-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.persistence.converter.Object2ByteArrayConvert.getResult(Object2ByteArrayConvert.java:61) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.getPropertyMappingValue(DefaultResultSetHandler.java:530) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.applyPropertyMappings(DefaultResultSetHandler.java:495) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.getRowValue(DefaultResultSetHandler.java:418) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.handleRowValuesForSimpleResultMap(DefaultResultSetHandler.java:366) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.handleRowValues(DefaultResultSetHandler.java:337) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.handleResultSet(DefaultResultSetHandler.java:310) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.handleResultSets(DefaultResultSetHandler.java:202) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:66) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:80) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:65) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:336) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:158) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:110) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:90) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:154) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:147) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:142) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:147) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:80) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.binding.MapperProxy$PlainMethodInvoker.invoke(MapperProxy.java:141) ~[mybatis-3.5.15.jar:3.5.15]
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:86) ~[mybatis-3.5.15.jar:3.5.15]
at com.sun.proxy.$Proxy48.selectTaskRuntimes(Unknown Source) ~[?:?]
at org.apache.amoro.server.optimizing.OptimizingQueue$TableOptimizingProcess.lambda$loadTaskRuntimes$13(OptimizingQueue.java:677) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
at org.apache.amoro.server.persistence.PersistentBase.getAs(PersistentBase.java:90) ~[amoro-ams-server-0.7.0-incubating.jar:0.7.0-incubating]
... 15 more
```
### Anything else
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's Code of Conduct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Upgrading the iceberg version can cause AMS startup to fail when processes are present. [amoro]
Posted by "klion26 (via GitHub)" <gi...@apache.org>.
klion26 commented on issue #3261:
URL: https://github.com/apache/amoro/issues/3261#issuecomment-2415972239
The fix moves the code into the try block, does this fix the problem described here? from the stack given above, it seems a class incompatible problem, could you please check the serialVersionUID of both version about BaseFile ?
`org.apache.iceberg.BaseFile; local class incompatible: stream classdesc serialVersionUID = -5355849892016662001, local class serialVersionUID = 4493876333706690896`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Upgrading the iceberg version can cause AMS startup to fail when processes are present. [amoro]
Posted by "zhoujinsong (via GitHub)" <gi...@apache.org>.
zhoujinsong closed issue #3261: [Bug]: Upgrading the iceberg version can cause AMS startup to fail when processes are present.
URL: https://github.com/apache/amoro/issues/3261
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Upgrading the iceberg version can cause AMS startup to fail when processes are present. [amoro]
Posted by "lintingbin (via GitHub)" <gi...@apache.org>.
lintingbin commented on issue #3261:
URL: https://github.com/apache/amoro/issues/3261#issuecomment-2416053092
> The fix moves the code into the try block, does this fix the problem described here? from the stack given above, it seems a class incompatible problem, could you please check the serialVersionUID of both version about BaseFile ?
>
> `org.apache.iceberg.BaseFile; local class incompatible: stream classdesc serialVersionUID = -5355849892016662001, local class serialVersionUID = 4493876333706690896`
The org.apache.iceberg.BaseFile does not define a serialVersionUID, so the serialVersionUID compiled from different versions of iceberg may be different. Currently, deserialization failure will throw an IllegalArgumentException, and catching the IllegalArgumentException should be able to solve this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org