You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "SONALI SAKSENA (Jira)" <ji...@apache.org> on 2026/09/30 13:06:00 UTC

[jira] [Updated] (ATLAS-5426) Address RowTooBigException seem while reading and writing data to HBase

     [ https://issues.apache.org/jira/browse/ATLAS-5426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

SONALI SAKSENA updated ATLAS-5426:
----------------------------------
    Description: 
The following exception is seen many times :
 
 {{org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.regionserver.RowTooBigException): org.apache.hadoop.hbase.regionserver.RowTooBigException: Max row size allowed: 1073741824, but the row is bigger than that, the row info: \x1Ae\xBA\xA2(\x89\xA0hive_colum\xEE/g:\x00\x1A)&`\xC0/1679398115122001/Put/vlen=5/seqid=0, already have process row cells = 19885699, it belong to region = atlas_janus,\x1Ae\xBA\xA2(\x89\xA0hive_colum\xEE,1655444318575.786a114bc1daf2de06998799539853cc.}}
This exception is seen several other escalations as well, and as a workaround we suggest updating the value of *hbase.table.max.rowsize* property to increase the rowsize value. It helps in some cases, but in some cases this error still occurs. There is a concern from HBase team that increasing this value can cause OOM error on HBase side.

We need to understand how janus graph stores data in HBase and see from Atlas side how we can avoid this exception

 

  was:
*Description:*

If an invalid or unknown {{typeName}} is encountered during notification processing, we should validate it upfront and throw an exception before processing the notification. This would prevent the notification from being processed, failing later, and being unnecessarily retried.

*Logs:*
 
 {{2026-09-24 06:27:31,201 [NotificationHookConsumer thread-0] INFO [NotificationHookConsumer.java:585] Message type: ENTITY_PARTIAL_UPDATE_V22026-09-24 06:27:31,203 [NotificationHookConsumer thread-0] INFO [GraphTransactionAdvisor.java:43] GraphTransaction intercept for org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.updateEntity2026-09-24 06:27:31,203 [NotificationHookConsumer thread-0] ERROR [GraphTransactionInterceptor.java:280] graph rollback due to exceptionorg.apache.atlas.exception.AtlasBaseException: trino_table: Unknown/invalid typename        at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.updateEntity(AtlasEntityStoreV2.java:366)        at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke(<generated>)        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:792)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)        at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:190)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:707)        at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$a46f3b8f.updateEntity(<generated>)        at org.apache.atlas.notification.SerialEntityProcessor.handleMessage(SerialEntityProcessor.java:584)        at org.apache.atlas.notification.SerialEntityProcessor.handleMessage(SerialEntityProcessor.java:324)        at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:587)        at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:501)        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)        at java.base/java.lang.Thread.run(Thread.java:840) }}


> Address RowTooBigException seem while reading and writing data to HBase
> -----------------------------------------------------------------------
>
>                 Key: ATLAS-5426
>                 URL: https://issues.apache.org/jira/browse/ATLAS-5426
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: SONALI SAKSENA
>            Assignee: SONALI SAKSENA
>            Priority: Major
>             Fix For: 3.0.0, 2.6.0
>
>
> The following exception is seen many times :
>  
>  {{org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.regionserver.RowTooBigException): org.apache.hadoop.hbase.regionserver.RowTooBigException: Max row size allowed: 1073741824, but the row is bigger than that, the row info: \x1Ae\xBA\xA2(\x89\xA0hive_colum\xEE/g:\x00\x1A)&`\xC0/1679398115122001/Put/vlen=5/seqid=0, already have process row cells = 19885699, it belong to region = atlas_janus,\x1Ae\xBA\xA2(\x89\xA0hive_colum\xEE,1655444318575.786a114bc1daf2de06998799539853cc.}}
> This exception is seen several other escalations as well, and as a workaround we suggest updating the value of *hbase.table.max.rowsize* property to increase the rowsize value. It helps in some cases, but in some cases this error still occurs. There is a concern from HBase team that increasing this value can cause OOM error on HBase side.
> We need to understand how janus graph stores data in HBase and see from Atlas side how we can avoid this exception
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)