[jira] [Created] (IMPALA-11401) Catalogd should log the table names causing OOM on array limit

["Quanlong Huang (Jira)" <ji...@apache.org>]

|  ![](cid:jira-generated-image-avatar-97461243-281f-4af9-9ca9-fdc4b7fbc9c6) |
[Quanlong
Huang](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=stigahuang)
**created** an issue  
---|---  
|  
---  
|  [IMPALA](https://issues.apache.org/jira/browse/IMPALA) /
[![Improvement](cid:jira-generated-image-
avatar-2ba4e233-8da2-440a-8ffa-2407958aed98)](https://issues.apache.org/jira/browse/IMPALA-11401)
[IMPALA-11401](https://issues.apache.org/jira/browse/IMPALA-11401)  
---  
[Catalogd should log the table names causing OOM on array
limit](https://issues.apache.org/jira/browse/IMPALA-11401)  
| Issue Type: |  ![Improvement](cid:jira-generated-image-
avatar-2ba4e233-8da2-440a-8ffa-2407958aed98) Improvement  
---|---  
Assignee: |  [Quanlong
Huang](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=stigahuang)  
Components: |  Catalog  
Created: |  28/Jun/22 13:17  
Priority: |  ![Critical](cid:jira-generated-image-static-
critical-b2ae9684-5bae-4896-b26d-c36deb4e39f4) Critical  
Reporter: |  [Quanlong
Huang](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=stigahuang)  
|

Catalogd could throw OOM errors when serializing thrift objects larger than
the 2GB byte array limit, e.g. in serializing responses of execDdl,
getPartialCatalogObject requests. Such kind of OOM errors are table-level
failures. They don't mean the server is running out of memory. Catalogd is
still able to process other RPC requests.

To improve the supportability of catalogd, it should log the table name and
some details of the request when throwing such OOM errors. Currently the log
contains no details:

    
    
    I0617 04:00:24.341722 534809 jni-util.cc:288] java.lang.OutOfMemoryError
            at java.io.ByteArrayOutputStream.hugeCapacity(ByteArrayOutputStream.java:123)
            at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:117)
            at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
            at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
            at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145)
            at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:202)
            at org.apache.hadoop.hive.metastore.api.FieldSchema$FieldSchemaStandardScheme.write(FieldSchema.java:531)
            at org.apache.hadoop.hive.metastore.api.FieldSchema$FieldSchemaStandardScheme.write(FieldSchema.java:480)
            at org.apache.hadoop.hive.metastore.api.FieldSchema.write(FieldSchema.java:418)
            at org.apache.hadoop.hive.metastore.api.StorageDescriptor$StorageDescriptorStandardScheme.write(StorageDescriptor.java:1451)
            at org.apache.hadoop.hive.metastore.api.StorageDescriptor$StorageDescriptorStandardScheme.write(StorageDescriptor.java:1278)
            at org.apache.hadoop.hive.metastore.api.StorageDescriptor.write(StorageDescriptor.java:1144)
            at org.apache.hadoop.hive.metastore.api.Partition$PartitionStandardScheme.write(Partition.java:1414)
            at org.apache.hadoop.hive.metastore.api.Partition$PartitionStandardScheme.write(Partition.java:1238)
            at org.apache.hadoop.hive.metastore.api.Partition.write(Partition.java:1099)
            at org.apache.impala.thrift.TPartialPartitionInfo$TPartialPartitionInfoStandardScheme.write(TPartialPartitionInfo.java:862)
            at org.apache.impala.thrift.TPartialPartitionInfo$TPartialPartitionInfoStandardScheme.write(TPartialPartitionInfo.java:759)
            at org.apache.impala.thrift.TPartialPartitionInfo.write(TPartialPartitionInfo.java:665)
            at org.apache.impala.thrift.TPartialTableInfo$TPartialTableInfoStandardScheme.write(TPartialTableInfo.java:914)
            at org.apache.impala.thrift.TPartialTableInfo$TPartialTableInfoStandardScheme.write(TPartialTableInfo.java:790)
            at org.apache.impala.thrift.TPartialTableInfo.write(TPartialTableInfo.java:688)
            at org.apache.impala.thrift.TGetPartialCatalogObjectResponse$TGetPartialCatalogObjectResponseStandardScheme.write(TGetPartialCatalogObjectResponse.java:977)
            at org.apache.impala.thrift.TGetPartialCatalogObjectResponse$TGetPartialCatalogObjectResponseStandardScheme.write(TGetPartialCatalogObjectResponse.java:857)
            at org.apache.impala.thrift.TGetPartialCatalogObjectResponse.write(TGetPartialCatalogObjectResponse.java:739)
            at org.apache.thrift.TSerializer.serialize(TSerializer.java:79)
            at org.apache.impala.service.JniCatalog.getPartialCatalogObject(JniCatalog.java:254)
    I0617 04:00:24.341833 534809 status.cc:129] OutOfMemoryError: null
        @           0xc1d7f3
        @          0x13a8679
        @           0xc05bf3
        @           0xbf3433
        @           0xd133c4
        @           0xd07d33
        @           0xd1f3a2
        @          0x10d214a
        @          0x10c5702
        @          0x144de71
        @          0x144f2ea
        @          0x1c9f8d1
        @     0x7ff6aca68ea4
        @     0x7ff6a9540b0c
    E0617 04:00:24.341909 534809 catalog-server.cc:209] OutOfMemoryError: null
      
  
---  
|  |  [ ![Add Comment](cid:jira-generated-image-static-comment-
icon-66eee5d4-efd6-4b75-86b5-a493f6688d56)
](https://issues.apache.org/jira/browse/IMPALA-11401#add-comment "Add
Comment") |  [Add
Comment](https://issues.apache.org/jira/browse/IMPALA-11401#add-comment "Add
Comment")  
---|---  
  
|  This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9) |  |
![Atlassian logo](https://issues.apache.org/jira/images/mail/atlassian-email-
logo.png)  
---