You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2023/05/08 23:59:00 UTC
[jira] [Resolved] (IMPALA-10842) Show full stacktraces of jvm threads in web UI
[ https://issues.apache.org/jira/browse/IMPALA-10842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Quanlong Huang resolved IMPALA-10842.
-------------------------------------
Resolution: Duplicate
> Show full stacktraces of jvm threads in web UI
> ----------------------------------------------
>
> Key: IMPALA-10842
> URL: https://issues.apache.org/jira/browse/IMPALA-10842
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Quanlong Huang
> Priority: Major
> Labels: observability, ramp-up
> Attachments: image-2021-08-05-11-01-20-978.png
>
>
> IMPALA-4885 adds a web page showing all the JVM threads, which saves us some time to ssh to the machine and perform jstack manually.
> However, the summary column doesn't show the full stacktrace if it has more than 8 frames. For instance, when debugging a long running DDL in catalogd, I search for "execDdl" in the search box and get nothing since "execDdl" is truncated:
> !image-2021-08-05-11-01-20-978.png|width=905,height=208!
> I finally did a jstack and find the thread:
> {code:java}
> "Thread-11" #42 prio=5 os_prio=0 tid=0x000000000f86c000 nid=0x4c99 runnable [0x00007f9a8f8e1000]
> java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> at java.net.SocketInputStream.read(SocketInputStream.java:171)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> - locked <0x00000005d4dbfa88> (a java.io.BufferedInputStream)
> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:425)
> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:321)
> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:225)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_objects_by_name_req(ThriftHiveMetastore.java:2492)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_objects_by_name_req(ThriftHiveMetastore.java:2479)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:2333)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTableObjectsByName(HiveMetaStoreClient.java:2314)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
> at com.sun.proxy.$Proxy11.getTableObjectsByName(Unknown Source)
> at org.apache.impala.service.CatalogOpExecutor.dropTablesFromKudu(CatalogOpExecutor.java:2349)
> at org.apache.impala.service.CatalogOpExecutor.dropDatabase(CatalogOpExecutor.java:2255)
> - locked <0x00000005d4a101e8> (a java.lang.Object)
> at org.apache.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:462)
> at org.apache.impala.service.JniCatalog.execDdl(JniCatalog.java:240)
> {code}
> The cause is that we simply use TheadInfo#toString() which truncates the stacktrace: https://github.com/apache/impala/blob/47d868fe586905b2f4ea347a97ce582a45aac405/fe/src/main/java/org/apache/impala/common/JniUtil.java#L281
> We can improve this to get the whole stacktrace.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org