You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Ahmed Hussein (JIRA)" <ji...@apache.org> on 2019/06/03 20:47:00 UTC
[jira] [Commented] (HADOOP-16266) Add more fine-grained processing
time metrics to the RPC layer
[ https://issues.apache.org/jira/browse/HADOOP-16266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855032#comment-16855032 ]
Ahmed Hussein commented on HADOOP-16266:
----------------------------------------
I had the same problem.
In addition to the IPC server exception, sometimes, I see a null pointer exception
{code:java}
2019-05-31 14:32:10,417 ERROR [main] impl.MetricsSourceAdapter (MetricsSourceAdapter.java:getMetrics(202)) - Error getting metrics from source ipc.8020.FairCallQueue
java.lang.NullPointerException
at org.apache.hadoop.ipc.FairCallQueue$MetricsProxy.getQueueSizes(FairCallQueue.java:382)
at org.apache.hadoop.ipc.FairCallQueue$MetricsProxy.getMetrics(FairCallQueue.java:410)
at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:200)
at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:183)
at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:156)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getNewMBeanClassName(DefaultMBeanServerInterceptor.java:333)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:319)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:100)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:73)
at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:222)
at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:101)
at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:268)
at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:233)
at org.apache.hadoop.ipc.FairCallQueue$MetricsProxy.<init>(FairCallQueue.java:361)
at org.apache.hadoop.ipc.FairCallQueue$MetricsProxy.getInstance(FairCallQueue.java:368)
at org.apache.hadoop.ipc.FairCallQueue.<init>(FairCallQueue.java:114)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.ipc.CallQueueManager.createCallQueueInstance(CallQueueManager.java:149)
at org.apache.hadoop.ipc.CallQueueManager.<init>(CallQueueManager.java:79)
at org.apache.hadoop.ipc.Server.<init>(Server.java:3059)
at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:1039)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:427)
at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:347)
at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:848)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.<init>(NameNodeRpcServer.java:467)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createRpcServer(NameNode.java:803)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:709)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:960)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:933)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1699)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1766)
2019-05-31 14:32:10,418 INFO [main] ipc.CallQueueManager (CallQueueManager.java:<init>(84)) - Using callQueue: class org.apache.hadoop.ipc.FairCallQueue, queueCapacity: 1000, scheduler: class org.apache.hadoop.ipc.DecayRpcScheduler, ipcBackoff: true.
{code}
> Add more fine-grained processing time metrics to the RPC layer
> --------------------------------------------------------------
>
> Key: HADOOP-16266
> URL: https://issues.apache.org/jira/browse/HADOOP-16266
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
> Reporter: Christopher Gregorian
> Assignee: Erik Krogen
> Priority: Minor
> Labels: rpc
> Fix For: 3.3.0
>
> Attachments: HADOOP-16266.001.patch, HADOOP-16266.002.patch, HADOOP-16266.003.patch, HADOOP-16266.004.patch, HADOOP-16266.005.patch, HADOOP-16266.006.patch, HADOOP-16266.007.patch, HADOOP-16266.008.patch, HADOOP-16266.009.patch, HADOOP-16266.010.patch, HADOOP-16266.011.patch
>
>
> Splitting off of HDFS-14403 to track the first part: introduces more fine-grained measuring of how a call's processing time is split up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org