[hdfs] [metrics] RpcAuthenticationSuccesses

[Anatoly <an...@slobozhanov.ru>]

Hi

There is a question about two hdfs metrics that arose as a result of my
attempts to calculate the load on the KDC for an industrial cluster

There are two parameters in hdfs metrics

RpcAuthenticationSuccesses - Total number of successful authentication
attempts

RpcAuthenticationFailures - Total number of authentication failures

I expect that any data request in the hadoop cluster will commit

the request to KDC -> get ticket,

the request to the NameNode

after which the request counter should activate either +1 to the metric if
successful, or +1 to the metric if unsuccessful

However, in a test cluster where I have

4 DataNodes and 2 NameNodes (HA), I see completely incomprehensible indicators
for these metrics.

By the way, at the same time, I noticed that the RpcAuthenticationSuccesses
readings gradually increase by +1 every 30 seconds

TEST 1

I made sure that

1\. Only HDFS-{NN,DN,JN, ZKFC} and YARN-{RM,NM} services work

2\. All other components that were – hive, spark HistoryServer, are disabled

3\. There are no YARN jobs running and no user requests to hdfs

At the time of testing, the value of RpcAuthenticationFailures indicators = 0

RpcAuthenticationSuccesses = 208322

To check the download, I run the spark-submit test - spark-
examples_2.12-3.5.0.jar with the number of performers = 1

The request was completed in 1 minute and 20 seconds

RpcAuthenticationSuccesses = 208338

In total, +16 was added to the original value during execution

Let's say +2 can be attributed to the moment I wrote about above +1 every 30
seconds. But what does +14 authentications mean?

TEST 2

RpcAuthenticationFailures = 0

RpcAuthenticationSuccesses = 208388

hdfs dfs -ls /

RpcAuthenticationFailures = 0

RpcAuthenticationSuccesses = 208389

Added +1. Why?

I started kinit long before the ls/request, i.e. the metric should not have
changed, I think so, but maybe I'm wrong

TEST 3

disabled

\- All DN are

\- Satndby NN

\- All YARN services (RM, NM)

still running

Three JN, ZKFC

One NN is active

The +1 counter continues to add +1 to the RpcAuthenticationSuccesses metric
every 30 seconds

Either I misunderstand the meaning of these indicators, or something is
considered wrong

Can you tell me how these indicators are calculated, I do not understand this
or is it an error in the calculations and if I do not understand the work of
these metrics, then how is it correct?

Thank you very much

\--

With best wishes,

Anatoliy