You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Xudong Sun (Jira)" <ji...@apache.org> on 2020/01/14 23:04:00 UTC
[jira] [Resolved] (HDFS-15070) Crashing bugs in NameNode when using
a valid configuration for `dfs.namenode.audit.loggers`
[ https://issues.apache.org/jira/browse/HDFS-15070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xudong Sun resolved HDFS-15070.
-------------------------------
Resolution: Duplicate
> Crashing bugs in NameNode when using a valid configuration for `dfs.namenode.audit.loggers`
> -------------------------------------------------------------------------------------------
>
> Key: HDFS-15070
> URL: https://issues.apache.org/jira/browse/HDFS-15070
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.10.0
> Reporter: Xudong Sun
> Priority: Critical
>
> I am using Hadoop-2.10.0.
> The configuration parameter `dfs.namenode.audit.loggers` allows `default` (which is the default value) and `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`.
> When I use `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`, namenode will not be started successfully because of an `InstantiationException` thrown from `org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers`.
> The root cause is that while initializing namenode, `initAuditLoggers` will be called and it will try to call the default constructor of `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` which doesn't have a default constructor. Thus the `InstantiationException` exception is thrown.
>
> *Symptom*
> *$ ./start-dfs.sh*
>
> {code:java}
> 2019-12-18 14:05:20,670 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.java.lang.RuntimeException: java.lang.InstantiationException: org.apache.hadoop.hdfs.server.namenode.top.TopAuditLoggerat org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1024)at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:858)at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:677)at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:674)at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:736)at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:961)at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:940)at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1714)at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1782)Caused by: java.lang.InstantiationException: org.apache.hadoop.hdfs.server.namenode.top.TopAuditLoggerat java.lang.Class.newInstance(Class.java:427)at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1017)... 8 moreCaused by: java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger.<init>()at java.lang.Class.getConstructor0(Class.java:3082)at java.lang.Class.newInstance(Class.java:412)
> ... 9 more
> {code}
>
>
> *Detailed Root Cause*
> There is no default constructor in `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`:
> {code:java}
> /**
> * An {@link AuditLogger} that sends logged data directly to the metrics
> * systems. It is used when the top service is used directly by the name node
> */
> @InterfaceAudience.Private
> public class TopAuditLogger implements AuditLogger {
> public static finalLogger LOG = LoggerFactory.getLogger(TopAuditLogger.class);
> private final TopMetrics topMetrics;
> public TopAuditLogger(TopMetrics topMetrics) {
> Preconditions.checkNotNull(topMetrics, "Cannot init with a null " +
> "TopMetrics");
> this.topMetrics = topMetrics;
> }
> @Override
> public void initialize(Configuration conf) {
> }{code}
> As long as the configuration parameter `dfs.namenode.audit.loggers` is set to `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`, `initAuditLoggers` will try to call its default constructor to make a new instance:
> {code:java}
> private List<AuditLogger> initAuditLoggers(Configuration conf) {
> // Initialize the custom access loggers if configured.
> Collection<String> alClasses =
> conf.getTrimmedStringCollection(DFS_NAMENODE_AUDIT_LOGGERS_KEY);
> List<AuditLogger> auditLoggers = Lists.newArrayList();
> if (alClasses != null && !alClasses.isEmpty()) {
> for (String className : alClasses) {
> try {
> AuditLogger logger;
> if (DFS_NAMENODE_DEFAULT_AUDIT_LOGGER_NAME.equals(className)) {
> logger = new DefaultAuditLogger();
> } else {
> logger = (AuditLogger) Class.forName(className).newInstance();
> }
> logger.initialize(conf);
> auditLoggers.add(logger);
> } catch (RuntimeException re) {
> throw re;
> } catch (Exception e) {
> throw new RuntimeException(e);
> }
> }
> }{code}
> `initAuditLoggers` tries to call the default constructor to make a new instance in:
> {code:java}
> logger = (AuditLogger) Class.forName(className).newInstance();{code}
> This is very different from the default configuration, `default`, which implements a default constructor so the default is fine.
>
> *How To Reproduce*
> The version of Hadoop: 2.10.0
> # Set the value of configuration parameter `dfs.namenode.audit.loggers` to `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` in "hdfs-site.xml"(the default value is `default`)
> # Start the namenode by running "start-dfs.sh"
> # The namenode will not be started successfully.
>
> {code:java}
> <property>
> <name>dfs.namenode.audit.loggers</name>
> <value>org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger</value>
> <description>
> List of classes implementing audit loggers that will receive audit events.
> These should be implementations of org.apache.hadoop.hdfs.server.namenode.AuditLogger.
> The special value "default" can be used to reference the default audit
> logger, which uses the configured log system. Installing custom audit loggers
> may affect the performance and stability of the NameNode. Refer to the custom
> logger's documentation for more details.
> </description>
> </property>
> {code}
>
> *How To Fix*
> Add a default constructor for `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org