You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Elvis (JIRA)" <ji...@apache.org> on 2018/11/21 10:08:00 UTC
[jira] [Commented] (HADOOP-14604) Fix the bug of aggregation log
have one replication in HDFS Federation environment
[ https://issues.apache.org/jira/browse/HADOOP-14604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694510#comment-16694510 ]
Elvis commented on HADOOP-14604:
--------------------------------
This patch may have some problems. Before applying this patch, the time between LOCALIZING and LOCALIZED is very fast. But after applying this patch, the time between LOCALIZING and LOCALIZED becomes more than 10s
> Fix the bug of aggregation log have one replication in HDFS Federation environment
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-14604
> URL: https://issues.apache.org/jira/browse/HADOOP-14604
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, viewfs
> Affects Versions: 2.6.0
> Environment: Hadoop 2.6.0 with HDFS Federation
> Reporter: dingguotao
> Assignee: Doris Gu
> Priority: Major
> Fix For: 2.9.0, 2.7.4, 3.0.0-alpha4, 2.8.2
>
> Attachments: HADOOP-14604.patch
>
>
> With HDFS Federation configured, the aggregation log have a default replication count of 1, rather than the value of dfs.replication in hdfs-site.xml configuration file.
> Before:
> {code:java}
> [op@NM-304-SA5212M4-BIGDATA-640 ~]$ hdfs dfs -ls /yarn/apps/op/logs/application_1498544648312_0001
> Found 3 items
> -rw-r----- 1 op hadoop 8270 2017-06-27 14:25 /yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-645_8041
> -rw-r----- 1 op hadoop 54469 2017-06-27 14:25 /yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-646_8041
> -rw-r----- 1 op hadoop 6537 2017-06-27 14:25 /yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-672_8041
> {code}
> The aggregation log have only 1 replication in default. But the value of dfs.replication in hdfs-site.xml configuration file is 3
> After apply this patch:
> {code:java}
> [op@NM-304-SA5212M4-BIGDATA-640 ~]$ hdfs dfs -ls /yarn/apps/op/logs/application_1498635214020_0001
> Found 3 items
> -rw-r----- 3 op hadoop 5729 2017-06-28 15:34 /yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-645_8041
> -rw-r----- 3 op hadoop 38439 2017-06-28 15:34 /yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-646_8041
> -rw-r----- 3 op hadoop 8270 2017-06-28 15:34 /yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-671_8041
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org