You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Jinglun (Jira)" <ji...@apache.org> on 2020/05/07 03:55:00 UTC
[jira] [Comment Edited] (HDFS-15294) RBF: Balance data across
federation namespaces with DistCp and snapshot diff
[ https://issues.apache.org/jira/browse/HDFS-15294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101318#comment-17101318 ]
Jinglun edited comment on HDFS-15294 at 5/7/20, 3:54 AM:
---------------------------------------------------------
Hi [~elgoiri], thanks your comments ! I can split this into 2 patches:
1.The The State Machine(BalanceProcedureScheduler).: Including the abstraction of job and scheduler model.
{code:java}
org.apache.hadoop.hdfs.procedure.BalanceProcedureScheduler;
org.apache.hadoop.hdfs.procedure.BalanceProcedureConfigKeys;
org.apache.hadoop.hdfs.procedure.BalanceProcedure;
org.apache.hadoop.hdfs.procedure.BalanceJob;
org.apache.hadoop.hdfs.procedure.BalanceJournal;
org.apache.hadoop.hdfs.procedure.HDFSJournal;
{code}
2.The DistCpFedBalance: It's an implementation of BalanceJob.
{code:java}
org.apache.hadoop.hdfs.server.federation.procedure.MountTableProcedure;
org.apache.hadoop.tools.DistCpFedBalance;
org.apache.hadoop.tools.DistCpProcedure;
org.apache.hadoop.tools.FedBalance;
org.apache.hadoop.tools.FedBalanceConfigs;
org.apache.hadoop.tools.FedBalanceContext;
org.apache.hadoop.tools.TrashProcedure;
{code}
The second patch 'DistCpFedBalance' relies on the first one.
was (Author: lijinglun):
Hi [~elgoiri], thanks your comments ! I can split this into 2 patches:
1.The BalanceProcedureScheduler: Including the abstraction of job and scheduler model.
{code:java}
org.apache.hadoop.hdfs.procedure.BalanceProcedureScheduler;
org.apache.hadoop.hdfs.procedure.BalanceProcedureConfigKeys;
org.apache.hadoop.hdfs.procedure.BalanceProcedure;
org.apache.hadoop.hdfs.procedure.BalanceJob;
org.apache.hadoop.hdfs.procedure.BalanceJournal;
org.apache.hadoop.hdfs.procedure.HDFSJournal;
{code}
2.The DistCpFedBalance: It's an implementation of BalanceJob.
{code:java}
org.apache.hadoop.hdfs.server.federation.procedure.MountTableProcedure;
org.apache.hadoop.tools.DistCpFedBalance;
org.apache.hadoop.tools.DistCpProcedure;
org.apache.hadoop.tools.FedBalance;
org.apache.hadoop.tools.FedBalanceConfigs;
org.apache.hadoop.tools.FedBalanceContext;
org.apache.hadoop.tools.TrashProcedure;
{code}
The second patch 'DistCpFedBalance' relies on the first one.
> RBF: Balance data across federation namespaces with DistCp and snapshot diff
> ----------------------------------------------------------------------------
>
> Key: HDFS-15294
> URL: https://issues.apache.org/jira/browse/HDFS-15294
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jinglun
> Assignee: Jinglun
> Priority: Major
> Attachments: BalanceProcedureScheduler.png, HDFS-15294.001.patch, HDFS-15294.002.patch, HDFS-15294.003.patch, HDFS-15294.003.reupload.patch, HDFS-15294.004.patch, HDFS-15294.005.patch, HDFS-15294.006.patch, HDFS-15294.007.patch, distcp-balance.pdf
>
>
> This jira introduces a new balance command 'fedbalance' that is ran by the administrator. The process is:
> 1. Use distcp and snapshot diff to sync data between src and dst until they are the same.
> 2. Update mount table in Router.
> 3. Delete the src to trash.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org