You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/01/10 07:25:03 UTC

[GitHub] [hadoop-ozone] avijayanhwx opened a new pull request #431: HDDS-2846. Handle Datanode Pipeline & Container Reports reports in Recon.

avijayanhwx opened a new pull request #431: HDDS-2846. Handle Datanode Pipeline & Container Reports reports in Recon.
URL: https://github.com/apache/hadoop-ozone/pull/431
 
 
   ## What changes were proposed in this pull request?
   Recon should be able to process Container and Pipeline reports from registered Datanodes. It still wont be able to understand new containers and pipelines though. Along with HDDS-2846, HDDS-2852 and HDDS-2869, Recon will be a fully functional passive SCM. 
   In addition to the above, some refactoring work also has been done.
   
   ## What is the link to the Apache JIRA
   https://issues.apache.org/jira/browse/HDDS-2846
   
   ## How was this patch tested?
   Manually tested on docker.
   Created key (container) on ozone cluster, and verified Recon gets the container info in Datanode heartbeat.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] avijayanhwx commented on issue #431: HDDS-2846. Handle Datanode Pipeline & Container Reports reports in Recon.

Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on issue #431: HDDS-2846. Handle Datanode Pipeline & Container Reports reports in Recon.
URL: https://github.com/apache/hadoop-ozone/pull/431#issuecomment-574794999
 
 
   @nandakumar131 Please review.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #431: HDDS-2846. Handle Datanode Pipeline & Container Reports reports in Recon.

Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on a change in pull request #431: HDDS-2846. Handle Datanode Pipeline & Container Reports reports in Recon.
URL: https://github.com/apache/hadoop-ozone/pull/431#discussion_r367772227
 
 

 ##########
 File path: hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconStorageContainerManagerFacade.java
 ##########
 @@ -27,52 +28,99 @@
 
 import org.apache.hadoop.hdds.conf.OzoneConfiguration;
 import org.apache.hadoop.hdds.scm.block.BlockManager;
+import org.apache.hadoop.hdds.scm.container.CloseContainerEventHandler;
+import org.apache.hadoop.hdds.scm.container.ContainerActionsHandler;
 import org.apache.hadoop.hdds.scm.container.ContainerManager;
+import org.apache.hadoop.hdds.scm.container.ContainerReportHandler;
+import org.apache.hadoop.hdds.scm.container.IncrementalContainerReportHandler;
 import org.apache.hadoop.hdds.scm.container.ReplicationManager;
 import org.apache.hadoop.hdds.scm.events.SCMEvents;
 import org.apache.hadoop.hdds.scm.net.NetworkTopology;
 import org.apache.hadoop.hdds.scm.net.NetworkTopologyImpl;
+import org.apache.hadoop.hdds.scm.node.DeadNodeHandler;
 import org.apache.hadoop.hdds.scm.node.NodeManager;
 import org.apache.hadoop.hdds.scm.node.NodeReportHandler;
+import org.apache.hadoop.hdds.scm.node.SCMNodeManager;
+import org.apache.hadoop.hdds.scm.node.StaleNodeHandler;
+import org.apache.hadoop.hdds.scm.pipeline.PipelineActionHandler;
 import org.apache.hadoop.hdds.scm.pipeline.PipelineManager;
+import org.apache.hadoop.hdds.scm.pipeline.PipelineReportHandler;
+import org.apache.hadoop.hdds.scm.safemode.SafeModeManager;
 import org.apache.hadoop.hdds.scm.server.SCMStorageConfig;
 import org.apache.hadoop.hdds.scm.server.OzoneStorageContainerManager;
 import org.apache.hadoop.hdds.server.events.EventQueue;
+import org.apache.hadoop.io.IOUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 /**
  * Recon's 'lite' version of SCM.
  */
-public class ReconStorageContainerManager
+public class ReconStorageContainerManagerFacade
 
 Review comment:
   Thanks for the review @swagle. I will address this in the next patch (HDDS-2869).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] arp7 merged pull request #431: HDDS-2846. Handle Datanode Pipeline & Container Reports reports in Recon.

Posted by GitBox <gi...@apache.org>.
arp7 merged pull request #431: HDDS-2846. Handle Datanode Pipeline & Container Reports reports in Recon.
URL: https://github.com/apache/hadoop-ozone/pull/431
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] swagle commented on a change in pull request #431: HDDS-2846. Handle Datanode Pipeline & Container Reports reports in Recon.

Posted by GitBox <gi...@apache.org>.
swagle commented on a change in pull request #431: HDDS-2846. Handle Datanode Pipeline & Container Reports reports in Recon.
URL: https://github.com/apache/hadoop-ozone/pull/431#discussion_r367753518
 
 

 ##########
 File path: hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconStorageContainerManagerFacade.java
 ##########
 @@ -27,52 +28,99 @@
 
 import org.apache.hadoop.hdds.conf.OzoneConfiguration;
 import org.apache.hadoop.hdds.scm.block.BlockManager;
+import org.apache.hadoop.hdds.scm.container.CloseContainerEventHandler;
+import org.apache.hadoop.hdds.scm.container.ContainerActionsHandler;
 import org.apache.hadoop.hdds.scm.container.ContainerManager;
+import org.apache.hadoop.hdds.scm.container.ContainerReportHandler;
+import org.apache.hadoop.hdds.scm.container.IncrementalContainerReportHandler;
 import org.apache.hadoop.hdds.scm.container.ReplicationManager;
 import org.apache.hadoop.hdds.scm.events.SCMEvents;
 import org.apache.hadoop.hdds.scm.net.NetworkTopology;
 import org.apache.hadoop.hdds.scm.net.NetworkTopologyImpl;
+import org.apache.hadoop.hdds.scm.node.DeadNodeHandler;
 import org.apache.hadoop.hdds.scm.node.NodeManager;
 import org.apache.hadoop.hdds.scm.node.NodeReportHandler;
+import org.apache.hadoop.hdds.scm.node.SCMNodeManager;
+import org.apache.hadoop.hdds.scm.node.StaleNodeHandler;
+import org.apache.hadoop.hdds.scm.pipeline.PipelineActionHandler;
 import org.apache.hadoop.hdds.scm.pipeline.PipelineManager;
+import org.apache.hadoop.hdds.scm.pipeline.PipelineReportHandler;
+import org.apache.hadoop.hdds.scm.safemode.SafeModeManager;
 import org.apache.hadoop.hdds.scm.server.SCMStorageConfig;
 import org.apache.hadoop.hdds.scm.server.OzoneStorageContainerManager;
 import org.apache.hadoop.hdds.server.events.EventQueue;
+import org.apache.hadoop.io.IOUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 /**
  * Recon's 'lite' version of SCM.
  */
-public class ReconStorageContainerManager
+public class ReconStorageContainerManagerFacade
 
 Review comment:
   Minor nit: Should drop the Facade from the class name, a facade merely provides access to underlying subsystems minus the management functions which this class actually does in addition to abstracting out access.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] avijayanhwx commented on issue #431: HDDS-2846. Handle Datanode Pipeline & Container Reports reports in Recon.

Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on issue #431: HDDS-2846. Handle Datanode Pipeline & Container Reports reports in Recon.
URL: https://github.com/apache/hadoop-ozone/pull/431#issuecomment-572907756
 
 
   @anuengineer  / @swagle Please review.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org