You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Bharat Viswanadham <bv...@hortonworks.com> on 2018/06/28 22:16:36 UTC

[DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Hi everyone,

I’d like to start a thread to discuss merging the HDDS-48 branch to trunk. The ContainerIO work refactors the HDDS Datanode IO path to enforce clean separation between the Container management and the Storage layers.

Note: HDDS/Ozone code is not compiled by default in trunk. The 'hdds' maven profile must be enabled to compile the branch payload.
 
The merge payload includes the following key improvements:
1. Support multiple container types on the datanode.
2. Adopt a new disk layout for the containers that supports future upgrades.
3. Support volume Choosing policy for container data locations.
4. Changed the format of the .container file to a human-readable format (yaml)
 
Below are the links for design documents attached to HDDS-48.
 
https://issues.apache.org/jira/secure/attachment/12923107/ContainerIO-StorageManagement-DesignDoc.pdf
https://issues.apache.org/jira/secure/attachment/12923108/HDDS DataNode Disk Layout.pdf
 
The branch is ready to merge. Over the next week we will clean up the unused classes, fix old integration tests and continue testing the changes.
 
Thanks to Hanisha Koneru, Arpit Agarwal, Anu Engineer, Jitendra Pandey,  Xiaoyu Yao, Ajay Kumar, Mukul Kumar Singh, Marton Elek and Shashikant Banerjee for their contributions in design, development and code reviews.

Thanks,
Bharat



---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Posted by Ajay Kumar <aj...@hortonworks.com>.
+1 (non binding)

On 6/29/18, 9:35 AM, "Jitendra Pandey" <ji...@hortonworks.com> wrote:

    +1. 
    This is an important piece of ozone infrastructure and other components depend on it. Therefore, it is important to merge it asap.
    
    On 6/29/18, 9:33 AM, "Arpit Agarwal" <aa...@hortonworks.com> wrote:
    
        +1 for merging this branch.
        
        Added common-dev@
        
        
        On 6/28/18, 3:16 PM, "Bharat Viswanadham" <bv...@hortonworks.com> wrote:
        
            Hi everyone,
            
            I’d like to start a thread to discuss merging the HDDS-48 branch to trunk. The ContainerIO work refactors the HDDS Datanode IO path to enforce clean separation between the Container management and the Storage layers.
            
            Note: HDDS/Ozone code is not compiled by default in trunk. The 'hdds' maven profile must be enabled to compile the branch payload.
             
            The merge payload includes the following key improvements:
            1. Support multiple container types on the datanode.
            2. Adopt a new disk layout for the containers that supports future upgrades.
            3. Support volume Choosing policy for container data locations.
            4. Changed the format of the .container file to a human-readable format (yaml)
             
            Below are the links for design documents attached to HDDS-48.
             
            https://issues.apache.org/jira/secure/attachment/12923107/ContainerIO-StorageManagement-DesignDoc.pdf
            https://issues.apache.org/jira/secure/attachment/12923108/HDDS DataNode Disk Layout.pdf
             
            The branch is ready to merge. Over the next week we will clean up the unused classes, fix old integration tests and continue testing the changes.
             
            Thanks to Hanisha Koneru, Arpit Agarwal, Anu Engineer, Jitendra Pandey,  Xiaoyu Yao, Ajay Kumar, Mukul Kumar Singh, Marton Elek and Shashikant Banerjee for their contributions in design, development and code reviews.
            
            Thanks,
            Bharat
            
            
            
            ---------------------------------------------------------------------
            To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
            For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
            
            
        
        
        ---------------------------------------------------------------------
        To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
        For additional commands, e-mail: common-dev-help@hadoop.apache.org
        
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
    For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Posted by Ajay Kumar <aj...@hortonworks.com>.
+1 (non binding)

On 6/29/18, 9:35 AM, "Jitendra Pandey" <ji...@hortonworks.com> wrote:

    +1. 
    This is an important piece of ozone infrastructure and other components depend on it. Therefore, it is important to merge it asap.
    
    On 6/29/18, 9:33 AM, "Arpit Agarwal" <aa...@hortonworks.com> wrote:
    
        +1 for merging this branch.
        
        Added common-dev@
        
        
        On 6/28/18, 3:16 PM, "Bharat Viswanadham" <bv...@hortonworks.com> wrote:
        
            Hi everyone,
            
            I’d like to start a thread to discuss merging the HDDS-48 branch to trunk. The ContainerIO work refactors the HDDS Datanode IO path to enforce clean separation between the Container management and the Storage layers.
            
            Note: HDDS/Ozone code is not compiled by default in trunk. The 'hdds' maven profile must be enabled to compile the branch payload.
             
            The merge payload includes the following key improvements:
            1. Support multiple container types on the datanode.
            2. Adopt a new disk layout for the containers that supports future upgrades.
            3. Support volume Choosing policy for container data locations.
            4. Changed the format of the .container file to a human-readable format (yaml)
             
            Below are the links for design documents attached to HDDS-48.
             
            https://issues.apache.org/jira/secure/attachment/12923107/ContainerIO-StorageManagement-DesignDoc.pdf
            https://issues.apache.org/jira/secure/attachment/12923108/HDDS DataNode Disk Layout.pdf
             
            The branch is ready to merge. Over the next week we will clean up the unused classes, fix old integration tests and continue testing the changes.
             
            Thanks to Hanisha Koneru, Arpit Agarwal, Anu Engineer, Jitendra Pandey,  Xiaoyu Yao, Ajay Kumar, Mukul Kumar Singh, Marton Elek and Shashikant Banerjee for their contributions in design, development and code reviews.
            
            Thanks,
            Bharat
            
            
            
            ---------------------------------------------------------------------
            To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
            For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
            
            
        
        
        ---------------------------------------------------------------------
        To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
        For additional commands, e-mail: common-dev-help@hadoop.apache.org
        
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
    For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Posted by Jitendra Pandey <ji...@hortonworks.com>.
+1. 
This is an important piece of ozone infrastructure and other components depend on it. Therefore, it is important to merge it asap.

On 6/29/18, 9:33 AM, "Arpit Agarwal" <aa...@hortonworks.com> wrote:

    +1 for merging this branch.
    
    Added common-dev@
    
    
    On 6/28/18, 3:16 PM, "Bharat Viswanadham" <bv...@hortonworks.com> wrote:
    
        Hi everyone,
        
        I’d like to start a thread to discuss merging the HDDS-48 branch to trunk. The ContainerIO work refactors the HDDS Datanode IO path to enforce clean separation between the Container management and the Storage layers.
        
        Note: HDDS/Ozone code is not compiled by default in trunk. The 'hdds' maven profile must be enabled to compile the branch payload.
         
        The merge payload includes the following key improvements:
        1. Support multiple container types on the datanode.
        2. Adopt a new disk layout for the containers that supports future upgrades.
        3. Support volume Choosing policy for container data locations.
        4. Changed the format of the .container file to a human-readable format (yaml)
         
        Below are the links for design documents attached to HDDS-48.
         
        https://issues.apache.org/jira/secure/attachment/12923107/ContainerIO-StorageManagement-DesignDoc.pdf
        https://issues.apache.org/jira/secure/attachment/12923108/HDDS DataNode Disk Layout.pdf
         
        The branch is ready to merge. Over the next week we will clean up the unused classes, fix old integration tests and continue testing the changes.
         
        Thanks to Hanisha Koneru, Arpit Agarwal, Anu Engineer, Jitendra Pandey,  Xiaoyu Yao, Ajay Kumar, Mukul Kumar Singh, Marton Elek and Shashikant Banerjee for their contributions in design, development and code reviews.
        
        Thanks,
        Bharat
        
        
        
        ---------------------------------------------------------------------
        To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
        For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
        
        
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
    For additional commands, e-mail: common-dev-help@hadoop.apache.org
    


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Posted by Jitendra Pandey <ji...@hortonworks.com>.
+1. 
This is an important piece of ozone infrastructure and other components depend on it. Therefore, it is important to merge it asap.

On 6/29/18, 9:33 AM, "Arpit Agarwal" <aa...@hortonworks.com> wrote:

    +1 for merging this branch.
    
    Added common-dev@
    
    
    On 6/28/18, 3:16 PM, "Bharat Viswanadham" <bv...@hortonworks.com> wrote:
    
        Hi everyone,
        
        I’d like to start a thread to discuss merging the HDDS-48 branch to trunk. The ContainerIO work refactors the HDDS Datanode IO path to enforce clean separation between the Container management and the Storage layers.
        
        Note: HDDS/Ozone code is not compiled by default in trunk. The 'hdds' maven profile must be enabled to compile the branch payload.
         
        The merge payload includes the following key improvements:
        1. Support multiple container types on the datanode.
        2. Adopt a new disk layout for the containers that supports future upgrades.
        3. Support volume Choosing policy for container data locations.
        4. Changed the format of the .container file to a human-readable format (yaml)
         
        Below are the links for design documents attached to HDDS-48.
         
        https://issues.apache.org/jira/secure/attachment/12923107/ContainerIO-StorageManagement-DesignDoc.pdf
        https://issues.apache.org/jira/secure/attachment/12923108/HDDS DataNode Disk Layout.pdf
         
        The branch is ready to merge. Over the next week we will clean up the unused classes, fix old integration tests and continue testing the changes.
         
        Thanks to Hanisha Koneru, Arpit Agarwal, Anu Engineer, Jitendra Pandey,  Xiaoyu Yao, Ajay Kumar, Mukul Kumar Singh, Marton Elek and Shashikant Banerjee for their contributions in design, development and code reviews.
        
        Thanks,
        Bharat
        
        
        
        ---------------------------------------------------------------------
        To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
        For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
        
        
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
    For additional commands, e-mail: common-dev-help@hadoop.apache.org
    


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Posted by Lokesh Jain <lo...@gmail.com>.
+1 (non-binding) 

Thanks
Lokesh

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Posted by Lokesh Jain <lo...@gmail.com>.
+1 (non-binding) 

Thanks
Lokesh

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Posted by Shashikant Banerjee <sb...@hortonworks.com>.
+1 (non-binding) for merging HDDS-48 to trunk.

Thanks
Shashi



On 6/29/18, 10:34 PM, "Xiaoyu Yao" <xy...@hortonworks.com> wrote:

    +1 for merging HDDS-48 to trunk. 
    
    On 6/29/18, 9:34 AM, "Arpit Agarwal" <aa...@hortonworks.com> wrote:
    
        +1 for merging this branch.
        
        Added common-dev@
        
        
        On 6/28/18, 3:16 PM, "Bharat Viswanadham" <bv...@hortonworks.com> wrote:
        
            Hi everyone,
            
            I’d like to start a thread to discuss merging the HDDS-48 branch to trunk. The ContainerIO work refactors the HDDS Datanode IO path to enforce clean separation between the Container management and the Storage layers.
            
            Note: HDDS/Ozone code is not compiled by default in trunk. The 'hdds' maven profile must be enabled to compile the branch payload.
             
            The merge payload includes the following key improvements:
            1. Support multiple container types on the datanode.
            2. Adopt a new disk layout for the containers that supports future upgrades.
            3. Support volume Choosing policy for container data locations.
            4. Changed the format of the .container file to a human-readable format (yaml)
             
            Below are the links for design documents attached to HDDS-48.
             
            https://issues.apache.org/jira/secure/attachment/12923107/ContainerIO-StorageManagement-DesignDoc.pdf
            https://issues.apache.org/jira/secure/attachment/12923108/HDDS DataNode Disk Layout.pdf
             
            The branch is ready to merge. Over the next week we will clean up the unused classes, fix old integration tests and continue testing the changes.
             
            Thanks to Hanisha Koneru, Arpit Agarwal, Anu Engineer, Jitendra Pandey,  Xiaoyu Yao, Ajay Kumar, Mukul Kumar Singh, Marton Elek and Shashikant Banerjee for their contributions in design, development and code reviews.
            
            Thanks,
            Bharat
            
            
            
            ---------------------------------------------------------------------
            To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
            For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
            
            
        
        
        ---------------------------------------------------------------------
        To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
        For additional commands, e-mail: common-dev-help@hadoop.apache.org
        
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
    For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Posted by Shashikant Banerjee <sb...@hortonworks.com>.
+1 (non-binding) for merging HDDS-48 to trunk.

Thanks
Shashi



On 6/29/18, 10:34 PM, "Xiaoyu Yao" <xy...@hortonworks.com> wrote:

    +1 for merging HDDS-48 to trunk. 
    
    On 6/29/18, 9:34 AM, "Arpit Agarwal" <aa...@hortonworks.com> wrote:
    
        +1 for merging this branch.
        
        Added common-dev@
        
        
        On 6/28/18, 3:16 PM, "Bharat Viswanadham" <bv...@hortonworks.com> wrote:
        
            Hi everyone,
            
            I’d like to start a thread to discuss merging the HDDS-48 branch to trunk. The ContainerIO work refactors the HDDS Datanode IO path to enforce clean separation between the Container management and the Storage layers.
            
            Note: HDDS/Ozone code is not compiled by default in trunk. The 'hdds' maven profile must be enabled to compile the branch payload.
             
            The merge payload includes the following key improvements:
            1. Support multiple container types on the datanode.
            2. Adopt a new disk layout for the containers that supports future upgrades.
            3. Support volume Choosing policy for container data locations.
            4. Changed the format of the .container file to a human-readable format (yaml)
             
            Below are the links for design documents attached to HDDS-48.
             
            https://issues.apache.org/jira/secure/attachment/12923107/ContainerIO-StorageManagement-DesignDoc.pdf
            https://issues.apache.org/jira/secure/attachment/12923108/HDDS DataNode Disk Layout.pdf
             
            The branch is ready to merge. Over the next week we will clean up the unused classes, fix old integration tests and continue testing the changes.
             
            Thanks to Hanisha Koneru, Arpit Agarwal, Anu Engineer, Jitendra Pandey,  Xiaoyu Yao, Ajay Kumar, Mukul Kumar Singh, Marton Elek and Shashikant Banerjee for their contributions in design, development and code reviews.
            
            Thanks,
            Bharat
            
            
            
            ---------------------------------------------------------------------
            To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
            For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
            
            
        
        
        ---------------------------------------------------------------------
        To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
        For additional commands, e-mail: common-dev-help@hadoop.apache.org
        
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
    For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Posted by Xiaoyu Yao <xy...@hortonworks.com>.
+1 for merging HDDS-48 to trunk. 

On 6/29/18, 9:34 AM, "Arpit Agarwal" <aa...@hortonworks.com> wrote:

    +1 for merging this branch.
    
    Added common-dev@
    
    
    On 6/28/18, 3:16 PM, "Bharat Viswanadham" <bv...@hortonworks.com> wrote:
    
        Hi everyone,
        
        I’d like to start a thread to discuss merging the HDDS-48 branch to trunk. The ContainerIO work refactors the HDDS Datanode IO path to enforce clean separation between the Container management and the Storage layers.
        
        Note: HDDS/Ozone code is not compiled by default in trunk. The 'hdds' maven profile must be enabled to compile the branch payload.
         
        The merge payload includes the following key improvements:
        1. Support multiple container types on the datanode.
        2. Adopt a new disk layout for the containers that supports future upgrades.
        3. Support volume Choosing policy for container data locations.
        4. Changed the format of the .container file to a human-readable format (yaml)
         
        Below are the links for design documents attached to HDDS-48.
         
        https://issues.apache.org/jira/secure/attachment/12923107/ContainerIO-StorageManagement-DesignDoc.pdf
        https://issues.apache.org/jira/secure/attachment/12923108/HDDS DataNode Disk Layout.pdf
         
        The branch is ready to merge. Over the next week we will clean up the unused classes, fix old integration tests and continue testing the changes.
         
        Thanks to Hanisha Koneru, Arpit Agarwal, Anu Engineer, Jitendra Pandey,  Xiaoyu Yao, Ajay Kumar, Mukul Kumar Singh, Marton Elek and Shashikant Banerjee for their contributions in design, development and code reviews.
        
        Thanks,
        Bharat
        
        
        
        ---------------------------------------------------------------------
        To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
        For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
        
        
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
    For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Posted by Xiaoyu Yao <xy...@hortonworks.com>.
+1 for merging HDDS-48 to trunk. 

On 6/29/18, 9:34 AM, "Arpit Agarwal" <aa...@hortonworks.com> wrote:

    +1 for merging this branch.
    
    Added common-dev@
    
    
    On 6/28/18, 3:16 PM, "Bharat Viswanadham" <bv...@hortonworks.com> wrote:
    
        Hi everyone,
        
        I’d like to start a thread to discuss merging the HDDS-48 branch to trunk. The ContainerIO work refactors the HDDS Datanode IO path to enforce clean separation between the Container management and the Storage layers.
        
        Note: HDDS/Ozone code is not compiled by default in trunk. The 'hdds' maven profile must be enabled to compile the branch payload.
         
        The merge payload includes the following key improvements:
        1. Support multiple container types on the datanode.
        2. Adopt a new disk layout for the containers that supports future upgrades.
        3. Support volume Choosing policy for container data locations.
        4. Changed the format of the .container file to a human-readable format (yaml)
         
        Below are the links for design documents attached to HDDS-48.
         
        https://issues.apache.org/jira/secure/attachment/12923107/ContainerIO-StorageManagement-DesignDoc.pdf
        https://issues.apache.org/jira/secure/attachment/12923108/HDDS DataNode Disk Layout.pdf
         
        The branch is ready to merge. Over the next week we will clean up the unused classes, fix old integration tests and continue testing the changes.
         
        Thanks to Hanisha Koneru, Arpit Agarwal, Anu Engineer, Jitendra Pandey,  Xiaoyu Yao, Ajay Kumar, Mukul Kumar Singh, Marton Elek and Shashikant Banerjee for their contributions in design, development and code reviews.
        
        Thanks,
        Bharat
        
        
        
        ---------------------------------------------------------------------
        To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
        For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
        
        
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
    For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Posted by Arpit Agarwal <aa...@hortonworks.com>.
+1 for merging this branch.

Added common-dev@


On 6/28/18, 3:16 PM, "Bharat Viswanadham" <bv...@hortonworks.com> wrote:

    Hi everyone,
    
    I’d like to start a thread to discuss merging the HDDS-48 branch to trunk. The ContainerIO work refactors the HDDS Datanode IO path to enforce clean separation between the Container management and the Storage layers.
    
    Note: HDDS/Ozone code is not compiled by default in trunk. The 'hdds' maven profile must be enabled to compile the branch payload.
     
    The merge payload includes the following key improvements:
    1. Support multiple container types on the datanode.
    2. Adopt a new disk layout for the containers that supports future upgrades.
    3. Support volume Choosing policy for container data locations.
    4. Changed the format of the .container file to a human-readable format (yaml)
     
    Below are the links for design documents attached to HDDS-48.
     
    https://issues.apache.org/jira/secure/attachment/12923107/ContainerIO-StorageManagement-DesignDoc.pdf
    https://issues.apache.org/jira/secure/attachment/12923108/HDDS DataNode Disk Layout.pdf
     
    The branch is ready to merge. Over the next week we will clean up the unused classes, fix old integration tests and continue testing the changes.
     
    Thanks to Hanisha Koneru, Arpit Agarwal, Anu Engineer, Jitendra Pandey,  Xiaoyu Yao, Ajay Kumar, Mukul Kumar Singh, Marton Elek and Shashikant Banerjee for their contributions in design, development and code reviews.
    
    Thanks,
    Bharat
    
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
    For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [DISCUSS]Merge ContainerIO branch (HDDS-48) in to trunk

Posted by Arpit Agarwal <aa...@hortonworks.com>.
+1 for merging this branch.

Added common-dev@


On 6/28/18, 3:16 PM, "Bharat Viswanadham" <bv...@hortonworks.com> wrote:

    Hi everyone,
    
    I’d like to start a thread to discuss merging the HDDS-48 branch to trunk. The ContainerIO work refactors the HDDS Datanode IO path to enforce clean separation between the Container management and the Storage layers.
    
    Note: HDDS/Ozone code is not compiled by default in trunk. The 'hdds' maven profile must be enabled to compile the branch payload.
     
    The merge payload includes the following key improvements:
    1. Support multiple container types on the datanode.
    2. Adopt a new disk layout for the containers that supports future upgrades.
    3. Support volume Choosing policy for container data locations.
    4. Changed the format of the .container file to a human-readable format (yaml)
     
    Below are the links for design documents attached to HDDS-48.
     
    https://issues.apache.org/jira/secure/attachment/12923107/ContainerIO-StorageManagement-DesignDoc.pdf
    https://issues.apache.org/jira/secure/attachment/12923108/HDDS DataNode Disk Layout.pdf
     
    The branch is ready to merge. Over the next week we will clean up the unused classes, fix old integration tests and continue testing the changes.
     
    Thanks to Hanisha Koneru, Arpit Agarwal, Anu Engineer, Jitendra Pandey,  Xiaoyu Yao, Ajay Kumar, Mukul Kumar Singh, Marton Elek and Shashikant Banerjee for their contributions in design, development and code reviews.
    
    Thanks,
    Bharat
    
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
    For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org