You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/04/08 19:21:24 UTC
[GitHub] [ozone] swagle commented on a change in pull request #2133: HDDS-5014. Upgrade usage primer documentation.
swagle commented on a change in pull request #2133:
URL: https://github.com/apache/ozone/pull/2133#discussion_r610025453
##########
File path: hadoop-hdds/docs/content/design/upgrade-dev-primer.md
##########
@@ -0,0 +1,96 @@
+---
+title: Upgrade - Developer Primer
+summary: Helpful resources for those who are bringing layout changes.
+date: 2021-02-15
+author: Aravindan Vijayan
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# Terminologies
+
+## Layout Feature
+A layout feature is any new Ozone feature that makes a backward incompatible change to the on disk layout. Each layout feature is associated with a layout version that it defines. A component has a list of monotonically increasing layout features (versions) that it supports.
+
+## Finalizing & Pre-Finalized State
+When upgrading a component from an older version to a newer version which has a higher layout version, the component automatically goes into a pre-finalized state after which an explicit ‘finalize’ action is required from the user to finalize it. In the pre-finalized state, commands/APIs/on disk structures used and created by newer layout features are meant to be unsupported or unused. After finalizing, the newer layout feature APIs are supported.
+
+## Downgrade
+Downgrade to a lower version is allowed from the pre-finalized state. This involves stopping the component, replacing the artifacts to the older version, and then starting it up again.
+
+# Useful framework tools to use
+
+## LayoutFeature
+ org.apache.hadoop.ozone.om.upgrade.OMLayoutFeature
+ org.apache.hadoop.hdds.upgrade.HDDSLayoutFeature
+Class to add a new layout feature being brought in. Layout version is typically 1 + last layout feature in that catalog.
+
+## LayoutVersionManager
+ org.apache.hadoop.ozone.om.upgrade.OMLayoutVersionManager
+ org.apache.hadoop.hdds.upgrade.HDDSLayoutVersionManager
+Every component carries an instance of this interface, which provides APIs to get runtime layout version, and if a feature is allowed based on that or not.
+
+## @DisallowedUntilLayoutVersion Annotation
+Method level annotation used to "disallow" an API if current layout version does not include the associated layout feature. Currently it is added only to the OM module, but can easily be moved down to a common module based on need on the HDDS layer.
+
+## @BelongsToLayoutVersion Annotation
+Annotation to mark an OM request class that it belongs to a specific Layout Version. Until that version is available post finalize, this request will not be supported. A newer version of an existing OM request can be created (by inheritance or a fully new class) and marked with a newer layout version. Until finalizing this layout version, the older request class is used. Post finalizing, the newer version of the request class is used.
+
+## LayoutVersionInstanceFactory<T>
+Generic factory which stores different instances of Type 'T' sharded by a key & version. A single key can be associated with different versions of 'T'.
+
+### Why does this class exist?
+A typical use case during upgrade is to have multiple versions of a class / method / object and choose them based on the current layout version at runtime. Before finalizing, an older version is typically needed, and after finalization, a newer version is needed. This class serves this purpose in a generic way. For example, we can create a Factory to create multiple versions of OMRequests sharded by Request Type & Layout Version Supported.
+
+## Upgrade Action (UpgradeActionOm & UpgradeActionHdds)
+Annotation to specify upgrade action run during specific upgrade phases. Each layout feature can optionally define an upgrade action for every supported phase. These are the supported phases of action callbacks.
+
+#### UNFINALIZED_STATE_VALIDATION
+A ‘validation’ action run every time a component is started up with this layout feature being unfinalized.
+
+#### ON_FIRST_UPGRADE_START
Review comment:
Are these actions expected to be idempotent, what is the behavior on failure of this action?
##########
File path: hadoop-hdds/docs/content/design/upgrade-dev-primer.md
##########
@@ -0,0 +1,96 @@
+---
+title: Upgrade - Developer Primer
+summary: Helpful resources for those who are bringing layout changes.
+date: 2021-02-15
+author: Aravindan Vijayan
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# Terminologies
+
+## Layout Feature
+A layout feature is any new Ozone feature that makes a backward incompatible change to the on disk layout. Each layout feature is associated with a layout version that it defines. A component has a list of monotonically increasing layout features (versions) that it supports.
+
+## Finalizing & Pre-Finalized State
+When upgrading a component from an older version to a newer version which has a higher layout version, the component automatically goes into a pre-finalized state after which an explicit ‘finalize’ action is required from the user to finalize it. In the pre-finalized state, commands/APIs/on disk structures used and created by newer layout features are meant to be unsupported or unused. After finalizing, the newer layout feature APIs are supported.
+
+## Downgrade
+Downgrade to a lower version is allowed from the pre-finalized state. This involves stopping the component, replacing the artifacts to the older version, and then starting it up again.
+
+# Useful framework tools to use
+
+## LayoutFeature
+ org.apache.hadoop.ozone.om.upgrade.OMLayoutFeature
+ org.apache.hadoop.hdds.upgrade.HDDSLayoutFeature
+Class to add a new layout feature being brought in. Layout version is typically 1 + last layout feature in that catalog.
+
+## LayoutVersionManager
+ org.apache.hadoop.ozone.om.upgrade.OMLayoutVersionManager
+ org.apache.hadoop.hdds.upgrade.HDDSLayoutVersionManager
+Every component carries an instance of this interface, which provides APIs to get runtime layout version, and if a feature is allowed based on that or not.
+
+## @DisallowedUntilLayoutVersion Annotation
+Method level annotation used to "disallow" an API if current layout version does not include the associated layout feature. Currently it is added only to the OM module, but can easily be moved down to a common module based on need on the HDDS layer.
+
+## @BelongsToLayoutVersion Annotation
+Annotation to mark an OM request class that it belongs to a specific Layout Version. Until that version is available post finalize, this request will not be supported. A newer version of an existing OM request can be created (by inheritance or a fully new class) and marked with a newer layout version. Until finalizing this layout version, the older request class is used. Post finalizing, the newer version of the request class is used.
+
+## LayoutVersionInstanceFactory<T>
+Generic factory which stores different instances of Type 'T' sharded by a key & version. A single key can be associated with different versions of 'T'.
+
+### Why does this class exist?
+A typical use case during upgrade is to have multiple versions of a class / method / object and choose them based on the current layout version at runtime. Before finalizing, an older version is typically needed, and after finalization, a newer version is needed. This class serves this purpose in a generic way. For example, we can create a Factory to create multiple versions of OMRequests sharded by Request Type & Layout Version Supported.
+
+## Upgrade Action (UpgradeActionOm & UpgradeActionHdds)
+Annotation to specify upgrade action run during specific upgrade phases. Each layout feature can optionally define an upgrade action for every supported phase. These are the supported phases of action callbacks.
+
+#### UNFINALIZED_STATE_VALIDATION
+A ‘validation’ action run every time a component is started up with this layout feature being unfinalized.
+
+#### ON_FIRST_UPGRADE_START
+A backward compatible action run exactly once when an upgraded cluster is detected with this new layout version.
+
+#### ON_FINALIZE
Review comment:
Same comment as above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org