You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by tweise <gi...@git.apache.org> on 2016/03/13 08:50:39 UTC
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
GitHub user tweise opened a pull request:
https://github.com/apache/incubator-apex-site/pull/19
APEXCORE-319 Document compatibility guidelines.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tweise/incubator-apex-site master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-apex-site/pull/19.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19
----
commit 3e7c325e1eff825d94e12b0fc93cbe733c10cf26
Author: Thomas Weise <th...@datatorrent.com>
Date: 2016-03-13T07:49:12Z
APEXCORE-319 Document compatibility guidelines.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by PramodSSImmaneni <gi...@git.apache.org>.
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56080429
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
--- End diff --
For API the policy mentioned above is
"Public APIs must be deprecated for at least one major release prior to their removal in a major release."
For CLI the policy mentioned is
"CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release."
I read this as keeping old APIs for an extra major release in a deprecated capacity, so for example if it is deprecated towards the end of the 3.x lifecycle it gets removed in 5.x and not 4.x. Looks like that is not what you are intending to say, then can we make the wording of the two statements above similar so as to avoid confusion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by tweise <gi...@git.apache.org>.
Github user tweise commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56071669
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
--- End diff --
They can only be removed in next major release. I don't understand the concern?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by davidyan74 <gi...@git.apache.org>.
Github user davidyan74 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56070632
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
--- End diff --
I think we don't need the "s" in backward-compatible
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by PramodSSImmaneni <gi...@git.apache.org>.
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56079841
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
--- End diff --
Not suggesting to undo a release. A bug fix is fine unless it changes the behavior that user relies upon. For example if you are throwing an exception with an error code and you made a mistake by reusing same error code for two different error conditions, you cannot "fix" it by just changing the error code, the caller may have already added error handling code for the two scenarios for the old error code. Think about mkdir() for example, in non mapr versions it creates all folders in the path if they don't exist but mapr sticks to the posix standard. Lets say you had such a function that didn't implement the standard correctly the first time and you want to change it, that would break user code. A new method should be created to handle such scenarios. Golden rule don't break user code (not my words).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by davidyan74 <gi...@git.apache.org>.
Github user davidyan74 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56071442
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
+
+###Configuration Files
+
+Configuration files are used for engine or application settings. Changes to keys and default values affect users.
+
+Policy
+
+Name, location, format, keys of configuration files should be deprecated in a prior minor release and can only be changed in major release. Best effort should be made to support the deprecated behavior for one more major release (not guaranteed).
+
+###Internal Wire compatibility
+
+Apex containers internally use RPC communication and netlet for the data flow. The protocols are private and user components are not exposed to it. Apex is a YARN application and automatically deployed. There is currently no situation where containers of different Apex engine versions need to be interoperable. Should such scenario become relevant in the future, wire compatibility needs to be specified.
+
+Policy
+
+N/A
+
+###Internal File formats
+
+Apex engine stores data in the file system for recovery. When using serialization (Kryo, Java etc.), changes to internal classes may affect the ability to relaunch an application with upgraded engine code from previous state. This is currently not supported. In the future, the serialization mechanism should guarantee backward compatibility.
+
+Policy
+
+Currently no compatibility guarantee. User to cold-restart application on engine upgrade.
+
+###Java Classpath
+
+Apex applications should not bundle Hadoop dependencies or Apex engine dependencies but use the dependencies provided in the target environment to avoid conflicts. The Apex application archetype can be used to generate a compliant project.
+
+Policy
+
+Apex engine dependencies can change as per semantic versioning.
+
+###Maven Build Artifacts
+
+Downstream projects reference the Apex engine dependencies and Malhar operator libraries for application development etc. Changes to the packaging (which classes are in which jar), the groupId, artifactId and which artifacts are deployed to Maven central impact upgrades.
+
+Policy
+
+The artifacts that contain the classes that form the public API as specified above cannot change on patch releases and should stay compatible within a major release.
--- End diff --
If we don't even allow patch version change in the dependency artifacts, maybe we should explicitly say that?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by davidyan74 <gi...@git.apache.org>.
Github user davidyan74 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56071559
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
+
+###Configuration Files
+
+Configuration files are used for engine or application settings. Changes to keys and default values affect users.
+
+Policy
+
+Name, location, format, keys of configuration files should be deprecated in a prior minor release and can only be changed in major release. Best effort should be made to support the deprecated behavior for one more major release (not guaranteed).
+
+###Internal Wire compatibility
+
+Apex containers internally use RPC communication and netlet for the data flow. The protocols are private and user components are not exposed to it. Apex is a YARN application and automatically deployed. There is currently no situation where containers of different Apex engine versions need to be interoperable. Should such scenario become relevant in the future, wire compatibility needs to be specified.
+
+Policy
+
+N/A
+
+###Internal File formats
+
+Apex engine stores data in the file system for recovery. When using serialization (Kryo, Java etc.), changes to internal classes may affect the ability to relaunch an application with upgraded engine code from previous state. This is currently not supported. In the future, the serialization mechanism should guarantee backward compatibility.
+
+Policy
+
+Currently no compatibility guarantee. User to cold-restart application on engine upgrade.
+
+###Java Classpath
+
+Apex applications should not bundle Hadoop dependencies or Apex engine dependencies but use the dependencies provided in the target environment to avoid conflicts. The Apex application archetype can be used to generate a compliant project.
+
+Policy
+
+Apex engine dependencies can change as per semantic versioning.
+
+###Maven Build Artifacts
+
+Downstream projects reference the Apex engine dependencies and Malhar operator libraries for application development etc. Changes to the packaging (which classes are in which jar), the groupId, artifactId and which artifacts are deployed to Maven central impact upgrades.
+
+Policy
+
+The artifacts that contain the classes that form the public API as specified above cannot change on patch releases and should stay compatible within a major release.
+
+###Hardware/Software Requirements
+
+Apex depends on Apache Hadoop. The community intends to support all major Hadoop distros and current versions. Apex currently supports Hadoop 2.2.0 and higher and Java 7 and higher. Apex is written in Java and has been tested on Linux based Hadoop clusters. There are no additional restrictions on the hardware architecture.
--- End diff --
Should we state minimum memory requirement?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by PramodSSImmaneni <gi...@git.apache.org>.
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56012382
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
+
+###Configuration Files
+
+Configuration files are used for engine or application settings. Changes to keys and default values affect users.
+
+Policy
+
+Name, location, format, keys of configuration files should be deprecated in a prior minor release and can only be changed in major release. Best effort should be made to support the deprecated behavior for one more major release (not guaranteed).
+
+###Internal Wire compatibility
+
+Apex containers internally use RPC communication and netlet for the data flow. The protocols are private and user components are not exposed to it. Apex is a YARN application and automatically deployed. There is currently no situation where containers of different Apex engine versions need to be interoperable. Should such scenario become relevant in the future, wire compatibility needs to be specified.
+
+Policy
+
+N/A
+
+###Internal File formats
+
+Apex engine stores data in the file system for recovery. When using serialization (Kryo, Java etc.), changes to internal classes may affect the ability to relaunch an application with upgraded engine code from previous state. This is currently not supported. In the future, the serialization mechanism should guarantee backward compatibility.
--- End diff --
How about saying "Apex engine stores data in the file system for recovery and the data is typically obtained from serialization (from Kryo, Java etc.). Changes..." as it is a little more clearer.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by tweise <gi...@git.apache.org>.
Github user tweise commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56679593
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
+
+###Configuration Files
+
+Configuration files are used for engine or application settings. Changes to keys and default values affect users.
+
+Policy
+
+Name, location, format, keys of configuration files should be deprecated in a prior minor release and can only be changed in major release. Best effort should be made to support the deprecated behavior for one more major release (not guaranteed).
+
+###Internal Wire compatibility
+
+Apex containers internally use RPC communication and netlet for the data flow. The protocols are private and user components are not exposed to it. Apex is a YARN application and automatically deployed. There is currently no situation where containers of different Apex engine versions need to be interoperable. Should such scenario become relevant in the future, wire compatibility needs to be specified.
+
+Policy
+
+N/A
+
+###Internal File formats
+
+Apex engine stores data in the file system for recovery. When using serialization (Kryo, Java etc.), changes to internal classes may affect the ability to relaunch an application with upgraded engine code from previous state. This is currently not supported. In the future, the serialization mechanism should guarantee backward compatibility.
+
+Policy
+
+Currently no compatibility guarantee. User to cold-restart application on engine upgrade.
+
+###Java Classpath
+
+Apex applications should not bundle Hadoop dependencies or Apex engine dependencies but use the dependencies provided in the target environment to avoid conflicts. The Apex application archetype can be used to generate a compliant project.
+
+Policy
+
+Apex engine dependencies can change as per semantic versioning.
+
+###Maven Build Artifacts
+
+Downstream projects reference the Apex engine dependencies and Malhar operator libraries for application development etc. Changes to the packaging (which classes are in which jar), the groupId, artifactId and which artifacts are deployed to Maven central impact upgrades.
+
+Policy
+
+The artifacts that contain the classes that form the public API as specified above cannot change on patch releases and should stay compatible within a major release.
+
+###Hardware/Software Requirements
+
+Apex depends on Apache Hadoop. The community intends to support all major Hadoop distros and current versions. Apex currently supports Hadoop 2.2.0 and higher and Java 7 and higher. Apex is written in Java and has been tested on Linux based Hadoop clusters. There are no additional restrictions on the hardware architecture.
--- End diff --
Memory requirements are primarily driven by application specific factors and Hadoop infrastructure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by davidyan74 <gi...@git.apache.org>.
Github user davidyan74 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56069972
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
--- End diff --
Depending "on"?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-apex-site/pull/19
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by PramodSSImmaneni <gi...@git.apache.org>.
Github user PramodSSImmaneni commented on the pull request:
https://github.com/apache/incubator-apex-site/pull/19#issuecomment-198470912
Looks good Thomas, thanks for addressing the comments
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by tweise <gi...@git.apache.org>.
Github user tweise commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56073541
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
+
+###Configuration Files
+
+Configuration files are used for engine or application settings. Changes to keys and default values affect users.
+
+Policy
+
+Name, location, format, keys of configuration files should be deprecated in a prior minor release and can only be changed in major release. Best effort should be made to support the deprecated behavior for one more major release (not guaranteed).
+
+###Internal Wire compatibility
+
+Apex containers internally use RPC communication and netlet for the data flow. The protocols are private and user components are not exposed to it. Apex is a YARN application and automatically deployed. There is currently no situation where containers of different Apex engine versions need to be interoperable. Should such scenario become relevant in the future, wire compatibility needs to be specified.
+
+Policy
+
+N/A
+
+###Internal File formats
+
+Apex engine stores data in the file system for recovery. When using serialization (Kryo, Java etc.), changes to internal classes may affect the ability to relaunch an application with upgraded engine code from previous state. This is currently not supported. In the future, the serialization mechanism should guarantee backward compatibility.
+
+Policy
+
+Currently no compatibility guarantee. User to cold-restart application on engine upgrade.
+
+###Java Classpath
+
+Apex applications should not bundle Hadoop dependencies or Apex engine dependencies but use the dependencies provided in the target environment to avoid conflicts. The Apex application archetype can be used to generate a compliant project.
+
+Policy
+
+Apex engine dependencies can change as per semantic versioning.
+
+###Maven Build Artifacts
+
+Downstream projects reference the Apex engine dependencies and Malhar operator libraries for application development etc. Changes to the packaging (which classes are in which jar), the groupId, artifactId and which artifacts are deployed to Maven central impact upgrades.
+
+Policy
+
+The artifacts that contain the classes that form the public API as specified above cannot change on patch releases and should stay compatible within a major release.
--- End diff --
Patch versions can move dependencies at patch level.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by tweise <gi...@git.apache.org>.
Github user tweise commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56071320
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
--- End diff --
We should be able to correct a bug. Consider a bug introduced in a minor release, we cannot undo the release. We can only correct it in a future release.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by PramodSSImmaneni <gi...@git.apache.org>.
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56080616
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
+
+###Configuration Files
+
+Configuration files are used for engine or application settings. Changes to keys and default values affect users.
+
+Policy
+
+Name, location, format, keys of configuration files should be deprecated in a prior minor release and can only be changed in major release. Best effort should be made to support the deprecated behavior for one more major release (not guaranteed).
--- End diff --
Sounds good. How about considering a migration tool for the properties in the policy?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by PramodSSImmaneni <gi...@git.apache.org>.
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56013223
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
+
+###Configuration Files
+
+Configuration files are used for engine or application settings. Changes to keys and default values affect users.
+
+Policy
+
+Name, location, format, keys of configuration files should be deprecated in a prior minor release and can only be changed in major release. Best effort should be made to support the deprecated behavior for one more major release (not guaranteed).
+
+###Internal Wire compatibility
+
+Apex containers internally use RPC communication and netlet for the data flow. The protocols are private and user components are not exposed to it. Apex is a YARN application and automatically deployed. There is currently no situation where containers of different Apex engine versions need to be interoperable. Should such scenario become relevant in the future, wire compatibility needs to be specified.
+
+Policy
+
+N/A
+
+###Internal File formats
+
+Apex engine stores data in the file system for recovery. When using serialization (Kryo, Java etc.), changes to internal classes may affect the ability to relaunch an application with upgraded engine code from previous state. This is currently not supported. In the future, the serialization mechanism should guarantee backward compatibility.
+
+Policy
+
+Currently no compatibility guarantee. User to cold-restart application on engine upgrade.
+
+###Java Classpath
+
+Apex applications should not bundle Hadoop dependencies or Apex engine dependencies but use the dependencies provided in the target environment to avoid conflicts. The Apex application archetype can be used to generate a compliant project.
+
+Policy
+
+Apex engine dependencies can change as per semantic versioning.
+
+###Maven Build Artifacts
+
+Downstream projects reference the Apex engine dependencies and Malhar operator libraries for application development etc. Changes to the packaging (which classes are in which jar), the groupId, artifactId and which artifacts are deployed to Maven central impact upgrades.
--- End diff --
Good point
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by PramodSSImmaneni <gi...@git.apache.org>.
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56013080
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
+
+###Configuration Files
+
+Configuration files are used for engine or application settings. Changes to keys and default values affect users.
+
+Policy
+
+Name, location, format, keys of configuration files should be deprecated in a prior minor release and can only be changed in major release. Best effort should be made to support the deprecated behavior for one more major release (not guaranteed).
+
+###Internal Wire compatibility
+
+Apex containers internally use RPC communication and netlet for the data flow. The protocols are private and user components are not exposed to it. Apex is a YARN application and automatically deployed. There is currently no situation where containers of different Apex engine versions need to be interoperable. Should such scenario become relevant in the future, wire compatibility needs to be specified.
+
+Policy
+
+N/A
+
+###Internal File formats
+
+Apex engine stores data in the file system for recovery. When using serialization (Kryo, Java etc.), changes to internal classes may affect the ability to relaunch an application with upgraded engine code from previous state. This is currently not supported. In the future, the serialization mechanism should guarantee backward compatibility.
+
+Policy
+
+Currently no compatibility guarantee. User to cold-restart application on engine upgrade.
+
+###Java Classpath
+
+Apex applications should not bundle Hadoop dependencies or Apex engine dependencies but use the dependencies provided in the target environment to avoid conflicts. The Apex application archetype can be used to generate a compliant project.
+
+Policy
+
+Apex engine dependencies can change as per semantic versioning.
--- End diff --
How about explicitly saying that following the practice described above automatically maintains the backwards compatibility that following semantic versioning of Apex
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by tweise <gi...@git.apache.org>.
Github user tweise commented on the pull request:
https://github.com/apache/incubator-apex-site/pull/19#issuecomment-198427975
@PramodSSImmaneni @davidyan74 addressed comments, please check.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by PramodSSImmaneni <gi...@git.apache.org>.
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56011204
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
+
+###Configuration Files
+
+Configuration files are used for engine or application settings. Changes to keys and default values affect users.
+
+Policy
+
+Name, location, format, keys of configuration files should be deprecated in a prior minor release and can only be changed in major release. Best effort should be made to support the deprecated behavior for one more major release (not guaranteed).
--- End diff --
Why not have uniform deprecation policy for all the units
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by tweise <gi...@git.apache.org>.
Github user tweise commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56071965
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
+
+###Configuration Files
+
+Configuration files are used for engine or application settings. Changes to keys and default values affect users.
+
+Policy
+
+Name, location, format, keys of configuration files should be deprecated in a prior minor release and can only be changed in major release. Best effort should be made to support the deprecated behavior for one more major release (not guaranteed).
--- End diff --
Configuration file changes have the largest impact and are hardest to spot (compare to a compile error). Hence my suggestion is to extend deprecation to extra major release.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by tweise <gi...@git.apache.org>.
Github user tweise commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56676627
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
--- End diff --
Thanks for catching this, deprecation can be in prior minor release.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by PramodSSImmaneni <gi...@git.apache.org>.
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56011075
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
+
+###REST API
+
+REST API compatibility corresponds to both the URLs and request/response content over the wire. REST APIs are specifically meant for stable use by clients across releases, even major releases.
+
+Policy
+
+The REST API is separately versioned. This is to allow for co-existence of old and new API should there be a need for backward incompatible changes in the future.
+
+###Command Line Interface (CLI)
+
+The CLI may be used either directly via the system shell or via shell scripts. Changing the path, removing or renaming command line options, the order of arguments, or the command return code and output break compatibility and may adversely affect users.
+
+Policy
+
+CLI commands are to be deprecated (warning when used) in a prior minor release before they are removed or incompatibly modified in a subsequent major release.
--- End diff --
What are CLI commands not preserved for one major version in deprecated capacity like the APIs
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by davidyan74 <gi...@git.apache.org>.
Github user davidyan74 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56070892
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
--- End diff --
I agree with pramod
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-apex-site pull request: APEXCORE-319 Document compatibil...
Posted by PramodSSImmaneni <gi...@git.apache.org>.
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-site/pull/19#discussion_r56010691
--- Diff: src/md/compatibility.md ---
@@ -0,0 +1,112 @@
+#Apache Apex Compatibility
+
+##Purpose
+
+This document captures the compatibility goals of the Apache Apex project. The different types of compatibility between Apex releases that affect contributors, downstream projects, and end-users are enumerated. For each type of compatibility we:
+
+* describe the impact on downstream projects or end-users
+* where applicable, call out the policy adopted when incompatible changes are permitted.
+
+Apache Apex follows [semantic versioning](http://semver.org/). Depending in the compatibility type, there may be different tools or mechanisms to ensure compatibility, for example by comparing artifacts during the build process.
+
+The type of change will inform the required target version number. Given a version number MAJOR.MINOR.PATCH, increment the:
+
+* MAJOR version when you make incompatible API changes,
+* MINOR version when you add functionality in a backwards-compatible manner, and
+* PATCH version when you make backwards-compatible bug fixes.
+
+Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+The overall goal is to avoid backward incompatible changes and major release upgrades. Accordingly we attempt to release new features with minor versions that are incremental to the prior release and offer our users a frictionless upgrade path. When planning contributions, please consider compatibility and release road map upfront. Specifically, certain changes that conflict with the versioning may need to be documented in JIRA and deferred until a future major release.
+
+##Compatibility types
+
+###Java API
+
+Public API compatibility is required to ensure end-user programs and downstream projects continue to work without modification.
+The public API consists of:
+
+* apex-core: all interfaces and classes in `api` and `common` modules
+* apex-malhar: all interfaces and classes in all modules except `demos`, `samples`, `benchmark`
+
+Interfaces and classes that are part of the public API and are annotated with [interface stability](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html) are treated according to the rules defined by the annotation.
+
+Policy
+
+Changes to the public API must follow semantic versioning.
+Public APIs must be deprecated for at least one major release prior to their removal in a major release.
+The japicmp plugin is used to enforce compatibility as part of the Travis pre-commit builds.
+
+###Semantic compatibility
+
+The behavior of APIs needs to remain consistent over versions, though changes for correctness may result in changes in behavior. Tests and javadocs specify the behavior. Over time, test suites should be expanded to verify compliance with the specification, effectively creating a formal specification for the subset of behaviors that can be easily tested.
+
+Policy
+
+The behavior of API may be changed to fix incorrect behavior, changes to be accompanied by tests coverage for the exact behavior.
--- End diff --
Thomas I am not sure if we should allow for behavior changes in same major version as code may already have been written that relies on supposedly "incorrect" behavior.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---