[jira] [Commented] (NIFI-2854) Enable repositories to support upgrades and rollback in well defined scenarios

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15654799#comment-15654799 ] 

ASF GitHub Bot commented on NIFI-2854:
--------------------------------------

GitHub user markap14 opened a pull request:

    https://github.com/apache/nifi/pull/1202

    NIFI-2854: Refactor repositories and swap files to use schema-based s…

    Thank you for submitting a contribution to Apache NiFi.
    
    In order to streamline the review of the contribution we ask you
    to ensure the following steps have been taken:
    
    ### For all changes:
    - [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
         in the commit message?
    
    - [ ] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
    
    - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
    
    - [ ] Is your initial contribution a single, squashed commit?
    
    ### For code changes:
    - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
    - [ ] Have you written or updated unit tests to verify your changes?
    - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? 
    - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
    - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
    - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
    
    ### For documentation related changes:
    - [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
    
    ### Note:
    Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
    
    …erialization so that nifi can be rolled back to a previous version after an upgrade.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/markap14/nifi NIFI-2854

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/nifi/pull/1202.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1202
    
----
commit 5167505596e505ac89ed1fe975388ad2f35a963e
Author: Mark Payne <ma...@hotmail.com>
Date:   2016-10-04T13:38:14Z

    NIFI-2854: Refactor repositories and swap files to use schema-based serialization so that nifi can be rolled back to a previous version after an upgrade.

----


> Enable repositories to support upgrades and rollback in well defined scenarios
> ------------------------------------------------------------------------------
>
>                 Key: NIFI-2854
>                 URL: https://issues.apache.org/jira/browse/NIFI-2854
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>             Fix For: 1.2.0
>
>
> The flowfile, swapfile, provenance, and content repositories play a very important roll in NiFi's ability to be safely upgraded and rolled back.  We need to have well documented behaviors, designs, and version adherence so that users can safely rely on these mechanisms.
> Once this is formalized and in place we should update our versioning guidance to reflect this as well.
> The following would be true from NiFi 1.2.0 onward
> * No changes to how the repositories are persisted to disk can be made which will break forward/backward compatibility and specifically this means that things like the way each is serialized to disk cannot change.
> * If changes are made which impact forward or backward compatibility they should be reserved for major releases only and should include a utility to help users with pre-existing data convert from some older format to the newer format.  It may not be feasible to have rollback on major releases.
> * The content repository should not be changed within a major release cycle in any way that will harm forward or backward compatibility.
> * The flow file repository can change in that new fields can be added to existing write ahead log record types but no fields can be removed nor can any new types be added.  Once a field is considered required it must remain required.  Changes may only be made across minor version changes - not incremental.
> * Swap File storage should follow very similar rules to the flow file repository.  Adding a schema to the swap file header may allow some variation there but the variation should only be hints to optimize how they're processed and not change their behavior otherwise. Changes are only permitted during minor version releases.
> * Provenance repository changes are only permitted during minor version releases.  These changes may include adding or removing fields from existing event types.  If a field is considered required it must always be considered required.  If a field is removed then it must not be a required field and there must be a sensible default an older version could use if that value is not found in new data once rolled back.  New event types may be added.  Fields or event types not known to older version, if seen after a rollback, will simply be ignored.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)