You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by Steve Lawrence <sl...@apache.org> on 2017/10/17 15:34:28 UTC
Code Contributor and Branch Workflow Proposals
I don't think there were any major objections on the infrastructure
discussion [1], so I'd like to propose a contributor and branch workflow
using that infrastructure.
== Code Contributor Workflow ==
This is the workflow used by all people that wish to contribute code to
Daffodil, regardless of if they are a committer or not. There are other
types of contributors (e.g. wiki, mailing list support, testing, etc.)
that this does not cover.
1. Create a bug in Daffodil JIRA, and assign it to yourself.
2. Create a github fork of the Daffodil github mirror and clone it.
This remote will be your 'origin' remote:
$ git clone git@github.com:username/daffodil.git
3. Add the upstream repository as a new remote, calling it 'upstream':
$ git remote add upstream git://git.apache.org/incubator-daffodil.git
$ git fetch upstream
4. Create a new branch off of the 'upstream/master' branch named
"daffodil-XYZ-description", where XYZ is the JIRA bug number and
"-description" is an optional very short description of the bug making
it easier to differentiate between multiple branches. For example:
$ git checkout -b daffodil-123-bitorder-feature upstream/master
5. Make changes to the branch, frequently adding new commits. Code
changes should follow the Daffodil style guidelines and should add
appropriate tests using the TDML format or unit tests. Tests in
src/test/scala-debug that are fixed should be moved into src/test/scala.
6. When changes are complete, rebase your commits onto upstream/master
and verify that all tests pass.
$ git fetch upstream
$ git rebase upstream/master
$ sbt test
Note that you should not use git pull or git merge to sync to the
upstream repo. Always fetch/rebase and avoid merge commits. Merge
commits in pull requests will be rejected.
7. If multiple commits were made, they should be rebased interactively
into the smallest number of logic commits that makes sense. Most
commonly this should be a single commit, but there may be some rare
cases where multiple commits make sense. Ensure the commit has an
appropriate and descriptive commit message. For traceability, the last
line must contain the JIRA bug, e.g. "DAFFODIL-123". Multiple bugs can
be separated by a comma.
8. Push your branch to your fork:
$ git push origin daffodil-123-bitorder-feature
9. Use the github interface to create a pull request for your new branch
10. Mark the JIRA bug as "Patch Available".
11. Wait for review comments. There must be at least two +1's from other
committers before the patch can be merged. If there are any comments or
the automated Travis CI build fails, repeat from step 5 making the
necessary changes, rebase, and push to your fork. The pull request will
be automatically updated once you push your changes.
12. Once at least two +1's are received from committers, a committer can
accept the pull request. This should be done using the "Rebase and
Merge" option to avoid merge commits.
13. Mark the JIRA bug as Resolved.
14. If you would like to clean up, you can now delete your feature
branch, either via the github user interface or
$ git push --delete origin daffodil-123-bitorder-feature
$ git branch -D daffodil-123-bitorder-feature
==== Branch/Release Workflow ====
For those that are familiar with our old branch workflow, you will
notice that the above uses the "master" branch as the main development
branch, rather than a version branch (e.g. 2.1.0). I propose that all
main development be done on the master branch. First, this is standard
practice, so those joining the project will immediately understand where
development is occurring. Second, this clearly differentiates between a
development branch and a long term support branch. Third, in the past,
our build system used git and tags/branches to determine the
version of Daffodil (e.g. if we were on the 2.1.0 branch, the version
was calculated as 2.1.0-SNAPSHOT). This actually made for a fairly
complex build system that was fragile and required that one builds from
a git repo. Also, when we create source releases, the git information is
not included, preventing one from building from source. So in addition
to switching to this 'master' development branch, I also propose we
switch to hardcoded versions in the sbt build configuration that are
manually modified. Below is a proposed branch/release workflow.
1. Perform all development on the 'master' branch as defined in the Code
Contributor Workflow. The sbt configuration should indicate this is a
snapshot:
version := "2.1.0-incubating-SNAPSHOT"
2. Before an official release is done, a pre-release must be created.
Create a commit changing the sbt configuration from a SNAPSHOT to a
release version:
version := "2.1.0-incubating"
3. Create a signed, annotated tag for the commit:
$ git tag -sa v2.1.0-incubating-rc1
4. If issues are found with the pre-release, create a commit to make it
a SNAPSHOT again:
version := "2.1.0-incubating-SNAPSHOT"
and add changes as necessary. Repeat from step 2, bumping the tagged rc
version, until the release candidate is accepted.
5. When the release candidate is accepted, create a new release tag in
the "rel" namespace, based on the accepted rc:
$ git tag -sa rel/v2.1.0-incubating v2.1.0-incubating-rc1
6. Create a support branch to support the minor version in the "support"
namespace e.g.:
$ git branch support/v2.1.x-incubating
If at any point patches need to be made to this support branch, a commit
should be created to bump the patch version and mark it is a SNAPSHOT:
version := "2.1.1-incubating-SNAPSHOT"
The same process is followed from Step 1 to release this branch. No new
branch needs to be created. Future patches will occur on the v2.1.x
branch. Some patches may need to be ported to other support branches or
the master branch.
7. Back on the master branch, a commit should be made to bump the minor
version and make it a SNAPSHOT:
version := "2.2.0-incubating-SNAPSHOT"
Repeat from step 1, with all development occurring on the master branch.
I've attached a graph of what this workflow might look like. It shows
three releases, 1.0.0, 1.0.1 (patch release), and a 1.1.0 release.
The 1.0.0 release requires two release candidates, whereas the other
releases only required one release candidate. Support branches are
created for 1.0.x and 1.1.x after the release.
[1]
https://lists.apache.org/thread.html/b41ace52a7be13dba32ee6c419e17150201fa994c830b5eed59c1e3a@%3Cdev.daffodil.apache.org%3E
Re: Code Contributor and Branch Workflow Proposals
Posted by Steve Lawrence <sl...@apache.org>.
I've created new pages on the wiki for the Code Contributor and
Branch/Tag Workflows, that should have all comments incorporated:
https://cwiki.apache.org/confluence/display/DAFFODIL/Code+Contributor+Workflow
https://cwiki.apache.org/confluence/display/DAFFODIL/Branch+and+Tag+Workflow
I've also created two main categories for wiki pages: "For Contributors"
for information for people that want to contribute to Daffodil (this is
similar to our old 'For Developers' and "For Users", for people that
just want to use the CLI or API.
On 10/17/2017 05:41 PM, Steve Lawrence wrote:
> That's a good question.
>
> I would say comments that belong in JIRA should probably be limited to
> discussions about the bug/feature (e.g. how to recreate a bug,
> investigations about the cause of a bug, how a bug fix/feature should be
> implemented, clarifications about a feature). I believe the Apache git
> integration handles creating information about pull requests and commits
> in the Work Log, so that does not need to be done manually. Also, some
> of these discussions may make sense to take place on the mailing list as
> well, depending on the complexity of the bug/feature.
>
> I would say information directly related to a change belongs in the
> actual commit message. Specifically, this should describe what changes
> were made, and more importantly, why those changes were made. Often
> times the 'what' is clear, but the 'why' is not as obvious. The git log
> is a very convenient place for this information. Since the git
> integration automatically references the commit hash, I do not think the
> commit message needs to be duplicated as a JIRA comment. Note, a really
> good source for explaining how to write a good commit message is here:
>
> https://chris.beams.io/posts/git-commit/
>
> Perhaps we should reference that link directly from the workflow.
>
> On 10/17/2017 01:06 PM, Mike Beckerle wrote:
>> Another thing to describe is what documentation of the patch goes in the jira ticket and what goes in the commit message, and what goes in both.
>>
>>
>> Otherwise I know we'll have the same information in both places via cut/paste because people won't be sure what goes where.
>>
>> ________________________________
>> From: Steve Lawrence <sl...@apache.org>
>> Sent: Tuesday, October 17, 2017 12:58:43 PM
>> To: dev@daffodil.apache.org; Mike Beckerle
>> Subject: Re: Code Contributor and Branch Workflow Proposals
>>
>> Yes, an existing ticket can be used. I'll make sure this is clarified.
>>
>> Regarding more than one, I think ideally there is a one-to-one
>> relationship between JIRA tickets and commits/pull requests, just to
>> make tracability easier. However, I'm sure there will be cases where
>> this isn't ideal (either multiple pull requests will be needed to
>> resolve a single JIRA ticket, or one pull request will resolve multiple
>> JIRA tickets) so I don't think we want to make it a hard rule that it
>> must be one-to-one. I think we can be flexible with this.
>>
>> Though, I think we can be pretty rigid about always needing to have at
>> least one JIRA ticket for each pull request.
>>
>> On 10/17/2017 12:45 PM, Mike Beckerle wrote:
>>> This looks great.
>>>
>>>
>>> Clarification: Is a new jira ticket required? Can one just use an existing jira ticket? (or more than one?)
>>>
>>> ________________________________
>>> From: Steve Lawrence <sl...@apache.org>
>>> Sent: Tuesday, October 17, 2017 11:34:28 AM
>>> To: dev@daffodil.apache.org
>>> Subject: Code Contributor and Branch Workflow Proposals
>>>
>>> I don't think there were any major objections on the infrastructure
>>> discussion [1], so I'd like to propose a contributor and branch workflow
>>> using that infrastructure.
>>>
>>> == Code Contributor Workflow ==
>>>
>>> This is the workflow used by all people that wish to contribute code to
>>> Daffodil, regardless of if they are a committer or not. There are other
>>> types of contributors (e.g. wiki, mailing list support, testing, etc.)
>>> that this does not cover.
>>>
>>> 1. Create a bug in Daffodil JIRA, and assign it to yourself.
>>>
>>> 2. Create a github fork of the Daffodil github mirror and clone it.
>>> This remote will be your 'origin' remote:
>>>
>>> $ git clone git@github.com:username/daffodil.git
>>>
>>> 3. Add the upstream repository as a new remote, calling it 'upstream':
>>>
>>> $ git remote add upstream git://git.apache.org/incubator-daffodil.git
>>> $ git fetch upstream
>>>
>>> 4. Create a new branch off of the 'upstream/master' branch named
>>> "daffodil-XYZ-description", where XYZ is the JIRA bug number and
>>> "-description" is an optional very short description of the bug making
>>> it easier to differentiate between multiple branches. For example:
>>>
>>> $ git checkout -b daffodil-123-bitorder-feature upstream/master
>>>
>>> 5. Make changes to the branch, frequently adding new commits. Code
>>> changes should follow the Daffodil style guidelines and should add
>>> appropriate tests using the TDML format or unit tests. Tests in
>>> src/test/scala-debug that are fixed should be moved into src/test/scala.
>>>
>>> 6. When changes are complete, rebase your commits onto upstream/master
>>> and verify that all tests pass.
>>>
>>> $ git fetch upstream
>>> $ git rebase upstream/master
>>> $ sbt test
>>>
>>> Note that you should not use git pull or git merge to sync to the
>>> upstream repo. Always fetch/rebase and avoid merge commits. Merge
>>> commits in pull requests will be rejected.
>>>
>>> 7. If multiple commits were made, they should be rebased interactively
>>> into the smallest number of logic commits that makes sense. Most
>>> commonly this should be a single commit, but there may be some rare
>>> cases where multiple commits make sense. Ensure the commit has an
>>> appropriate and descriptive commit message. For traceability, the last
>>> line must contain the JIRA bug, e.g. "DAFFODIL-123". Multiple bugs can
>>> be separated by a comma.
>>>
>>> 8. Push your branch to your fork:
>>>
>>> $ git push origin daffodil-123-bitorder-feature
>>>
>>> 9. Use the github interface to create a pull request for your new branch
>>>
>>> 10. Mark the JIRA bug as "Patch Available".
>>>
>>> 11. Wait for review comments. There must be at least two +1's from other
>>> committers before the patch can be merged. If there are any comments or
>>> the automated Travis CI build fails, repeat from step 5 making the
>>> necessary changes, rebase, and push to your fork. The pull request will
>>> be automatically updated once you push your changes.
>>>
>>> 12. Once at least two +1's are received from committers, a committer can
>>> accept the pull request. This should be done using the "Rebase and
>>> Merge" option to avoid merge commits.
>>>
>>> 13. Mark the JIRA bug as Resolved.
>>>
>>> 14. If you would like to clean up, you can now delete your feature
>>> branch, either via the github user interface or
>>>
>>> $ git push --delete origin daffodil-123-bitorder-feature
>>> $ git branch -D daffodil-123-bitorder-feature
>>>
>>>
>>>
>>> ==== Branch/Release Workflow ====
>>>
>>> For those that are familiar with our old branch workflow, you will
>>> notice that the above uses the "master" branch as the main development
>>> branch, rather than a version branch (e.g. 2.1.0). I propose that all
>>> main development be done on the master branch. First, this is standard
>>> practice, so those joining the project will immediately understand where
>>> development is occurring. Second, this clearly differentiates between a
>>> development branch and a long term support branch. Third, in the past,
>>> our build system used git and tags/branches to determine the
>>> version of Daffodil (e.g. if we were on the 2.1.0 branch, the version
>>> was calculated as 2.1.0-SNAPSHOT). This actually made for a fairly
>>> complex build system that was fragile and required that one builds from
>>> a git repo. Also, when we create source releases, the git information is
>>> not included, preventing one from building from source. So in addition
>>> to switching to this 'master' development branch, I also propose we
>>> switch to hardcoded versions in the sbt build configuration that are
>>> manually modified. Below is a proposed branch/release workflow.
>>>
>>> 1. Perform all development on the 'master' branch as defined in the Code
>>> Contributor Workflow. The sbt configuration should indicate this is a
>>> snapshot:
>>>
>>> version := "2.1.0-incubating-SNAPSHOT"
>>>
>>> 2. Before an official release is done, a pre-release must be created.
>>> Create a commit changing the sbt configuration from a SNAPSHOT to a
>>> release version:
>>>
>>> version := "2.1.0-incubating"
>>>
>>> 3. Create a signed, annotated tag for the commit:
>>>
>>> $ git tag -sa v2.1.0-incubating-rc1
>>>
>>> 4. If issues are found with the pre-release, create a commit to make it
>>> a SNAPSHOT again:
>>>
>>> version := "2.1.0-incubating-SNAPSHOT"
>>>
>>> and add changes as necessary. Repeat from step 2, bumping the tagged rc
>>> version, until the release candidate is accepted.
>>>
>>> 5. When the release candidate is accepted, create a new release tag in
>>> the "rel" namespace, based on the accepted rc:
>>>
>>> $ git tag -sa rel/v2.1.0-incubating v2.1.0-incubating-rc1
>>>
>>> 6. Create a support branch to support the minor version in the "support"
>>> namespace e.g.:
>>>
>>> $ git branch support/v2.1.x-incubating
>>>
>>> If at any point patches need to be made to this support branch, a commit
>>> should be created to bump the patch version and mark it is a SNAPSHOT:
>>>
>>> version := "2.1.1-incubating-SNAPSHOT"
>>>
>>> The same process is followed from Step 1 to release this branch. No new
>>> branch needs to be created. Future patches will occur on the v2.1.x
>>> branch. Some patches may need to be ported to other support branches or
>>> the master branch.
>>>
>>> 7. Back on the master branch, a commit should be made to bump the minor
>>> version and make it a SNAPSHOT:
>>>
>>> version := "2.2.0-incubating-SNAPSHOT"
>>>
>>> Repeat from step 1, with all development occurring on the master branch.
>>>
>>> I've attached a graph of what this workflow might look like. It shows
>>> three releases, 1.0.0, 1.0.1 (patch release), and a 1.1.0 release.
>>> The 1.0.0 release requires two release candidates, whereas the other
>>> releases only required one release candidate. Support branches are
>>> created for 1.0.x and 1.1.x after the release.
>>>
>>>
>>> [1]
>>> https://lists.apache.org/thread.html/b41ace52a7be13dba32ee6c419e17150201fa994c830b5eed59c1e3a@%3Cdev.daffodil.apache.org%3E
>>>
>>>
>>
>>
>
Re: Code Contributor and Branch Workflow Proposals
Posted by Steve Lawrence <sl...@apache.org>.
That's a good question.
I would say comments that belong in JIRA should probably be limited to
discussions about the bug/feature (e.g. how to recreate a bug,
investigations about the cause of a bug, how a bug fix/feature should be
implemented, clarifications about a feature). I believe the Apache git
integration handles creating information about pull requests and commits
in the Work Log, so that does not need to be done manually. Also, some
of these discussions may make sense to take place on the mailing list as
well, depending on the complexity of the bug/feature.
I would say information directly related to a change belongs in the
actual commit message. Specifically, this should describe what changes
were made, and more importantly, why those changes were made. Often
times the 'what' is clear, but the 'why' is not as obvious. The git log
is a very convenient place for this information. Since the git
integration automatically references the commit hash, I do not think the
commit message needs to be duplicated as a JIRA comment. Note, a really
good source for explaining how to write a good commit message is here:
https://chris.beams.io/posts/git-commit/
Perhaps we should reference that link directly from the workflow.
On 10/17/2017 01:06 PM, Mike Beckerle wrote:
> Another thing to describe is what documentation of the patch goes in the jira ticket and what goes in the commit message, and what goes in both.
>
>
> Otherwise I know we'll have the same information in both places via cut/paste because people won't be sure what goes where.
>
> ________________________________
> From: Steve Lawrence <sl...@apache.org>
> Sent: Tuesday, October 17, 2017 12:58:43 PM
> To: dev@daffodil.apache.org; Mike Beckerle
> Subject: Re: Code Contributor and Branch Workflow Proposals
>
> Yes, an existing ticket can be used. I'll make sure this is clarified.
>
> Regarding more than one, I think ideally there is a one-to-one
> relationship between JIRA tickets and commits/pull requests, just to
> make tracability easier. However, I'm sure there will be cases where
> this isn't ideal (either multiple pull requests will be needed to
> resolve a single JIRA ticket, or one pull request will resolve multiple
> JIRA tickets) so I don't think we want to make it a hard rule that it
> must be one-to-one. I think we can be flexible with this.
>
> Though, I think we can be pretty rigid about always needing to have at
> least one JIRA ticket for each pull request.
>
> On 10/17/2017 12:45 PM, Mike Beckerle wrote:
>> This looks great.
>>
>>
>> Clarification: Is a new jira ticket required? Can one just use an existing jira ticket? (or more than one?)
>>
>> ________________________________
>> From: Steve Lawrence <sl...@apache.org>
>> Sent: Tuesday, October 17, 2017 11:34:28 AM
>> To: dev@daffodil.apache.org
>> Subject: Code Contributor and Branch Workflow Proposals
>>
>> I don't think there were any major objections on the infrastructure
>> discussion [1], so I'd like to propose a contributor and branch workflow
>> using that infrastructure.
>>
>> == Code Contributor Workflow ==
>>
>> This is the workflow used by all people that wish to contribute code to
>> Daffodil, regardless of if they are a committer or not. There are other
>> types of contributors (e.g. wiki, mailing list support, testing, etc.)
>> that this does not cover.
>>
>> 1. Create a bug in Daffodil JIRA, and assign it to yourself.
>>
>> 2. Create a github fork of the Daffodil github mirror and clone it.
>> This remote will be your 'origin' remote:
>>
>> $ git clone git@github.com:username/daffodil.git
>>
>> 3. Add the upstream repository as a new remote, calling it 'upstream':
>>
>> $ git remote add upstream git://git.apache.org/incubator-daffodil.git
>> $ git fetch upstream
>>
>> 4. Create a new branch off of the 'upstream/master' branch named
>> "daffodil-XYZ-description", where XYZ is the JIRA bug number and
>> "-description" is an optional very short description of the bug making
>> it easier to differentiate between multiple branches. For example:
>>
>> $ git checkout -b daffodil-123-bitorder-feature upstream/master
>>
>> 5. Make changes to the branch, frequently adding new commits. Code
>> changes should follow the Daffodil style guidelines and should add
>> appropriate tests using the TDML format or unit tests. Tests in
>> src/test/scala-debug that are fixed should be moved into src/test/scala.
>>
>> 6. When changes are complete, rebase your commits onto upstream/master
>> and verify that all tests pass.
>>
>> $ git fetch upstream
>> $ git rebase upstream/master
>> $ sbt test
>>
>> Note that you should not use git pull or git merge to sync to the
>> upstream repo. Always fetch/rebase and avoid merge commits. Merge
>> commits in pull requests will be rejected.
>>
>> 7. If multiple commits were made, they should be rebased interactively
>> into the smallest number of logic commits that makes sense. Most
>> commonly this should be a single commit, but there may be some rare
>> cases where multiple commits make sense. Ensure the commit has an
>> appropriate and descriptive commit message. For traceability, the last
>> line must contain the JIRA bug, e.g. "DAFFODIL-123". Multiple bugs can
>> be separated by a comma.
>>
>> 8. Push your branch to your fork:
>>
>> $ git push origin daffodil-123-bitorder-feature
>>
>> 9. Use the github interface to create a pull request for your new branch
>>
>> 10. Mark the JIRA bug as "Patch Available".
>>
>> 11. Wait for review comments. There must be at least two +1's from other
>> committers before the patch can be merged. If there are any comments or
>> the automated Travis CI build fails, repeat from step 5 making the
>> necessary changes, rebase, and push to your fork. The pull request will
>> be automatically updated once you push your changes.
>>
>> 12. Once at least two +1's are received from committers, a committer can
>> accept the pull request. This should be done using the "Rebase and
>> Merge" option to avoid merge commits.
>>
>> 13. Mark the JIRA bug as Resolved.
>>
>> 14. If you would like to clean up, you can now delete your feature
>> branch, either via the github user interface or
>>
>> $ git push --delete origin daffodil-123-bitorder-feature
>> $ git branch -D daffodil-123-bitorder-feature
>>
>>
>>
>> ==== Branch/Release Workflow ====
>>
>> For those that are familiar with our old branch workflow, you will
>> notice that the above uses the "master" branch as the main development
>> branch, rather than a version branch (e.g. 2.1.0). I propose that all
>> main development be done on the master branch. First, this is standard
>> practice, so those joining the project will immediately understand where
>> development is occurring. Second, this clearly differentiates between a
>> development branch and a long term support branch. Third, in the past,
>> our build system used git and tags/branches to determine the
>> version of Daffodil (e.g. if we were on the 2.1.0 branch, the version
>> was calculated as 2.1.0-SNAPSHOT). This actually made for a fairly
>> complex build system that was fragile and required that one builds from
>> a git repo. Also, when we create source releases, the git information is
>> not included, preventing one from building from source. So in addition
>> to switching to this 'master' development branch, I also propose we
>> switch to hardcoded versions in the sbt build configuration that are
>> manually modified. Below is a proposed branch/release workflow.
>>
>> 1. Perform all development on the 'master' branch as defined in the Code
>> Contributor Workflow. The sbt configuration should indicate this is a
>> snapshot:
>>
>> version := "2.1.0-incubating-SNAPSHOT"
>>
>> 2. Before an official release is done, a pre-release must be created.
>> Create a commit changing the sbt configuration from a SNAPSHOT to a
>> release version:
>>
>> version := "2.1.0-incubating"
>>
>> 3. Create a signed, annotated tag for the commit:
>>
>> $ git tag -sa v2.1.0-incubating-rc1
>>
>> 4. If issues are found with the pre-release, create a commit to make it
>> a SNAPSHOT again:
>>
>> version := "2.1.0-incubating-SNAPSHOT"
>>
>> and add changes as necessary. Repeat from step 2, bumping the tagged rc
>> version, until the release candidate is accepted.
>>
>> 5. When the release candidate is accepted, create a new release tag in
>> the "rel" namespace, based on the accepted rc:
>>
>> $ git tag -sa rel/v2.1.0-incubating v2.1.0-incubating-rc1
>>
>> 6. Create a support branch to support the minor version in the "support"
>> namespace e.g.:
>>
>> $ git branch support/v2.1.x-incubating
>>
>> If at any point patches need to be made to this support branch, a commit
>> should be created to bump the patch version and mark it is a SNAPSHOT:
>>
>> version := "2.1.1-incubating-SNAPSHOT"
>>
>> The same process is followed from Step 1 to release this branch. No new
>> branch needs to be created. Future patches will occur on the v2.1.x
>> branch. Some patches may need to be ported to other support branches or
>> the master branch.
>>
>> 7. Back on the master branch, a commit should be made to bump the minor
>> version and make it a SNAPSHOT:
>>
>> version := "2.2.0-incubating-SNAPSHOT"
>>
>> Repeat from step 1, with all development occurring on the master branch.
>>
>> I've attached a graph of what this workflow might look like. It shows
>> three releases, 1.0.0, 1.0.1 (patch release), and a 1.1.0 release.
>> The 1.0.0 release requires two release candidates, whereas the other
>> releases only required one release candidate. Support branches are
>> created for 1.0.x and 1.1.x after the release.
>>
>>
>> [1]
>> https://lists.apache.org/thread.html/b41ace52a7be13dba32ee6c419e17150201fa994c830b5eed59c1e3a@%3Cdev.daffodil.apache.org%3E
>>
>>
>
>
Re: Code Contributor and Branch Workflow Proposals
Posted by Mike Beckerle <mb...@tresys.com>.
Another thing to describe is what documentation of the patch goes in the jira ticket and what goes in the commit message, and what goes in both.
Otherwise I know we'll have the same information in both places via cut/paste because people won't be sure what goes where.
________________________________
From: Steve Lawrence <sl...@apache.org>
Sent: Tuesday, October 17, 2017 12:58:43 PM
To: dev@daffodil.apache.org; Mike Beckerle
Subject: Re: Code Contributor and Branch Workflow Proposals
Yes, an existing ticket can be used. I'll make sure this is clarified.
Regarding more than one, I think ideally there is a one-to-one
relationship between JIRA tickets and commits/pull requests, just to
make tracability easier. However, I'm sure there will be cases where
this isn't ideal (either multiple pull requests will be needed to
resolve a single JIRA ticket, or one pull request will resolve multiple
JIRA tickets) so I don't think we want to make it a hard rule that it
must be one-to-one. I think we can be flexible with this.
Though, I think we can be pretty rigid about always needing to have at
least one JIRA ticket for each pull request.
On 10/17/2017 12:45 PM, Mike Beckerle wrote:
> This looks great.
>
>
> Clarification: Is a new jira ticket required? Can one just use an existing jira ticket? (or more than one?)
>
> ________________________________
> From: Steve Lawrence <sl...@apache.org>
> Sent: Tuesday, October 17, 2017 11:34:28 AM
> To: dev@daffodil.apache.org
> Subject: Code Contributor and Branch Workflow Proposals
>
> I don't think there were any major objections on the infrastructure
> discussion [1], so I'd like to propose a contributor and branch workflow
> using that infrastructure.
>
> == Code Contributor Workflow ==
>
> This is the workflow used by all people that wish to contribute code to
> Daffodil, regardless of if they are a committer or not. There are other
> types of contributors (e.g. wiki, mailing list support, testing, etc.)
> that this does not cover.
>
> 1. Create a bug in Daffodil JIRA, and assign it to yourself.
>
> 2. Create a github fork of the Daffodil github mirror and clone it.
> This remote will be your 'origin' remote:
>
> $ git clone git@github.com:username/daffodil.git
>
> 3. Add the upstream repository as a new remote, calling it 'upstream':
>
> $ git remote add upstream git://git.apache.org/incubator-daffodil.git
> $ git fetch upstream
>
> 4. Create a new branch off of the 'upstream/master' branch named
> "daffodil-XYZ-description", where XYZ is the JIRA bug number and
> "-description" is an optional very short description of the bug making
> it easier to differentiate between multiple branches. For example:
>
> $ git checkout -b daffodil-123-bitorder-feature upstream/master
>
> 5. Make changes to the branch, frequently adding new commits. Code
> changes should follow the Daffodil style guidelines and should add
> appropriate tests using the TDML format or unit tests. Tests in
> src/test/scala-debug that are fixed should be moved into src/test/scala.
>
> 6. When changes are complete, rebase your commits onto upstream/master
> and verify that all tests pass.
>
> $ git fetch upstream
> $ git rebase upstream/master
> $ sbt test
>
> Note that you should not use git pull or git merge to sync to the
> upstream repo. Always fetch/rebase and avoid merge commits. Merge
> commits in pull requests will be rejected.
>
> 7. If multiple commits were made, they should be rebased interactively
> into the smallest number of logic commits that makes sense. Most
> commonly this should be a single commit, but there may be some rare
> cases where multiple commits make sense. Ensure the commit has an
> appropriate and descriptive commit message. For traceability, the last
> line must contain the JIRA bug, e.g. "DAFFODIL-123". Multiple bugs can
> be separated by a comma.
>
> 8. Push your branch to your fork:
>
> $ git push origin daffodil-123-bitorder-feature
>
> 9. Use the github interface to create a pull request for your new branch
>
> 10. Mark the JIRA bug as "Patch Available".
>
> 11. Wait for review comments. There must be at least two +1's from other
> committers before the patch can be merged. If there are any comments or
> the automated Travis CI build fails, repeat from step 5 making the
> necessary changes, rebase, and push to your fork. The pull request will
> be automatically updated once you push your changes.
>
> 12. Once at least two +1's are received from committers, a committer can
> accept the pull request. This should be done using the "Rebase and
> Merge" option to avoid merge commits.
>
> 13. Mark the JIRA bug as Resolved.
>
> 14. If you would like to clean up, you can now delete your feature
> branch, either via the github user interface or
>
> $ git push --delete origin daffodil-123-bitorder-feature
> $ git branch -D daffodil-123-bitorder-feature
>
>
>
> ==== Branch/Release Workflow ====
>
> For those that are familiar with our old branch workflow, you will
> notice that the above uses the "master" branch as the main development
> branch, rather than a version branch (e.g. 2.1.0). I propose that all
> main development be done on the master branch. First, this is standard
> practice, so those joining the project will immediately understand where
> development is occurring. Second, this clearly differentiates between a
> development branch and a long term support branch. Third, in the past,
> our build system used git and tags/branches to determine the
> version of Daffodil (e.g. if we were on the 2.1.0 branch, the version
> was calculated as 2.1.0-SNAPSHOT). This actually made for a fairly
> complex build system that was fragile and required that one builds from
> a git repo. Also, when we create source releases, the git information is
> not included, preventing one from building from source. So in addition
> to switching to this 'master' development branch, I also propose we
> switch to hardcoded versions in the sbt build configuration that are
> manually modified. Below is a proposed branch/release workflow.
>
> 1. Perform all development on the 'master' branch as defined in the Code
> Contributor Workflow. The sbt configuration should indicate this is a
> snapshot:
>
> version := "2.1.0-incubating-SNAPSHOT"
>
> 2. Before an official release is done, a pre-release must be created.
> Create a commit changing the sbt configuration from a SNAPSHOT to a
> release version:
>
> version := "2.1.0-incubating"
>
> 3. Create a signed, annotated tag for the commit:
>
> $ git tag -sa v2.1.0-incubating-rc1
>
> 4. If issues are found with the pre-release, create a commit to make it
> a SNAPSHOT again:
>
> version := "2.1.0-incubating-SNAPSHOT"
>
> and add changes as necessary. Repeat from step 2, bumping the tagged rc
> version, until the release candidate is accepted.
>
> 5. When the release candidate is accepted, create a new release tag in
> the "rel" namespace, based on the accepted rc:
>
> $ git tag -sa rel/v2.1.0-incubating v2.1.0-incubating-rc1
>
> 6. Create a support branch to support the minor version in the "support"
> namespace e.g.:
>
> $ git branch support/v2.1.x-incubating
>
> If at any point patches need to be made to this support branch, a commit
> should be created to bump the patch version and mark it is a SNAPSHOT:
>
> version := "2.1.1-incubating-SNAPSHOT"
>
> The same process is followed from Step 1 to release this branch. No new
> branch needs to be created. Future patches will occur on the v2.1.x
> branch. Some patches may need to be ported to other support branches or
> the master branch.
>
> 7. Back on the master branch, a commit should be made to bump the minor
> version and make it a SNAPSHOT:
>
> version := "2.2.0-incubating-SNAPSHOT"
>
> Repeat from step 1, with all development occurring on the master branch.
>
> I've attached a graph of what this workflow might look like. It shows
> three releases, 1.0.0, 1.0.1 (patch release), and a 1.1.0 release.
> The 1.0.0 release requires two release candidates, whereas the other
> releases only required one release candidate. Support branches are
> created for 1.0.x and 1.1.x after the release.
>
>
> [1]
> https://lists.apache.org/thread.html/b41ace52a7be13dba32ee6c419e17150201fa994c830b5eed59c1e3a@%3Cdev.daffodil.apache.org%3E
>
>
Re: Code Contributor and Branch Workflow Proposals
Posted by Steve Lawrence <sl...@apache.org>.
Yes, an existing ticket can be used. I'll make sure this is clarified.
Regarding more than one, I think ideally there is a one-to-one
relationship between JIRA tickets and commits/pull requests, just to
make tracability easier. However, I'm sure there will be cases where
this isn't ideal (either multiple pull requests will be needed to
resolve a single JIRA ticket, or one pull request will resolve multiple
JIRA tickets) so I don't think we want to make it a hard rule that it
must be one-to-one. I think we can be flexible with this.
Though, I think we can be pretty rigid about always needing to have at
least one JIRA ticket for each pull request.
On 10/17/2017 12:45 PM, Mike Beckerle wrote:
> This looks great.
>
>
> Clarification: Is a new jira ticket required? Can one just use an existing jira ticket? (or more than one?)
>
> ________________________________
> From: Steve Lawrence <sl...@apache.org>
> Sent: Tuesday, October 17, 2017 11:34:28 AM
> To: dev@daffodil.apache.org
> Subject: Code Contributor and Branch Workflow Proposals
>
> I don't think there were any major objections on the infrastructure
> discussion [1], so I'd like to propose a contributor and branch workflow
> using that infrastructure.
>
> == Code Contributor Workflow ==
>
> This is the workflow used by all people that wish to contribute code to
> Daffodil, regardless of if they are a committer or not. There are other
> types of contributors (e.g. wiki, mailing list support, testing, etc.)
> that this does not cover.
>
> 1. Create a bug in Daffodil JIRA, and assign it to yourself.
>
> 2. Create a github fork of the Daffodil github mirror and clone it.
> This remote will be your 'origin' remote:
>
> $ git clone git@github.com:username/daffodil.git
>
> 3. Add the upstream repository as a new remote, calling it 'upstream':
>
> $ git remote add upstream git://git.apache.org/incubator-daffodil.git
> $ git fetch upstream
>
> 4. Create a new branch off of the 'upstream/master' branch named
> "daffodil-XYZ-description", where XYZ is the JIRA bug number and
> "-description" is an optional very short description of the bug making
> it easier to differentiate between multiple branches. For example:
>
> $ git checkout -b daffodil-123-bitorder-feature upstream/master
>
> 5. Make changes to the branch, frequently adding new commits. Code
> changes should follow the Daffodil style guidelines and should add
> appropriate tests using the TDML format or unit tests. Tests in
> src/test/scala-debug that are fixed should be moved into src/test/scala.
>
> 6. When changes are complete, rebase your commits onto upstream/master
> and verify that all tests pass.
>
> $ git fetch upstream
> $ git rebase upstream/master
> $ sbt test
>
> Note that you should not use git pull or git merge to sync to the
> upstream repo. Always fetch/rebase and avoid merge commits. Merge
> commits in pull requests will be rejected.
>
> 7. If multiple commits were made, they should be rebased interactively
> into the smallest number of logic commits that makes sense. Most
> commonly this should be a single commit, but there may be some rare
> cases where multiple commits make sense. Ensure the commit has an
> appropriate and descriptive commit message. For traceability, the last
> line must contain the JIRA bug, e.g. "DAFFODIL-123". Multiple bugs can
> be separated by a comma.
>
> 8. Push your branch to your fork:
>
> $ git push origin daffodil-123-bitorder-feature
>
> 9. Use the github interface to create a pull request for your new branch
>
> 10. Mark the JIRA bug as "Patch Available".
>
> 11. Wait for review comments. There must be at least two +1's from other
> committers before the patch can be merged. If there are any comments or
> the automated Travis CI build fails, repeat from step 5 making the
> necessary changes, rebase, and push to your fork. The pull request will
> be automatically updated once you push your changes.
>
> 12. Once at least two +1's are received from committers, a committer can
> accept the pull request. This should be done using the "Rebase and
> Merge" option to avoid merge commits.
>
> 13. Mark the JIRA bug as Resolved.
>
> 14. If you would like to clean up, you can now delete your feature
> branch, either via the github user interface or
>
> $ git push --delete origin daffodil-123-bitorder-feature
> $ git branch -D daffodil-123-bitorder-feature
>
>
>
> ==== Branch/Release Workflow ====
>
> For those that are familiar with our old branch workflow, you will
> notice that the above uses the "master" branch as the main development
> branch, rather than a version branch (e.g. 2.1.0). I propose that all
> main development be done on the master branch. First, this is standard
> practice, so those joining the project will immediately understand where
> development is occurring. Second, this clearly differentiates between a
> development branch and a long term support branch. Third, in the past,
> our build system used git and tags/branches to determine the
> version of Daffodil (e.g. if we were on the 2.1.0 branch, the version
> was calculated as 2.1.0-SNAPSHOT). This actually made for a fairly
> complex build system that was fragile and required that one builds from
> a git repo. Also, when we create source releases, the git information is
> not included, preventing one from building from source. So in addition
> to switching to this 'master' development branch, I also propose we
> switch to hardcoded versions in the sbt build configuration that are
> manually modified. Below is a proposed branch/release workflow.
>
> 1. Perform all development on the 'master' branch as defined in the Code
> Contributor Workflow. The sbt configuration should indicate this is a
> snapshot:
>
> version := "2.1.0-incubating-SNAPSHOT"
>
> 2. Before an official release is done, a pre-release must be created.
> Create a commit changing the sbt configuration from a SNAPSHOT to a
> release version:
>
> version := "2.1.0-incubating"
>
> 3. Create a signed, annotated tag for the commit:
>
> $ git tag -sa v2.1.0-incubating-rc1
>
> 4. If issues are found with the pre-release, create a commit to make it
> a SNAPSHOT again:
>
> version := "2.1.0-incubating-SNAPSHOT"
>
> and add changes as necessary. Repeat from step 2, bumping the tagged rc
> version, until the release candidate is accepted.
>
> 5. When the release candidate is accepted, create a new release tag in
> the "rel" namespace, based on the accepted rc:
>
> $ git tag -sa rel/v2.1.0-incubating v2.1.0-incubating-rc1
>
> 6. Create a support branch to support the minor version in the "support"
> namespace e.g.:
>
> $ git branch support/v2.1.x-incubating
>
> If at any point patches need to be made to this support branch, a commit
> should be created to bump the patch version and mark it is a SNAPSHOT:
>
> version := "2.1.1-incubating-SNAPSHOT"
>
> The same process is followed from Step 1 to release this branch. No new
> branch needs to be created. Future patches will occur on the v2.1.x
> branch. Some patches may need to be ported to other support branches or
> the master branch.
>
> 7. Back on the master branch, a commit should be made to bump the minor
> version and make it a SNAPSHOT:
>
> version := "2.2.0-incubating-SNAPSHOT"
>
> Repeat from step 1, with all development occurring on the master branch.
>
> I've attached a graph of what this workflow might look like. It shows
> three releases, 1.0.0, 1.0.1 (patch release), and a 1.1.0 release.
> The 1.0.0 release requires two release candidates, whereas the other
> releases only required one release candidate. Support branches are
> created for 1.0.x and 1.1.x after the release.
>
>
> [1]
> https://lists.apache.org/thread.html/b41ace52a7be13dba32ee6c419e17150201fa994c830b5eed59c1e3a@%3Cdev.daffodil.apache.org%3E
>
>
Re: Code Contributor and Branch Workflow Proposals
Posted by Mike Beckerle <mb...@tresys.com>.
This looks great.
Clarification: Is a new jira ticket required? Can one just use an existing jira ticket? (or more than one?)
________________________________
From: Steve Lawrence <sl...@apache.org>
Sent: Tuesday, October 17, 2017 11:34:28 AM
To: dev@daffodil.apache.org
Subject: Code Contributor and Branch Workflow Proposals
I don't think there were any major objections on the infrastructure
discussion [1], so I'd like to propose a contributor and branch workflow
using that infrastructure.
== Code Contributor Workflow ==
This is the workflow used by all people that wish to contribute code to
Daffodil, regardless of if they are a committer or not. There are other
types of contributors (e.g. wiki, mailing list support, testing, etc.)
that this does not cover.
1. Create a bug in Daffodil JIRA, and assign it to yourself.
2. Create a github fork of the Daffodil github mirror and clone it.
This remote will be your 'origin' remote:
$ git clone git@github.com:username/daffodil.git
3. Add the upstream repository as a new remote, calling it 'upstream':
$ git remote add upstream git://git.apache.org/incubator-daffodil.git
$ git fetch upstream
4. Create a new branch off of the 'upstream/master' branch named
"daffodil-XYZ-description", where XYZ is the JIRA bug number and
"-description" is an optional very short description of the bug making
it easier to differentiate between multiple branches. For example:
$ git checkout -b daffodil-123-bitorder-feature upstream/master
5. Make changes to the branch, frequently adding new commits. Code
changes should follow the Daffodil style guidelines and should add
appropriate tests using the TDML format or unit tests. Tests in
src/test/scala-debug that are fixed should be moved into src/test/scala.
6. When changes are complete, rebase your commits onto upstream/master
and verify that all tests pass.
$ git fetch upstream
$ git rebase upstream/master
$ sbt test
Note that you should not use git pull or git merge to sync to the
upstream repo. Always fetch/rebase and avoid merge commits. Merge
commits in pull requests will be rejected.
7. If multiple commits were made, they should be rebased interactively
into the smallest number of logic commits that makes sense. Most
commonly this should be a single commit, but there may be some rare
cases where multiple commits make sense. Ensure the commit has an
appropriate and descriptive commit message. For traceability, the last
line must contain the JIRA bug, e.g. "DAFFODIL-123". Multiple bugs can
be separated by a comma.
8. Push your branch to your fork:
$ git push origin daffodil-123-bitorder-feature
9. Use the github interface to create a pull request for your new branch
10. Mark the JIRA bug as "Patch Available".
11. Wait for review comments. There must be at least two +1's from other
committers before the patch can be merged. If there are any comments or
the automated Travis CI build fails, repeat from step 5 making the
necessary changes, rebase, and push to your fork. The pull request will
be automatically updated once you push your changes.
12. Once at least two +1's are received from committers, a committer can
accept the pull request. This should be done using the "Rebase and
Merge" option to avoid merge commits.
13. Mark the JIRA bug as Resolved.
14. If you would like to clean up, you can now delete your feature
branch, either via the github user interface or
$ git push --delete origin daffodil-123-bitorder-feature
$ git branch -D daffodil-123-bitorder-feature
==== Branch/Release Workflow ====
For those that are familiar with our old branch workflow, you will
notice that the above uses the "master" branch as the main development
branch, rather than a version branch (e.g. 2.1.0). I propose that all
main development be done on the master branch. First, this is standard
practice, so those joining the project will immediately understand where
development is occurring. Second, this clearly differentiates between a
development branch and a long term support branch. Third, in the past,
our build system used git and tags/branches to determine the
version of Daffodil (e.g. if we were on the 2.1.0 branch, the version
was calculated as 2.1.0-SNAPSHOT). This actually made for a fairly
complex build system that was fragile and required that one builds from
a git repo. Also, when we create source releases, the git information is
not included, preventing one from building from source. So in addition
to switching to this 'master' development branch, I also propose we
switch to hardcoded versions in the sbt build configuration that are
manually modified. Below is a proposed branch/release workflow.
1. Perform all development on the 'master' branch as defined in the Code
Contributor Workflow. The sbt configuration should indicate this is a
snapshot:
version := "2.1.0-incubating-SNAPSHOT"
2. Before an official release is done, a pre-release must be created.
Create a commit changing the sbt configuration from a SNAPSHOT to a
release version:
version := "2.1.0-incubating"
3. Create a signed, annotated tag for the commit:
$ git tag -sa v2.1.0-incubating-rc1
4. If issues are found with the pre-release, create a commit to make it
a SNAPSHOT again:
version := "2.1.0-incubating-SNAPSHOT"
and add changes as necessary. Repeat from step 2, bumping the tagged rc
version, until the release candidate is accepted.
5. When the release candidate is accepted, create a new release tag in
the "rel" namespace, based on the accepted rc:
$ git tag -sa rel/v2.1.0-incubating v2.1.0-incubating-rc1
6. Create a support branch to support the minor version in the "support"
namespace e.g.:
$ git branch support/v2.1.x-incubating
If at any point patches need to be made to this support branch, a commit
should be created to bump the patch version and mark it is a SNAPSHOT:
version := "2.1.1-incubating-SNAPSHOT"
The same process is followed from Step 1 to release this branch. No new
branch needs to be created. Future patches will occur on the v2.1.x
branch. Some patches may need to be ported to other support branches or
the master branch.
7. Back on the master branch, a commit should be made to bump the minor
version and make it a SNAPSHOT:
version := "2.2.0-incubating-SNAPSHOT"
Repeat from step 1, with all development occurring on the master branch.
I've attached a graph of what this workflow might look like. It shows
three releases, 1.0.0, 1.0.1 (patch release), and a 1.1.0 release.
The 1.0.0 release requires two release candidates, whereas the other
releases only required one release candidate. Support branches are
created for 1.0.x and 1.1.x after the release.
[1]
https://lists.apache.org/thread.html/b41ace52a7be13dba32ee6c419e17150201fa994c830b5eed59c1e3a@%3Cdev.daffodil.apache.org%3E