Re: Project proposal for ICFOSS

[Noah Slater <ns...@apache.org>]

On 30 July 2013 20:13, Yashin Mehaboobe <ya...@gmail.com> wrote:

>
> 1.User runs bootstrap
>

Yep. But remember: only people building CouchDB from a Git checkout will
ever need to run ./boostrap. When we distribute CouchDB from our website,
all of the files that ./boostrap generates are included for them.


> 2.Bootstrap fetches the git commit messages too and stores in an rst file
>

Sounds good.


> 3.The python script uses this rst file to generate documentation.
>

Yep! But note: the Python script that generates the RST files is called by
make. So we have two steps here:

./bootstrap run on a pristine source checkout, and only run by devs. It
talks to Git, and generates an RST file.

Make is configured to expect the RST file, as if it were a regular part of
the source. When you run "make", the regular rules for building the docs
should pick up the RST file and include it into the docs.

-- 
NS

Re: Project proposal for ICFOSS

[Noah Slater <ns...@apache.org>]

On 10 August 2013 10:34, Yashin Mehaboobe <ya...@gmail.com> wrote:

> First of all, so sorry for the late reply. Secondly, I've been selected to
> work on this project so thank you so much for letting me work on this :) .
> I'll have a rudimentary code up and running as soon as possible.
>
> Noah: So instead of integrating the Git and JIRA requests into bootstrap I
> will create a separate python script that will pull them into an RST file
> that can be improved upon if needed. That's what you meant right?
>

Sounds about right! Could even make it so that it lives in couchdb-admin.

https://git-wip-us.apache.org/repos/asf?p=couchdb-admin.git

Nice and self-contained.

When we just add a section about how to use it to the release procedure.

http://wiki.apache.org/couchdb/Release_Procedure

-- 
Noah Slater
https://twitter.com/nslater

Re: Project proposal for ICFOSS

[Noah Slater <ns...@apache.org>]

On 10 August 2013 16:12, Alexander Shorin <kx...@gmail.com> wrote:

>
> AFAIK, CVE issues are announces at dev@ and security@ (?) mailing
> lists. Also I'd put them at:
>
> https://github.com/apache/couchdb/tree/1781-reorganize-and-improve-docs/share/doc/src/cve


Your script can access mbox archives, if it would help.

See:

https://git-wip-us.apache.org/repos/asf?p=couchdb-admin.git;a=blob_plain;f=release/build_cve_list.sh;hb=HEAD
https://git-wip-us.apache.org/repos/asf?p=couchdb-admin.git;a=blob_plain;f=release/check_docs.sh;hb=HEAD

If you'd like some mbox files to play around with (as you don't have SSH
access to ASF infra) let me know.

-- 
Noah Slater
https://twitter.com/nslater

Re: Project proposal for ICFOSS

[Alexander Shorin <kx...@gmail.com>]

On Sat, Aug 10, 2013 at 1:34 PM, Yashin Mehaboobe <ya...@gmail.com> wrote:
>
> Alexander: I was thinking of starting to create the RST file from current
> release only. From now on, a unique file will be created for each branch
> release (with the commit hash/version as the name?).

Is this structure is ok for you?
https://github.com/apache/couchdb/tree/1781-reorganize-and-improve-docs/share/doc/src/whatsnew

It's per release branch however, but should be more easy to inject new
release changes.

> About the integration of upgrade notes and CVE warnings into the file,
> could you please suggest from where I can pull the data? Is it usually
> present in the Git repository or the JIRA?

Probably, there is no such source to pull from. CVE information
theoretically could be found at
http://cve.mitre.org/cgi-bin/cvename.cgi
but they still has no info about three latest issues: 2012-5641,
2012-5649 and 2012-5650.

AFAIK, CVE issues are announces at dev@ and security@ (?) mailing
lists. Also I'd put them at:
https://github.com/apache/couchdb/tree/1781-reorganize-and-improve-docs/share/doc/src/cve

However, I'm not sure how you plan to parse such data.

Upgrade noted still have to be written manually since only devs and
testers are knows what have changes and what actions are needed to
upgrade CouchDB to the newest release, so generated file with changes
have to be placed under version control - that's how I see the problem
and solution, but there may be others.

--
,,,^..^,,,

Re: Project proposal for ICFOSS

[Yashin Mehaboobe <ya...@gmail.com>]

First of all, so sorry for the late reply. Secondly, I've been selected to
work on this project so thank you so much for letting me work on this :) .
I'll have a rudimentary code up and running as soon as possible.

Noah: So instead of integrating the Git and JIRA requests into bootstrap I
will create a separate python script that will pull them into an RST file
that can be improved upon if needed. That's what you meant right?

Alexander: I was thinking of starting to create the RST file from current
release only. From now on, a unique file will be created for each branch
release (with the commit hash/version as the name?). About the integration
of upgrade notes and CVE warnings into the file, could you please suggest
from where I can pull the data? Is it usually present in the Git repository
or the JIRA?

On Sun, Aug 4, 2013 at 2:09 AM, Noah Slater <ns...@apache.org> wrote:

> Hmm.
>
> If we're planning to commit the file, then the generation should not be in
> ./boostrap at all.
>
> Instead, we should ship a utility script that spits out an RST file that
> serves as a *starting point* for editing, and then committing.
>
>
> On 3 August 2013 20:33, Alexander Shorin <kx...@gmail.com> wrote:
>
> > Hi Yashin,
> >
> > Do you plan to generate since RST file with all changes or only just
> > for current release?
> >
> > Why I'm asking...There is already change history for old releases that
> > is not possible to generate automatically: it was quite untrivial to
> > automate for 1.3.0 release for me since there was a lot of false
> > positive matches or no matches at all. Completely regenerate past
> > history also isn't good idea since it will be very heavy operation
> > with a lot of JIRA requests.
> >
> > So the idea: to generate changes only for current branch release. Get
> > the commit hash when previous release was tagged and scan commits till
> > HEAD for features, JIRA issues etc. The output will be the file for
> > specific branch.
> >
> > For example how it may looks:
> > http://kxepal.iriscouch.org/docs/1.3/whatsnew/index.html
> >
> > By the way. This generated change log have to be stored on disk and be
> > committed: how we're planning to write Upgrade notes, CVE warnings,
> > Breaking changes and Migration guides for new release? Or, if not, how
> > and from where the script will embed them into generated .rst file(s)?
> >
> > --
> > ,,,^..^,,,
> >
> >
> > On Sat, Aug 3, 2013 at 12:18 AM, Noah Slater <ns...@apache.org> wrote:
> > > On 30 July 2013 20:13, Yashin Mehaboobe <ya...@gmail.com> wrote:
> > >
> > >>
> > >> 1.User runs bootstrap
> > >>
> > >
> > > Yep. But remember: only people building CouchDB from a Git checkout
> will
> > > ever need to run ./boostrap. When we distribute CouchDB from our
> website,
> > > all of the files that ./boostrap generates are included for them.
> > >
> > >
> > >> 2.Bootstrap fetches the git commit messages too and stores in an rst
> > file
> > >>
> > >
> > > Sounds good.
> > >
> > >
> > >> 3.The python script uses this rst file to generate documentation.
> > >>
> > >
> > > Yep! But note: the Python script that generates the RST files is called
> > by
> > > make. So we have two steps here:
> > >
> > > ./bootstrap run on a pristine source checkout, and only run by devs. It
> > > talks to Git, and generates an RST file.
> > >
> > > Make is configured to expect the RST file, as if it were a regular part
> > of
> > > the source. When you run "make", the regular rules for building the
> docs
> > > should pick up the RST file and include it into the docs.
> > >
> > > --
> > > NS
> >
>
>
>
> --
> Noah Slater
> https://twitter.com/nslater
>



-- 
- Yashin Mehaboobe

Re: Project proposal for ICFOSS

[Noah Slater <ns...@apache.org>]

Hmm.

If we're planning to commit the file, then the generation should not be in
./boostrap at all.

Instead, we should ship a utility script that spits out an RST file that
serves as a *starting point* for editing, and then committing.


On 3 August 2013 20:33, Alexander Shorin <kx...@gmail.com> wrote:

> Hi Yashin,
>
> Do you plan to generate since RST file with all changes or only just
> for current release?
>
> Why I'm asking...There is already change history for old releases that
> is not possible to generate automatically: it was quite untrivial to
> automate for 1.3.0 release for me since there was a lot of false
> positive matches or no matches at all. Completely regenerate past
> history also isn't good idea since it will be very heavy operation
> with a lot of JIRA requests.
>
> So the idea: to generate changes only for current branch release. Get
> the commit hash when previous release was tagged and scan commits till
> HEAD for features, JIRA issues etc. The output will be the file for
> specific branch.
>
> For example how it may looks:
> http://kxepal.iriscouch.org/docs/1.3/whatsnew/index.html
>
> By the way. This generated change log have to be stored on disk and be
> committed: how we're planning to write Upgrade notes, CVE warnings,
> Breaking changes and Migration guides for new release? Or, if not, how
> and from where the script will embed them into generated .rst file(s)?
>
> --
> ,,,^..^,,,
>
>
> On Sat, Aug 3, 2013 at 12:18 AM, Noah Slater <ns...@apache.org> wrote:
> > On 30 July 2013 20:13, Yashin Mehaboobe <ya...@gmail.com> wrote:
> >
> >>
> >> 1.User runs bootstrap
> >>
> >
> > Yep. But remember: only people building CouchDB from a Git checkout will
> > ever need to run ./boostrap. When we distribute CouchDB from our website,
> > all of the files that ./boostrap generates are included for them.
> >
> >
> >> 2.Bootstrap fetches the git commit messages too and stores in an rst
> file
> >>
> >
> > Sounds good.
> >
> >
> >> 3.The python script uses this rst file to generate documentation.
> >>
> >
> > Yep! But note: the Python script that generates the RST files is called
> by
> > make. So we have two steps here:
> >
> > ./bootstrap run on a pristine source checkout, and only run by devs. It
> > talks to Git, and generates an RST file.
> >
> > Make is configured to expect the RST file, as if it were a regular part
> of
> > the source. When you run "make", the regular rules for building the docs
> > should pick up the RST file and include it into the docs.
> >
> > --
> > NS
>



-- 
Noah Slater
https://twitter.com/nslater

Re: Project proposal for ICFOSS

[Alexander Shorin <kx...@gmail.com>]

Hi Yashin,

Do you plan to generate since RST file with all changes or only just
for current release?

Why I'm asking...There is already change history for old releases that
is not possible to generate automatically: it was quite untrivial to
automate for 1.3.0 release for me since there was a lot of false
positive matches or no matches at all. Completely regenerate past
history also isn't good idea since it will be very heavy operation
with a lot of JIRA requests.

So the idea: to generate changes only for current branch release. Get
the commit hash when previous release was tagged and scan commits till
HEAD for features, JIRA issues etc. The output will be the file for
specific branch.

For example how it may looks:
http://kxepal.iriscouch.org/docs/1.3/whatsnew/index.html

By the way. This generated change log have to be stored on disk and be
committed: how we're planning to write Upgrade notes, CVE warnings,
Breaking changes and Migration guides for new release? Or, if not, how
and from where the script will embed them into generated .rst file(s)?

--
,,,^..^,,,


On Sat, Aug 3, 2013 at 12:18 AM, Noah Slater <ns...@apache.org> wrote:
> On 30 July 2013 20:13, Yashin Mehaboobe <ya...@gmail.com> wrote:
>
>>
>> 1.User runs bootstrap
>>
>
> Yep. But remember: only people building CouchDB from a Git checkout will
> ever need to run ./boostrap. When we distribute CouchDB from our website,
> all of the files that ./boostrap generates are included for them.
>
>
>> 2.Bootstrap fetches the git commit messages too and stores in an rst file
>>
>
> Sounds good.
>
>
>> 3.The python script uses this rst file to generate documentation.
>>
>
> Yep! But note: the Python script that generates the RST files is called by
> make. So we have two steps here:
>
> ./bootstrap run on a pristine source checkout, and only run by devs. It
> talks to Git, and generates an RST file.
>
> Make is configured to expect the RST file, as if it were a regular part of
> the source. When you run "make", the regular rules for building the docs
> should pick up the RST file and include it into the docs.
>
> --
> NS