[VOTE] Release 4.5.1, release candidate #0

[Enrico Olivelli <eo...@gmail.com>]

Hi everyone,
Please review and vote on the release candidate #0 for the version
4.5.1, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* Release notes [1]
* The official Apache source and binary distributions to be deployed
to dist.apache.org [2]
* All artifacts to be deployed to the Maven Central Repository [3]
* Source code tag "release-4.5.1" [4]

BookKeeper's KEYS file contains PGP keys we used to sign this
release:https://dist.apache.org/repos/dist/release/bookkeeper/KEYS

Please download these packages and review this release candidate:

- Review release notes
- Download the source package (verify md5, shasum, and asc) and follow the
instructions to build and run the bookkeeper service.
- Download the binary package (verify md5, shasum, and asc) and follow the
instructions to run the bookkeeper service.
- Review maven repo, release tag, licenses, and any other things you think
it is important to a release.

The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Enrico Olivelli

[1] https://github.com/apache/bookkeeper/pull/678
[2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.5.1-rc0/
[3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1015/org/apache/bookkeeper/
[4] https://github.com/apache/bookkeeper/tree/release-4.5.1

Re: rc naming conventions, was :Re: [VOTE] Release 4.5.1, release candidate #0

[Ivan Kelly <iv...@apache.org>]

>> However, the point still stands. The final release should have a
>> different sha1 than the candidates, otherwise people's maven caches
>> get messed up, leading to hidden problems down the line.
>>
>
> I think that people who tests an rc should be able to clean up their own
> local repo.
They'll only clean it if they notice something is amiss.

-Ivan

rc naming conventions, was :Re: [VOTE] Release 4.5.1, release candidate #0

[Enrico Olivelli <eo...@gmail.com>]

Il mer 8 nov 2017, 16:45 Ivan Kelly <iv...@apache.org> ha scritto:

> > This has been done deliberately. Like for 4.5.0.
> > I am ok if you PMC want to do it differently.
> > Or maybe I have misunderstood the guide.
> > You already stated this on slack. I apologize I went forward following
> the
> > guide
> Oh, I'm misremembering how we used to do it, another project I worked
> on had it different.
>
> However, the point still stands. The final release should have a
> different sha1 than the candidates, otherwise people's maven caches
> get messed up, leading to hidden problems down the line.
>

I think that people who tests an rc should be able to clean up their own
local repo. Rcs are not tested on CI or other automatic systems.
Anyway this is just my opinion, I am open to whatever convention.

Enrico

>
> >> I ran the tests twice, and both times the following failed for me.
> >> They're probably flakes, but they failed on 2/2 so they should be
> >> investigated. This is on a machine that sometimes runs master tests
> >> cleanly (we really need to sort out our flakes).
> >> - BookKeeperAdminTest.testDecommissionBookie
> >> -
> >>
> TestRackawareEnsemblePlacementPolicyUsingScript.testNewEnsembleWithEnoughRacks
> >> - BookKeeperDiskSpaceWeightedLedgerPlacementTest
> >>
> >
> > Ok let's dig into this.
> It would be good for others to confirm whether they see the same
> failures, even if only periodically.
>
> -Ivan
>
-- 


-- Enrico Olivelli

Re: [VOTE] Release 4.5.1, release candidate #0

[Sijie Guo <gu...@gmail.com>]

Only "TestRackawareEnsemblePlacementPolicyUsingScript" failed for me. But I
suspected it might be related to my environment. will look into it.

Failed tests:

TestRackawareEnsemblePlacementPolicyUsingScript.testNewEnsembleWithEnoughRacks:320
expected:<3> but was:<2>

- Sijie



On Wed, Nov 8, 2017 at 9:21 AM, Sijie Guo <gu...@gmail.com> wrote:

>
>
> On Wed, Nov 8, 2017 at 7:45 AM, Ivan Kelly <iv...@apache.org> wrote:
>
>> > This has been done deliberately. Like for 4.5.0.
>> > I am ok if you PMC want to do it differently.
>> > Or maybe I have misunderstood the guide.
>> > You already stated this on slack. I apologize I went forward following
>> the
>> > guide
>> Oh, I'm misremembering how we used to do it, another project I worked
>> on had it different.
>>
>> However, the point still stands. The final release should have a
>> different sha1 than the candidates, otherwise people's maven caches
>> get messed up, leading to hidden problems down the line.
>
>
> Are you talking about a tag with "-rc" or artifacts with "-rc"?
>
> I don't think we've ever used "rc" for the staging artifacts. if you take
> a look at the "staging repositories" at nexus,
> I don't see any projects using "rc" for the staging artifacts.
>
> The purpose of a staging area is once a RC is approved, you can just click
> release to push it to stable and it will be synced to maven central.
> There should be no other changes in between.
>
> You can read the ASF guide about "publishing maven artifacts" -
> http://www.apache.org/dev/publishing-maven-artifacts.html
>
> However, I do see different projects have different convention about the
> rc tag name. some projects are using the final tag names and dropping
> the tag if a rc doesn't pass; some projects are using tag names with
> "-rc<rc_num>", and clean up the tag names after an RC is approved.
> We have been always using the final tag names for releases, but if a rc
> tag is preferable, we can go down in that route.
>
>
>
>>
>> >> I ran the tests twice, and both times the following failed for me.
>> >> They're probably flakes, but they failed on 2/2 so they should be
>> >> investigated. This is on a machine that sometimes runs master tests
>> >> cleanly (we really need to sort out our flakes).
>> >> - BookKeeperAdminTest.testDecommissionBookie
>> >> -
>> >> TestRackawareEnsemblePlacementPolicyUsingScript.testNewEnsem
>> bleWithEnoughRacks
>> >> - BookKeeperDiskSpaceWeightedLedgerPlacementTest
>> >>
>> >
>> > Ok let's dig into this.
>> It would be good for others to confirm whether they see the same
>> failures, even if only periodically.
>>
>
> verifying the build on my laptop. will update the results here once it is
> done.
>
>
>>
>> -Ivan
>>
>
>

Re: [VOTE] Release 4.5.1, release candidate #0

[Ivan Kelly <iv...@apache.org>]

> Let's not hijack the release thread. We should move forward with 4.5.1 and
> 4.6.0 release.
4.5.1rc0 is already defunct due to too many flakes and a bad notice
file. There's been two changes since, so I would expect a new rc next
week.

> Ivan, if you have a better idea on improving the release process, let's do
> a separate BP.
I don't think anyone else agrees that it's a bad idea to have signed
binaries going around with the same filename but a different sha, so
I'm going to draw a line under it, and move on.

-Ivan

Re: [VOTE] Release 4.5.1, release candidate #0

[Sijie Guo <gu...@gmail.com>]

Let's not hijack the release thread. We should move forward with 4.5.1 and
4.6.0 release.

Ivan, if you have a better idea on improving the release process, let's do
a separate BP.

- Sijie

On Thu, Nov 9, 2017 at 1:04 AM, Ivan Kelly <iv...@apache.org> wrote:

>
> We don't have to change it now for 4.5.1 or 4.6.0. I'm suggesting we
> think about changing it in the future because it can lead to issues.
> Any time you've solved a problem by deleting your .m2 cache, it's this
> issue.
>
> I will see if I can find another ASF project that does this.
>
>
>

Re: [VOTE] Release 4.5.1, release candidate #0

[Sijie Guo <gu...@gmail.com>]

On Thu, Nov 9, 2017 at 2:48 AM, Ivan Kelly <iv...@apache.org> wrote:

> >> (maybe I missed something here) I checked the maven release process. the
> >> versioning is about the binary/source distribution, which we are doing
> the
> >> same thing, no?
> > maven.apache.org seems down right now :/. I'll check later.
> Ok, their release doc does say "The version used should be the
> eventual version with -RC1, -RC2, etc. appended.", but they don't seem
> to have been following this for a couple of years.
>

As I said, '-RC' suffix is for binary and source distributions, which we
are doing exactly the same thing.

I don't think maven itself push things to artifactory.


>
> Guava, although not an apache project, uses the suffix
> https://github.com/google/guava/blob/v23.0-rc1/pom.xml.
>

I don't think the "RC" has the same meaning here in the ASF projects. If
you just take a look their releases,
you will know a "RC" is an actual "release" and public to all the users.

You can check this on maven central. There are tons of RC versions:
https://mvnrepository.com/artifact/com.google.guava/guava/23.0-rc1


>
> -Ivan
>

Re: [VOTE] Release 4.5.1, release candidate #0

[Ivan Kelly <iv...@apache.org>]

>> (maybe I missed something here) I checked the maven release process. the
>> versioning is about the binary/source distribution, which we are doing the
>> same thing, no?
> maven.apache.org seems down right now :/. I'll check later.
Ok, their release doc does say "The version used should be the
eventual version with -RC1, -RC2, etc. appended.", but they don't seem
to have been following this for a couple of years.

Guava, although not an apache project, uses the suffix
https://github.com/google/guava/blob/v23.0-rc1/pom.xml.

-Ivan

Re: [VOTE] Release 4.5.1, release candidate #0

[Sijie Guo <gu...@gmail.com>]

On Thu, Nov 9, 2017 at 1:04 AM, Ivan Kelly <iv...@apache.org> wrote:

> > it doesn't have the notion of gitsha. but it has the timestamp (I also
> have
> > limited knowledge on this. I can be wrong) and
> > a policy to check for updates. If a RC is bad, a new RC will be produced
> > with new timestamp. Maven should be able to resolve
> > and find there is an update for it.
> If it's a final version number, maven won't update.
>
> Try this with bk master:
> cd ~/.m2/repository/com/google/guava/guava/20.0
> rm guava-20.0.jar guava-20.0.jar.sha1
> touch guava-20.0.jar
> sha1sum guava-20.0.jar | awk '{print $1}' > guava-20.0.jar.sha1
>
> Now try to build. Even though guava 20 jar doesn't match what is in
> remote repos, it won't be updated. The same will happen if there is an
> RC with the same filename as the final release.
>

I am not sure if this method is correct here. first, you are producing a
jar which has higher timestamp.
secondly, if you run maven install, I think there will be a local metadata
file for it.


>
> >> Now, consider the case where one of our users wants to test a new
> >> version of BK works well with the system they have built on top of it,
> >> so they change the version in the file, add the temp repo to the
> >> repos, commit and kick off their CI to run unit, integration, scale
> >> tests etc. If their build system isn't purging the environment every
> >> time, they will end up with a RC artifact as the final artifact for
> >> that version, which is bad.
> >
> > If the RC is bad, a new RC will come out to fix the bad RC. They will run
> > the procedure to verify new RC, no?
> Maybe, maybe not. Really we want our end users to test release
> candidates against their systems, but due to time constraints they may
> not. Or if they're using this CI workflow (I've used this before to
> test a ZK rc), it's quite possible the tests will run on a different
> slave, so it'll use a different cache. It's also quite possible that
> they will use the same slaves for the builds that generate their own
> release builds, so a bookkeeper RC could make it into the released
> product of one of our users.
>

Are there use cases rather than CI workflow? In CI workflow, there are
already mature options for handling maven cache.
I am not sure this is a real big point on changing release flow.


>
> > I don't really see how bad is the current process. Instead, changing it
> to
> > use "-rc" suffix for publishing artifacts, changes the whole release
> process
> > that every other projects are using. It also sounds we are bypassing the
> > staging repository feature provided by Nexus and reinvent a new process
> > for publishing artifacts. I am not sure it really worth doing this
> change.
> > Personally I would like to see if there is a project doing this process
> > before changing it.
> We don't have to change it now for 4.5.1 or 4.6.0. I'm suggesting we
> think about changing it in the future because it can lead to issues.
> Any time you've solved a problem by deleting your .m2 cache, it's this
> issue.
>
> I will see if I can find another ASF project that does this.


> > we don't have the mechanisms to do this gitsha and package matching. we
> > trust release manager.
> > BUT at least, what release manager claimed (tag, binary/source
> > distribution, maven artifacts) are all reviewed and approved.
> > if you do additional RC to final convert, it means new checkin, new tag,
> > new binary/source package, and new artifacts. no one really review and
> > verify all these stuff again.
> This is another thing I meant to bring up. This last step should
> pretty much be automated. Generating an RC should be automated.


Maven release plugin and Nexus are already there for simplifying things.
but you still need to run
it yourself. The ASF policy requires signatures on all the artifacts
published to artifactory.

Besides that, ASF artifactory doesn't allow a direct push. You have to
stage the artifacts into a
staging repo for reviews and release from the staging repo.


> The
> only thing a release manager should have to do is create a branch and
> update versions in the pom, and this should be covered almost entirely
> by the maven release plugin.
> With RC artifact generation and final artifact generation automated,
> this last review becomes useless.
>




>
> >> Maven itself.
> >> http://maven.apache.org/developers/release/maven-core-release.html
> >
> >
> > (maybe I missed something here) I checked the maven release process. the
> > versioning is about the binary/source distribution, which we are doing
> the
> > same thing, no?
> maven.apache.org seems down right now :/. I'll check later.
>
> -Ivan
>

Re: [VOTE] Release 4.5.1, release candidate #0

[Ivan Kelly <iv...@apache.org>]

> it doesn't have the notion of gitsha. but it has the timestamp (I also have
> limited knowledge on this. I can be wrong) and
> a policy to check for updates. If a RC is bad, a new RC will be produced
> with new timestamp. Maven should be able to resolve
> and find there is an update for it.
If it's a final version number, maven won't update.

Try this with bk master:
cd ~/.m2/repository/com/google/guava/guava/20.0
rm guava-20.0.jar guava-20.0.jar.sha1
touch guava-20.0.jar
sha1sum guava-20.0.jar | awk '{print $1}' > guava-20.0.jar.sha1

Now try to build. Even though guava 20 jar doesn't match what is in
remote repos, it won't be updated. The same will happen if there is an
RC with the same filename as the final release.

>> Now, consider the case where one of our users wants to test a new
>> version of BK works well with the system they have built on top of it,
>> so they change the version in the file, add the temp repo to the
>> repos, commit and kick off their CI to run unit, integration, scale
>> tests etc. If their build system isn't purging the environment every
>> time, they will end up with a RC artifact as the final artifact for
>> that version, which is bad.
>
> If the RC is bad, a new RC will come out to fix the bad RC. They will run
> the procedure to verify new RC, no?
Maybe, maybe not. Really we want our end users to test release
candidates against their systems, but due to time constraints they may
not. Or if they're using this CI workflow (I've used this before to
test a ZK rc), it's quite possible the tests will run on a different
slave, so it'll use a different cache. It's also quite possible that
they will use the same slaves for the builds that generate their own
release builds, so a bookkeeper RC could make it into the released
product of one of our users.


> I don't really see how bad is the current process. Instead, changing it to
> use "-rc" suffix for publishing artifacts, changes the whole release process
> that every other projects are using. It also sounds we are bypassing the
> staging repository feature provided by Nexus and reinvent a new process
> for publishing artifacts. I am not sure it really worth doing this change.
> Personally I would like to see if there is a project doing this process
> before changing it.
We don't have to change it now for 4.5.1 or 4.6.0. I'm suggesting we
think about changing it in the future because it can lead to issues.
Any time you've solved a problem by deleting your .m2 cache, it's this
issue.

I will see if I can find another ASF project that does this.

> we don't have the mechanisms to do this gitsha and package matching. we
> trust release manager.
> BUT at least, what release manager claimed (tag, binary/source
> distribution, maven artifacts) are all reviewed and approved.
> if you do additional RC to final convert, it means new checkin, new tag,
> new binary/source package, and new artifacts. no one really review and
> verify all these stuff again.
This is another thing I meant to bring up. This last step should
pretty much be automated. Generating an RC should be automated. The
only thing a release manager should have to do is create a branch and
update versions in the pom, and this should be covered almost entirely
by the maven release plugin.
With RC artifact generation and final artifact generation automated,
this last review becomes useless.

>> Maven itself.
>> http://maven.apache.org/developers/release/maven-core-release.html
>
>
> (maybe I missed something here) I checked the maven release process. the
> versioning is about the binary/source distribution, which we are doing the
> same thing, no?
maven.apache.org seems down right now :/. I'll check later.

-Ivan

Re: [VOTE] Release 4.5.1, release candidate #0

[Sijie Guo <gu...@gmail.com>]

On Wed, Nov 8, 2017 at 1:02 PM, Ivan Kelly <iv...@apache.org> wrote:

> > I am not sure why this will screw up the caches. if a RC is approved as
> the
> > final, then there is no sha change and it is final.
> > Your local installation from a RC repo should be exactly same as the one
> > available in the central. If a RC is not approved,
> > then it means changes will come in next RC, if you already cache the
> > previous RC locally, when a new RC is up, the gitsha
> > is changed, maven will update your local cache.
> Does maven check the gitsha when deciding whether to pull a version?
> My understanding is that, if the version in the dependency is a
> non-snapshot version, and it finds an artifact with that version, it
> will just use that version.
>

it doesn't have the notion of gitsha. but it has the timestamp (I also have
limited knowledge on this. I can be wrong) and
a policy to check for updates. If a RC is bad, a new RC will be produced
with new timestamp. Maven should be able to resolve
and find there is an update for it.

e.g.
https://repository.apache.org/content/repositories/orgapachebookkeeper-1015/org/apache/bookkeeper/bookkeeper-server/maven-metadata.xml


>
> Now, consider the case where one of our users wants to test a new
> version of BK works well with the system they have built on top of it,
> so they change the version in the file, add the temp repo to the
> repos, commit and kick off their CI to run unit, integration, scale
> tests etc. If their build system isn't purging the environment every
> time, they will end up with a RC artifact as the final artifact for
> that version, which is bad.
>

If the RC is bad, a new RC will come out to fix the bad RC. They will run
the procedure to verify new RC, no?

I don't really see how bad is the current process. Instead, changing it to
use "-rc" suffix for publishing artifacts, changes the whole release process
that every other projects are using. It also sounds we are bypassing the
staging repository feature provided by Nexus and reinvent a new process
for publishing artifacts. I am not sure it really worth doing this change.
Personally I would like to see if there is a project doing this process
before changing it.


>
> > The point is if you produce artifacts locally using a RC should be
> exactly
> > same as it is final in public. Otherwise, there is no
> > point for review. Because the process of converting a RC to final will
> > never be reviewed.
> Converting the RC to final will produce a new gitsha, yes. But are we
> actually checking that the gitsha matches the version in maven and the
> tarballs generated. I, at least, generally trust that the release
> manager generated the artifacts from the tag they claimed.
>

we don't have the mechanisms to do this gitsha and package matching. we
trust release manager.
BUT at least, what release manager claimed (tag, binary/source
distribution, maven artifacts) are all reviewed and approved.
if you do additional RC to final convert, it means new checkin, new tag,
new binary/source package, and new artifacts. no one really review and
verify all these stuff again.


>
> > "an rc tag"? Do you mean a git tag name with "-rc" suffix? or do you mean
> > the staging artifacts version with "-rc" suffix?
> I mean an -rc suffix on the version.
>
> > I have seen projects using "-rc" suffix for tag name. but I don't see
> > projects using "-rc" suffix in the artifact version.
> >
> > Can you point me an ASF project that is doing in this way? If it works
> well
> > in other projects, we can learn from them.
> Maven itself.
> http://maven.apache.org/developers/release/maven-core-release.html


(maybe I missed something here) I checked the maven release process. the
versioning is about the binary/source distribution, which we are doing the
same thing, no?

I don't see what is the process of publishing artifacts (does maven publish
artifacts?).
the proposal of using "-rc" is actually breaking the process of having a
staging repo in nexus. Can you point me the section how maven does that?


>
>
> -Ivan
>

Re: [VOTE] Release 4.5.1, release candidate #0

[Ivan Kelly <iv...@apache.org>]

> I am not sure why this will screw up the caches. if a RC is approved as the
> final, then there is no sha change and it is final.
> Your local installation from a RC repo should be exactly same as the one
> available in the central. If a RC is not approved,
> then it means changes will come in next RC, if you already cache the
> previous RC locally, when a new RC is up, the gitsha
> is changed, maven will update your local cache.
Does maven check the gitsha when deciding whether to pull a version?
My understanding is that, if the version in the dependency is a
non-snapshot version, and it finds an artifact with that version, it
will just use that version.

Now, consider the case where one of our users wants to test a new
version of BK works well with the system they have built on top of it,
so they change the version in the file, add the temp repo to the
repos, commit and kick off their CI to run unit, integration, scale
tests etc. If their build system isn't purging the environment every
time, they will end up with a RC artifact as the final artifact for
that version, which is bad.

> The point is if you produce artifacts locally using a RC should be exactly
> same as it is final in public. Otherwise, there is no
> point for review. Because the process of converting a RC to final will
> never be reviewed.
Converting the RC to final will produce a new gitsha, yes. But are we
actually checking that the gitsha matches the version in maven and the
tarballs generated. I, at least, generally trust that the release
manager generated the artifacts from the tag they claimed.

> "an rc tag"? Do you mean a git tag name with "-rc" suffix? or do you mean
> the staging artifacts version with "-rc" suffix?
I mean an -rc suffix on the version.

> I have seen projects using "-rc" suffix for tag name. but I don't see
> projects using "-rc" suffix in the artifact version.
>
> Can you point me an ASF project that is doing in this way? If it works well
> in other projects, we can learn from them.
Maven itself.
http://maven.apache.org/developers/release/maven-core-release.html

-Ivan

Re: [VOTE] Release 4.5.1, release candidate #0

[Sijie Guo <gu...@gmail.com>]

On Wed, Nov 8, 2017 at 9:57 AM, Ivan Kelly <iv...@apache.org> wrote:

> > However, I do see different projects have different convention about the
> rc
> > tag name. some projects are using the final tag names and dropping
> > the tag if a rc doesn't pass; some projects are using tag names with
> > "-rc<rc_num>", and clean up the tag names after an RC is approved.
> > We have been always using the final tag names for releases, but if a rc
> tag
> > is preferable, we can go down in that route.
> As I said before, leaving the rc off the version can really screw up
> people's local maven caches. Even if they don't pull from the RC repo,
> doing a mvn install will create the artifacts locally, and they'll
> never be updated because maven sees it as a final version.
>

I am not sure why this will screw up the caches. if a RC is approved as the
final, then there is no sha change and it is final.
Your local installation from a RC repo should be exactly same as the one
available in the central. If a RC is not approved,
then it means changes will come in next RC, if you already cache the
previous RC locally, when a new RC is up, the gitsha
is changed, maven will update your local cache.

The point is if you produce artifacts locally using a RC should be exactly
same as it is final in public. Otherwise, there is no
point for review. Because the process of converting a RC to final will
never be reviewed.


>
> So my preference would be to add an rc tag. That said, we've never
> done it before, so it would be a change in policy.
>

"an rc tag"? Do you mean a git tag name with "-rc" suffix? or do you mean
the staging artifacts version with "-rc" suffix?

I have seen projects using "-rc" suffix for tag name. but I don't see
projects using "-rc" suffix in the artifact version.

Can you point me an ASF project that is doing in this way? If it works well
in other projects, we can learn from them.


> http://people.apache.org/~ivank/
>
> -Ivan
>

Re: [VOTE] Release 4.5.1, release candidate #0

[Ivan Kelly <iv...@apache.org>]

> However, I do see different projects have different convention about the rc
> tag name. some projects are using the final tag names and dropping
> the tag if a rc doesn't pass; some projects are using tag names with
> "-rc<rc_num>", and clean up the tag names after an RC is approved.
> We have been always using the final tag names for releases, but if a rc tag
> is preferable, we can go down in that route.
As I said before, leaving the rc off the version can really screw up
people's local maven caches. Even if they don't pull from the RC repo,
doing a mvn install will create the artifacts locally, and they'll
never be updated because maven sees it as a final version.

So my preference would be to add an rc tag. That said, we've never
done it before, so it would be a change in policy.
http://people.apache.org/~ivank/

-Ivan

Re: [VOTE] Release 4.5.1, release candidate #0

[Sijie Guo <gu...@gmail.com>]

On Wed, Nov 8, 2017 at 7:45 AM, Ivan Kelly <iv...@apache.org> wrote:

> > This has been done deliberately. Like for 4.5.0.
> > I am ok if you PMC want to do it differently.
> > Or maybe I have misunderstood the guide.
> > You already stated this on slack. I apologize I went forward following
> the
> > guide
> Oh, I'm misremembering how we used to do it, another project I worked
> on had it different.
>
> However, the point still stands. The final release should have a
> different sha1 than the candidates, otherwise people's maven caches
> get messed up, leading to hidden problems down the line.


Are you talking about a tag with "-rc" or artifacts with "-rc"?

I don't think we've ever used "rc" for the staging artifacts. if you take a
look at the "staging repositories" at nexus,
I don't see any projects using "rc" for the staging artifacts.

The purpose of a staging area is once a RC is approved, you can just click
release to push it to stable and it will be synced to maven central.
There should be no other changes in between.

You can read the ASF guide about "publishing maven artifacts" -
http://www.apache.org/dev/publishing-maven-artifacts.html

However, I do see different projects have different convention about the rc
tag name. some projects are using the final tag names and dropping
the tag if a rc doesn't pass; some projects are using tag names with
"-rc<rc_num>", and clean up the tag names after an RC is approved.
We have been always using the final tag names for releases, but if a rc tag
is preferable, we can go down in that route.



>
> >> I ran the tests twice, and both times the following failed for me.
> >> They're probably flakes, but they failed on 2/2 so they should be
> >> investigated. This is on a machine that sometimes runs master tests
> >> cleanly (we really need to sort out our flakes).
> >> - BookKeeperAdminTest.testDecommissionBookie
> >> -
> >> TestRackawareEnsemblePlacementPolicyUsingScript.
> testNewEnsembleWithEnoughRacks
> >> - BookKeeperDiskSpaceWeightedLedgerPlacementTest
> >>
> >
> > Ok let's dig into this.
> It would be good for others to confirm whether they see the same
> failures, even if only periodically.
>

verifying the build on my laptop. will update the results here once it is
done.


>
> -Ivan
>

Re: [VOTE] Release 4.5.1, release candidate #0

[Ivan Kelly <iv...@apache.org>]

> This has been done deliberately. Like for 4.5.0.
> I am ok if you PMC want to do it differently.
> Or maybe I have misunderstood the guide.
> You already stated this on slack. I apologize I went forward following the
> guide
Oh, I'm misremembering how we used to do it, another project I worked
on had it different.

However, the point still stands. The final release should have a
different sha1 than the candidates, otherwise people's maven caches
get messed up, leading to hidden problems down the line.

>> I ran the tests twice, and both times the following failed for me.
>> They're probably flakes, but they failed on 2/2 so they should be
>> investigated. This is on a machine that sometimes runs master tests
>> cleanly (we really need to sort out our flakes).
>> - BookKeeperAdminTest.testDecommissionBookie
>> -
>> TestRackawareEnsemblePlacementPolicyUsingScript.testNewEnsembleWithEnoughRacks
>> - BookKeeperDiskSpaceWeightedLedgerPlacementTest
>>
>
> Ok let's dig into this.
It would be good for others to confirm whether they see the same
failures, even if only periodically.

-Ivan

Re: [VOTE] Release 4.5.1, release candidate #0

[Ivan Kelly <iv...@apache.org>]

> I think JLine was there before. We excluded this dependency at some point.
It was a transitive from zk, that it uses for commandline.

-Ivan

Re: [VOTE] Release 4.5.1, release candidate #0

[Sijie Guo <gu...@gmail.com>]

On Wed, Nov 8, 2017 at 7:40 AM, Enrico Olivelli <eo...@gmail.com> wrote:

> Ivan,
> Thank you for your review
>
> Il mer 8 nov 2017, 16:31 Ivan Kelly <iv...@apache.org> ha scritto:
>
> > Hey Enrico,
> >
> > Thanks for putting this together. I'm afraid it's a -1 from me on this
> > bundle.
> >
> > Artifacts should have rc in the version. Otherwise they'll eclipse the
> > final release in people's maven caches
>
>
> This has been done deliberately. Like for 4.5.0.
> I am ok if you PMC want to do it differently.
> Or maybe I have misunderstood the guide.
> You already stated this on slack. I apologize I went forward following the
> guide
>
> .
> >
> > JLine is in the notice, but no file shipped.
> >
>
> Ok I will create an issue and fix it, I think it was so in 4.5.0 package.
> Anyway it is a simple fix
>

I think JLine was there before. We excluded this dependency at some point.

- Sijie


>
> >
> > I ran the tests twice, and both times the following failed for me.
> > They're probably flakes, but they failed on 2/2 so they should be
> > investigated. This is on a machine that sometimes runs master tests
> > cleanly (we really need to sort out our flakes).
> > - BookKeeperAdminTest.testDecommissionBookie
> > -
> > TestRackawareEnsemblePlacementPolicyUsingScript.
> testNewEnsembleWithEnoughRacks
> > - BookKeeperDiskSpaceWeightedLedgerPlacementTest
> >
>
> Ok let's dig into this.
>
> So I am CANCELLING thia vote for rc0
>
> Enrico
>
>
>
> > -Ivan
> >
> > On Mon, Nov 6, 2017 at 11:31 AM, Enrico Olivelli <eo...@gmail.com>
> > wrote:
> > > Hi everyone,
> > > Please review and vote on the release candidate #0 for the version
> > > 4.5.1, as follows:
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > > The complete staging area is available for your review, which includes:
> > > * Release notes [1]
> > > * The official Apache source and binary distributions to be deployed
> > > to dist.apache.org [2]
> > > * All artifacts to be deployed to the Maven Central Repository [3]
> > > * Source code tag "release-4.5.1" [4]
> > >
> > > BookKeeper's KEYS file contains PGP keys we used to sign this
> > > release:https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > >
> > > Please download these packages and review this release candidate:
> > >
> > > - Review release notes
> > > - Download the source package (verify md5, shasum, and asc) and follow
> > the
> > > instructions to build and run the bookkeeper service.
> > > - Download the binary package (verify md5, shasum, and asc) and follow
> > the
> > > instructions to run the bookkeeper service.
> > > - Review maven repo, release tag, licenses, and any other things you
> > think
> > > it is important to a release.
> > >
> > > The vote will be open for at least 72 hours. It is adopted by majority
> > > approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Enrico Olivelli
> > >
> > > [1] https://github.com/apache/bookkeeper/pull/678
> > > [2]
> > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.5.1-rc0/
> > > [3]
> > https://repository.apache.org/content/repositories/
> orgapachebookkeeper-1015/org/apache/bookkeeper/
> > > [4] https://github.com/apache/bookkeeper/tree/release-4.5.1
> >
> --
>
>
> -- Enrico Olivelli
>

Re: [VOTE] Release 4.5.1, release candidate #0

[Enrico Olivelli <eo...@gmail.com>]

Ivan,
Thank you for your review

Il mer 8 nov 2017, 16:31 Ivan Kelly <iv...@apache.org> ha scritto:

> Hey Enrico,
>
> Thanks for putting this together. I'm afraid it's a -1 from me on this
> bundle.
>
> Artifacts should have rc in the version. Otherwise they'll eclipse the
> final release in people's maven caches


This has been done deliberately. Like for 4.5.0.
I am ok if you PMC want to do it differently.
Or maybe I have misunderstood the guide.
You already stated this on slack. I apologize I went forward following the
guide

.
>
> JLine is in the notice, but no file shipped.
>

Ok I will create an issue and fix it, I think it was so in 4.5.0 package.
Anyway it is a simple fix

>
> I ran the tests twice, and both times the following failed for me.
> They're probably flakes, but they failed on 2/2 so they should be
> investigated. This is on a machine that sometimes runs master tests
> cleanly (we really need to sort out our flakes).
> - BookKeeperAdminTest.testDecommissionBookie
> -
> TestRackawareEnsemblePlacementPolicyUsingScript.testNewEnsembleWithEnoughRacks
> - BookKeeperDiskSpaceWeightedLedgerPlacementTest
>

Ok let's dig into this.

So I am CANCELLING thia vote for rc0

Enrico



> -Ivan
>
> On Mon, Nov 6, 2017 at 11:31 AM, Enrico Olivelli <eo...@gmail.com>
> wrote:
> > Hi everyone,
> > Please review and vote on the release candidate #0 for the version
> > 4.5.1, as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which includes:
> > * Release notes [1]
> > * The official Apache source and binary distributions to be deployed
> > to dist.apache.org [2]
> > * All artifacts to be deployed to the Maven Central Repository [3]
> > * Source code tag "release-4.5.1" [4]
> >
> > BookKeeper's KEYS file contains PGP keys we used to sign this
> > release:https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> >
> > Please download these packages and review this release candidate:
> >
> > - Review release notes
> > - Download the source package (verify md5, shasum, and asc) and follow
> the
> > instructions to build and run the bookkeeper service.
> > - Download the binary package (verify md5, shasum, and asc) and follow
> the
> > instructions to run the bookkeeper service.
> > - Review maven repo, release tag, licenses, and any other things you
> think
> > it is important to a release.
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Enrico Olivelli
> >
> > [1] https://github.com/apache/bookkeeper/pull/678
> > [2]
> https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.5.1-rc0/
> > [3]
> https://repository.apache.org/content/repositories/orgapachebookkeeper-1015/org/apache/bookkeeper/
> > [4] https://github.com/apache/bookkeeper/tree/release-4.5.1
>
-- 


-- Enrico Olivelli

Re: [VOTE] Release 4.5.1, release candidate #0

[Ivan Kelly <iv...@apache.org>]

Hey Enrico,

Thanks for putting this together. I'm afraid it's a -1 from me on this bundle.

Artifacts should have rc in the version. Otherwise they'll eclipse the
final release in people's maven caches.

JLine is in the notice, but no file shipped.

I ran the tests twice, and both times the following failed for me.
They're probably flakes, but they failed on 2/2 so they should be
investigated. This is on a machine that sometimes runs master tests
cleanly (we really need to sort out our flakes).
- BookKeeperAdminTest.testDecommissionBookie
- TestRackawareEnsemblePlacementPolicyUsingScript.testNewEnsembleWithEnoughRacks
- BookKeeperDiskSpaceWeightedLedgerPlacementTest

-Ivan

On Mon, Nov 6, 2017 at 11:31 AM, Enrico Olivelli <eo...@gmail.com> wrote:
> Hi everyone,
> Please review and vote on the release candidate #0 for the version
> 4.5.1, as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * Release notes [1]
> * The official Apache source and binary distributions to be deployed
> to dist.apache.org [2]
> * All artifacts to be deployed to the Maven Central Repository [3]
> * Source code tag "release-4.5.1" [4]
>
> BookKeeper's KEYS file contains PGP keys we used to sign this
> release:https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
>
> Please download these packages and review this release candidate:
>
> - Review release notes
> - Download the source package (verify md5, shasum, and asc) and follow the
> instructions to build and run the bookkeeper service.
> - Download the binary package (verify md5, shasum, and asc) and follow the
> instructions to run the bookkeeper service.
> - Review maven repo, release tag, licenses, and any other things you think
> it is important to a release.
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Enrico Olivelli
>
> [1] https://github.com/apache/bookkeeper/pull/678
> [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.5.1-rc0/
> [3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1015/org/apache/bookkeeper/
> [4] https://github.com/apache/bookkeeper/tree/release-4.5.1

Re: [VOTE] Release 4.5.1, release candidate #0

[Enrico Olivelli <eo...@gmail.com>]

Il mar 7 nov 2017, 01:47 Jia Zhai <zh...@gmail.com> ha scritto:

> +1,
>
> - verified packages checksum (md5, asc and sha1 all good)
>
> - the source package build and test all run successfully.
>
> - NOTICE, DISCLAIME, License headers in binary package look good.
>
> A minor issue is that the docker and site directory only contains a
> README.md, and all other files are missing. Is this deliberate?
>

No. I think that this maybe is caused from the changes on the assembly
plugin config.
Anyway on the tar ball of 4.5.0  there is no docker stuff and there is the
website source code.
I think this is not blocking the release.
Thank you Jia


> On Mon, Nov 6, 2017 at 6:31 PM, Enrico Olivelli <eo...@gmail.com>
> wrote:
>
> > Hi everyone,
> > Please review and vote on the release candidate #0 for the version
> > 4.5.1, as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which includes:
> > * Release notes [1]
> > * The official Apache source and binary distributions to be deployed
> > to dist.apache.org [2]
> > * All artifacts to be deployed to the Maven Central Repository [3]
> > * Source code tag "release-4.5.1" [4]
> >
> > BookKeeper's KEYS file contains PGP keys we used to sign this
> > release:https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> >
> > Please download these packages and review this release candidate:
> >
> > - Review release notes
> > - Download the source package (verify md5, shasum, and asc) and follow
> the
> > instructions to build and run the bookkeeper service.
> > - Download the binary package (verify md5, shasum, and asc) and follow
> the
> > instructions to run the bookkeeper service.
> > - Review maven repo, release tag, licenses, and any other things you
> think
> > it is important to a release.
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Enrico Olivelli
> >
> > [1] https://github.com/apache/bookkeeper/pull/678
> > [2] https://dist.apache.org/repos/dist/dev/bookkeeper/
> > bookkeeper-4.5.1-rc0/
> > [3] https://repository.apache.org/content/repositories/
> > orgapachebookkeeper-1015/org/apache/bookkeeper/
> > [4] https://github.com/apache/bookkeeper/tree/release-4.5.1
> >
>
-- 


-- Enrico Olivelli

Re: [VOTE] Release 4.5.1, release candidate #0

[Jia Zhai <zh...@gmail.com>]

+1,

- verified packages checksum (md5, asc and sha1 all good)

- the source package build and test all run successfully.

- NOTICE, DISCLAIME, License headers in binary package look good.

A minor issue is that the docker and site directory only contains a
README.md, and all other files are missing. Is this deliberate?

On Mon, Nov 6, 2017 at 6:31 PM, Enrico Olivelli <eo...@gmail.com> wrote:

> Hi everyone,
> Please review and vote on the release candidate #0 for the version
> 4.5.1, as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * Release notes [1]
> * The official Apache source and binary distributions to be deployed
> to dist.apache.org [2]
> * All artifacts to be deployed to the Maven Central Repository [3]
> * Source code tag "release-4.5.1" [4]
>
> BookKeeper's KEYS file contains PGP keys we used to sign this
> release:https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
>
> Please download these packages and review this release candidate:
>
> - Review release notes
> - Download the source package (verify md5, shasum, and asc) and follow the
> instructions to build and run the bookkeeper service.
> - Download the binary package (verify md5, shasum, and asc) and follow the
> instructions to run the bookkeeper service.
> - Review maven repo, release tag, licenses, and any other things you think
> it is important to a release.
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Enrico Olivelli
>
> [1] https://github.com/apache/bookkeeper/pull/678
> [2] https://dist.apache.org/repos/dist/dev/bookkeeper/
> bookkeeper-4.5.1-rc0/
> [3] https://repository.apache.org/content/repositories/
> orgapachebookkeeper-1015/org/apache/bookkeeper/
> [4] https://github.com/apache/bookkeeper/tree/release-4.5.1
>