You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@impala.apache.org by Jim Apple <jb...@cloudera.com> on 2016/09/07 21:08:36 UTC
Prepare to vote on release candidate 1
I have put a release candidate, along with checksums and a
cryptographic signature, in
https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
I will be calling for a vote from the PPMC soon. This thread is not
the vote thread.
That vote will only pass, according to our bylaws, if it has 3 binding
+1 votes and more binding +1 votes than -1 votes. Only the votes of
PPMC members are binding, but anyone may vote.
If that vote passes, I will ask the Incubator PMC to approve the
release candidate, following the rules on
http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
To +1 a release candidate, you will need to verify it. This includes:
1. Verifying the signature. You can import my code-signing public key
from gpg using the instructions in
https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
You can also find that key at
https://pgp.mit.edu/pks/lookup?op=get&search=0x91EE43066850196C
or
http://home.apache.org/keys/committer/jbapple
You will be able to verify the signature by typing
gpg --verify apache-impala-incubating-2.7.0-rc1.tar.gz.asc
apache-impala-incubating-2.7.0-rc1.tar.gz
This should happen on a machine you are the sole administrator of and
that you have physical control of:
http://www.apache.org/dev/release.html#owned-controlled-hardware
2. Build and test. You can do the testing on another machine - for
instance, you can upload the RC1 tree to a git repo and point your CI
tool at that repo.
3. "verify[ing] that the package meets the requirements of the ASF
policy on releases"
I suppose that means http://www.apache.org/dev/release.html. I am
asking for clarification on that from our incubating mentors.
Thank you!
Re: Prepare to vote on release candidate 1
Posted by Jim Apple <jb...@cloudera.com>.
I like this idea. For RC2, I'll plan to add a directory in
https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
called "RC2", and put the files there with the names starting with
"apache-impala-incubating-2.7.0.tar.gz".
On Wed, Sep 7, 2016 at 5:01 PM, Tim Armstrong <ta...@cloudera.com>
wrote:
> I'm in the process of testing this.
>
> One nit that I think we should fix: apache-impala-incubating-2.7.
> 0-rc1.tar.gz
> unpacks to incubator-impala. I think we should stick with the normal
> convention of unpacking to a directory with the same name as the tarball
> (or maybe apache-impala-incubating-2.7.0).
>
> On Wed, Sep 7, 2016 at 2:16 PM, Henry Robinson <he...@cloudera.com> wrote:
>
> > Jim, this is great work (by everyone) and the culmination of a ton of
> > effort. Thanks for getting us to this stage!
> >
> > On 7 September 2016 at 14:08, Jim Apple <jb...@cloudera.com> wrote:
> >
> > > I have put a release candidate, along with checksums and a
> > > cryptographic signature, in
> > >
> > > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
> > >
> > > I will be calling for a vote from the PPMC soon. This thread is not
> > > the vote thread.
> > >
> > > That vote will only pass, according to our bylaws, if it has 3 binding
> > > +1 votes and more binding +1 votes than -1 votes. Only the votes of
> > > PPMC members are binding, but anyone may vote.
> > >
> > > If that vote passes, I will ask the Incubator PMC to approve the
> > > release candidate, following the rules on
> > >
> > > http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
> > >
> > > To +1 a release candidate, you will need to verify it. This includes:
> > >
> > > 1. Verifying the signature. You can import my code-signing public key
> > > from gpg using the instructions in
> > >
> > > https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
> > >
> > > You can also find that key at
> > >
> > > https://pgp.mit.edu/pks/lookup?op=get&search=0x91EE43066850196C
> > >
> > > or
> > >
> > > http://home.apache.org/keys/committer/jbapple
> > >
> > > You will be able to verify the signature by typing
> > >
> > > gpg --verify apache-impala-incubating-2.7.0-rc1.tar.gz.asc
> > > apache-impala-incubating-2.7.0-rc1.tar.gz
> > >
> > > This should happen on a machine you are the sole administrator of and
> > > that you have physical control of:
> > >
> > > http://www.apache.org/dev/release.html#owned-controlled-hardware
> > >
> > > 2. Build and test. You can do the testing on another machine - for
> > > instance, you can upload the RC1 tree to a git repo and point your CI
> > > tool at that repo.
> > >
> > > 3. "verify[ing] that the package meets the requirements of the ASF
> > > policy on releases"
> > >
> > > I suppose that means http://www.apache.org/dev/release.html. I am
> > > asking for clarification on that from our incubating mentors.
> > >
> > > Thank you!
> > >
> >
> >
> >
> > --
> > Henry Robinson
> > Software Engineer
> > Cloudera
> > 415-994-6679
> >
>
Re: Prepare to vote on release candidate 1
Posted by Jim Apple <jb...@cloudera.com>.
Just a quick update with where I am with your RC1 feedback. First, let me
thank you for all of the feedback - it has been great for Impala, in my
opinion.
Here is a list of things that I am still planning to do for RC2:
1. Change the tarball name to remove the '-rc1'; instead, I will put the
next tarball, checksums, and signature into an 'rc2' directory.
2. Change the tarball so it unpacks into apache-impala-incubating-2.7.0.
3. Fix some 'Cloudera' licensing: pending at
https://gerrit.cloudera.org/#/c/4386/
4. Make buildall.sh work when not in a git repo: CI verification happening
at https://gerrit.cloudera.org/#/c/4336/
5. Version numbering: Apparently it is a problem for Cloudera downstream if
the version doesn't have the string 'cdh5'. If they (with my Apache hat on)
/ we (with my Cloudera hat on) can't fix this ASAP, I will change the
version string in the 2.7.0 branch (though not on master) and let Cloudera
deal with whatever Cloudera problems there are.
6. Git tag (and provide a git SHA for) the next RC
7. Get a clean RAT run: in progress at
https://gerrit.cloudera.org/#/c/4361/2 . This may take a while as I sort
out the testdata/data/mstr and /tests/comparison/leopard directories.
On Fri, Sep 9, 2016 at 11:52 AM, Jim Apple <jb...@cloudera.com> wrote:
> >> - would be nice if the version number didn't have 'cdh5' in it (eg
> >> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
> >> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
> >
> > Lars is working on this now:
> >
> > https://gerrit.cloudera.org/#/c/4187/3
> >
> > Some possibilities for 2.7.0:
> >
> > 1. Leave it. Pros: expedient; Cons: inaccurate, not Apache
> >
> > 2. Wait until we can change this is master without confusing existing
> > tools. Pros: clean; Cons: delay in releasing
> >
> > 3. Make a commit just to branch-2.7.0. Pros: Doesn't confuse existing
> > tools; Cons: Makes branch-2.7.0 not a pure subset of master.
> >
> > What does everyone think?
>
> Just a reminder that your input on this is valued. I slightly prefer #3.
>
Re: Prepare to vote on release candidate 1
Posted by Jim Apple <jb...@cloudera.com>.
>> - would be nice if the version number didn't have 'cdh5' in it (eg
>> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
>> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
>
> Lars is working on this now:
>
> https://gerrit.cloudera.org/#/c/4187/3
>
> Some possibilities for 2.7.0:
>
> 1. Leave it. Pros: expedient; Cons: inaccurate, not Apache
>
> 2. Wait until we can change this is master without confusing existing
> tools. Pros: clean; Cons: delay in releasing
>
> 3. Make a commit just to branch-2.7.0. Pros: Doesn't confuse existing
> tools; Cons: Makes branch-2.7.0 not a pure subset of master.
>
> What does everyone think?
Just a reminder that your input on this is valued. I slightly prefer #3.
Re: Prepare to vote on release candidate 1
Posted by Jim Apple <jb...@cloudera.com>.
Thank you for all of the suggestions
> - A couple places seem to keep a Cloudera copyright/license - eg
> common/thrift/beeswax.thrift, FrontEndTestBase.java, etc. (I grepped for
> Cloudera and found these). Should probably check over these and fix before
> the "real" RC.
Reopened https://issues.cloudera.org/browse/IMPALA-3918
> - I tried building using 'buildall.sh' but it failed with 'fatal: Not a git
> repository' when trying to do a git clean. Maybe some tweaks need to be
> made so that Impala can be built from a tarball? I worked around it using
> 'git init' inside my extracted directory.
https://gerrit.cloudera.org/#/c/4336/
When this is in master, I'll cherry-pick it to branch-2.7.0
> - would be nice if the version number didn't have 'cdh5' in it (eg
> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
Lars is working on this now:
https://gerrit.cloudera.org/#/c/4187/3
Some possibilities for 2.7.0:
1. Leave it. Pros: expedient; Cons: inaccurate, not Apache
2. Wait until we can change this is master without confusing existing
tools. Pros: clean; Cons: delay in releasing
3. Make a commit just to branch-2.7.0. Pros: Doesn't confuse existing
tools; Cons: Makes branch-2.7.0 not a pure subset of master.
What does everyone think?
> - can't seem to use testdata/bin/run-all.sh to start DFS. I get:
> Starting hdfs (Web UI - http://localhost:5070)
> Failed to start hdfs-datanode. The end of the log
> (/tmp/incubator-impala/testdata/cluster/cdh5/node-2/var/log/hdfs-datanode.out)
> is:
> Failed to start hdfs-datanode. The end of the log
> (/tmp/incubator-impala/testdata/cluster/cdh5/node-3/var/log/hdfs-datanode.out)
> is:
> Failed to start hdfs-datanode. The end of the log
> (/tmp/incubator-impala/testdata/cluster/cdh5/node-1/var/log/hdfs-datanode.out)
> is:
>
> That said, it appears that impalad built OK (with build-all.sh -notests)
I could fix this by making sure IMPALA_HOME was set correctly and
sourcing bin/impala-config.sh.
> some nits/suggestions (ok to address in a later release:
> - a few mentions of 'Cloudera Impala' in the various pom files
Opened https://issues.cloudera.org/browse/IMPALA-4093
> - would be great if buildall.sh could check if there is enough remaining
> space on the drive before building. I had 20GB free but still ran out of
> space trying to do the default build (had to rebuild with -notests)
Opened https://issues.cloudera.org/browse/IMPALA-4094
> - consider setting up a cloudfront distribution in front of the
> native-toolchain S3 bucket? It downloads pretty slowly for me
Filed https://issues.cloudera.org/browse/IMPALA-4095
> -- related: consider stripping debug info from some of the built deps? eg
> Kudu is 500+MB which seems unnecessary for a test dependency.
Which object is this? I don't find it.
$ find . -iname '*kudu*' -size +100M | wc -l
0
Thanks again, Todd! Your input as our mentor has been invaluable.
Re: Prepare to vote on release candidate 1
Posted by Tim Armstrong <ta...@cloudera.com>.
I was able to run exhaustive tests successfully after committing the entire
tarball to a fresh git repository.
On Thu, Sep 8, 2016 at 9:27 AM, Jim Apple <jb...@cloudera.com> wrote:
> Sure, I can tag the tree where I git archive'd my next RC. Just to be
> sure I understand you - what exactly do you mean by the "git SHA"?
>
> On Thu, Sep 8, 2016 at 2:24 AM, Tom White <to...@cloudera.com> wrote:
> > Jim,
> >
> > Can you provide a git tag and git SHA for the release when you send
> > out the vote so we can check the tarball against the tag.
> >
> > Thanks!
> > Tom
> >
> > On Thu, Sep 8, 2016 at 1:37 AM, Todd Lipcon <to...@cloudera.com> wrote:
> >> - agree with Tim that it would be nice if the tarball extracted into a
> >> directory named apache-impala-incubating-2.7.0 (to match the tarball
> prefix)
> >>
> >> - A couple places seem to keep a Cloudera copyright/license - eg
> >> common/thrift/beeswax.thrift, FrontEndTestBase.java, etc. (I grepped for
> >> Cloudera and found these). Should probably check over these and fix
> before
> >> the "real" RC.
> >>
> >> - I tried building using 'buildall.sh' but it failed with 'fatal: Not a
> git
> >> repository' when trying to do a git clean. Maybe some tweaks need to be
> >> made so that Impala can be built from a tarball? I worked around it
> using
> >> 'git init' inside my extracted directory.
> >>
> >> - would be nice if the version number didn't have 'cdh5' in it (eg
> >> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
> >> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
> >>
> >> - can't seem to use testdata/bin/run-all.sh to start DFS. I get:
> >> Starting hdfs (Web UI - http://localhost:5070)
> >> Failed to start hdfs-datanode. The end of the log
> >> (/tmp/incubator-impala/testdata/cluster/cdh5/node-2/
> var/log/hdfs-datanode.out)
> >> is:
> >> Failed to start hdfs-datanode. The end of the log
> >> (/tmp/incubator-impala/testdata/cluster/cdh5/node-3/
> var/log/hdfs-datanode.out)
> >> is:
> >> Failed to start hdfs-datanode. The end of the log
> >> (/tmp/incubator-impala/testdata/cluster/cdh5/node-1/
> var/log/hdfs-datanode.out)
> >> is:
> >>
> >> That said, it appears that impalad built OK (with build-all.sh -notests)
> >>
> >>
> >> some nits/suggestions (ok to address in a later release:
> >> - a few mentions of 'Cloudera Impala' in the various pom files
> >> - would be great if buildall.sh could check if there is enough remaining
> >> space on the drive before building. I had 20GB free but still ran out of
> >> space trying to do the default build (had to rebuild with -notests)
> >> - consider setting up a cloudfront distribution in front of the
> >> native-toolchain S3 bucket? It downloads pretty slowly for me
> >> -- related: consider stripping debug info from some of the built deps?
> eg
> >> Kudu is 500+MB which seems unnecessary for a test dependency.
> >>
> >>
> >> On Wed, Sep 7, 2016 at 5:01 PM, Tim Armstrong <ta...@cloudera.com>
> >> wrote:
> >>
> >>> I'm in the process of testing this.
> >>>
> >>> One nit that I think we should fix: apache-impala-incubating-2.7.
> >>> 0-rc1.tar.gz
> >>> unpacks to incubator-impala. I think we should stick with the normal
> >>> convention of unpacking to a directory with the same name as the
> tarball
> >>> (or maybe apache-impala-incubating-2.7.0).
> >>>
> >>> On Wed, Sep 7, 2016 at 2:16 PM, Henry Robinson <he...@cloudera.com>
> wrote:
> >>>
> >>> > Jim, this is great work (by everyone) and the culmination of a ton of
> >>> > effort. Thanks for getting us to this stage!
> >>> >
> >>> > On 7 September 2016 at 14:08, Jim Apple <jb...@cloudera.com>
> wrote:
> >>> >
> >>> > > I have put a release candidate, along with checksums and a
> >>> > > cryptographic signature, in
> >>> > >
> >>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
> >>> > >
> >>> > > I will be calling for a vote from the PPMC soon. This thread is not
> >>> > > the vote thread.
> >>> > >
> >>> > > That vote will only pass, according to our bylaws, if it has 3
> binding
> >>> > > +1 votes and more binding +1 votes than -1 votes. Only the votes of
> >>> > > PPMC members are binding, but anyone may vote.
> >>> > >
> >>> > > If that vote passes, I will ask the Incubator PMC to approve the
> >>> > > release candidate, following the rules on
> >>> > >
> >>> > > http://incubator.apache.org/incubation/Incubation_Policy.
> html#Releases
> >>> > >
> >>> > > To +1 a release candidate, you will need to verify it. This
> includes:
> >>> > >
> >>> > > 1. Verifying the signature. You can import my code-signing public
> key
> >>> > > from gpg using the instructions in
> >>> > >
> >>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
> >>> > >
> >>> > > You can also find that key at
> >>> > >
> >>> > > https://pgp.mit.edu/pks/lookup?op=get&search=0x91EE43066850196C
> >>> > >
> >>> > > or
> >>> > >
> >>> > > http://home.apache.org/keys/committer/jbapple
> >>> > >
> >>> > > You will be able to verify the signature by typing
> >>> > >
> >>> > > gpg --verify apache-impala-incubating-2.7.0-rc1.tar.gz.asc
> >>> > > apache-impala-incubating-2.7.0-rc1.tar.gz
> >>> > >
> >>> > > This should happen on a machine you are the sole administrator of
> and
> >>> > > that you have physical control of:
> >>> > >
> >>> > > http://www.apache.org/dev/release.html#owned-controlled-hardware
> >>> > >
> >>> > > 2. Build and test. You can do the testing on another machine - for
> >>> > > instance, you can upload the RC1 tree to a git repo and point your
> CI
> >>> > > tool at that repo.
> >>> > >
> >>> > > 3. "verify[ing] that the package meets the requirements of the ASF
> >>> > > policy on releases"
> >>> > >
> >>> > > I suppose that means http://www.apache.org/dev/release.html. I am
> >>> > > asking for clarification on that from our incubating mentors.
> >>> > >
> >>> > > Thank you!
> >>> > >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Henry Robinson
> >>> > Software Engineer
> >>> > Cloudera
> >>> > 415-994-6679
> >>> >
> >>>
> >>
> >>
> >>
> >> --
> >> Todd Lipcon
> >> Software Engineer, Cloudera
>
Re: Prepare to vote on release candidate 1
Posted by Jim Apple <jb...@cloudera.com>.
Do you mean of the tree, ala
d4b8675dd7a717c80305f5c35ef0179e9c2034b3 is the hash of the tree
https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=tree;h=d4b8675dd7a717c80305f5c35ef0179e9c2034b3;hb=fcb5c6821d1a0b2d49212dd791c4556dd5ac6c9e
which I got by looking at this commit
https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=commit;h=fcb5c6821d1a0b2d49212dd791c4556dd5ac6c9e
which is the latest commit to branch-2.7.0
On Fri, Sep 9, 2016 at 2:04 AM, Tom White <to...@cloudera.com> wrote:
> On Thu, Sep 8, 2016 at 5:27 PM, Jim Apple <jb...@cloudera.com> wrote:
>> Sure, I can tag the tree where I git archive'd my next RC. Just to be
>> sure I understand you - what exactly do you mean by the "git SHA"?
>
> I just mean git hash.
>
>>
>> On Thu, Sep 8, 2016 at 2:24 AM, Tom White <to...@cloudera.com> wrote:
>>> Jim,
>>>
>>> Can you provide a git tag and git SHA for the release when you send
>>> out the vote so we can check the tarball against the tag.
>>>
>>> Thanks!
>>> Tom
>>>
>>> On Thu, Sep 8, 2016 at 1:37 AM, Todd Lipcon <to...@cloudera.com> wrote:
>>>> - agree with Tim that it would be nice if the tarball extracted into a
>>>> directory named apache-impala-incubating-2.7.0 (to match the tarball prefix)
>>>>
>>>> - A couple places seem to keep a Cloudera copyright/license - eg
>>>> common/thrift/beeswax.thrift, FrontEndTestBase.java, etc. (I grepped for
>>>> Cloudera and found these). Should probably check over these and fix before
>>>> the "real" RC.
>>>>
>>>> - I tried building using 'buildall.sh' but it failed with 'fatal: Not a git
>>>> repository' when trying to do a git clean. Maybe some tweaks need to be
>>>> made so that Impala can be built from a tarball? I worked around it using
>>>> 'git init' inside my extracted directory.
>>>>
>>>> - would be nice if the version number didn't have 'cdh5' in it (eg
>>>> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
>>>> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
>>>>
>>>> - can't seem to use testdata/bin/run-all.sh to start DFS. I get:
>>>> Starting hdfs (Web UI - http://localhost:5070)
>>>> Failed to start hdfs-datanode. The end of the log
>>>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-2/var/log/hdfs-datanode.out)
>>>> is:
>>>> Failed to start hdfs-datanode. The end of the log
>>>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-3/var/log/hdfs-datanode.out)
>>>> is:
>>>> Failed to start hdfs-datanode. The end of the log
>>>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-1/var/log/hdfs-datanode.out)
>>>> is:
>>>>
>>>> That said, it appears that impalad built OK (with build-all.sh -notests)
>>>>
>>>>
>>>> some nits/suggestions (ok to address in a later release:
>>>> - a few mentions of 'Cloudera Impala' in the various pom files
>>>> - would be great if buildall.sh could check if there is enough remaining
>>>> space on the drive before building. I had 20GB free but still ran out of
>>>> space trying to do the default build (had to rebuild with -notests)
>>>> - consider setting up a cloudfront distribution in front of the
>>>> native-toolchain S3 bucket? It downloads pretty slowly for me
>>>> -- related: consider stripping debug info from some of the built deps? eg
>>>> Kudu is 500+MB which seems unnecessary for a test dependency.
>>>>
>>>>
>>>> On Wed, Sep 7, 2016 at 5:01 PM, Tim Armstrong <ta...@cloudera.com>
>>>> wrote:
>>>>
>>>>> I'm in the process of testing this.
>>>>>
>>>>> One nit that I think we should fix: apache-impala-incubating-2.7.
>>>>> 0-rc1.tar.gz
>>>>> unpacks to incubator-impala. I think we should stick with the normal
>>>>> convention of unpacking to a directory with the same name as the tarball
>>>>> (or maybe apache-impala-incubating-2.7.0).
>>>>>
>>>>> On Wed, Sep 7, 2016 at 2:16 PM, Henry Robinson <he...@cloudera.com> wrote:
>>>>>
>>>>> > Jim, this is great work (by everyone) and the culmination of a ton of
>>>>> > effort. Thanks for getting us to this stage!
>>>>> >
>>>>> > On 7 September 2016 at 14:08, Jim Apple <jb...@cloudera.com> wrote:
>>>>> >
>>>>> > > I have put a release candidate, along with checksums and a
>>>>> > > cryptographic signature, in
>>>>> > >
>>>>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
>>>>> > >
>>>>> > > I will be calling for a vote from the PPMC soon. This thread is not
>>>>> > > the vote thread.
>>>>> > >
>>>>> > > That vote will only pass, according to our bylaws, if it has 3 binding
>>>>> > > +1 votes and more binding +1 votes than -1 votes. Only the votes of
>>>>> > > PPMC members are binding, but anyone may vote.
>>>>> > >
>>>>> > > If that vote passes, I will ask the Incubator PMC to approve the
>>>>> > > release candidate, following the rules on
>>>>> > >
>>>>> > > http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
>>>>> > >
>>>>> > > To +1 a release candidate, you will need to verify it. This includes:
>>>>> > >
>>>>> > > 1. Verifying the signature. You can import my code-signing public key
>>>>> > > from gpg using the instructions in
>>>>> > >
>>>>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
>>>>> > >
>>>>> > > You can also find that key at
>>>>> > >
>>>>> > > https://pgp.mit.edu/pks/lookup?op=get&search=0x91EE43066850196C
>>>>> > >
>>>>> > > or
>>>>> > >
>>>>> > > http://home.apache.org/keys/committer/jbapple
>>>>> > >
>>>>> > > You will be able to verify the signature by typing
>>>>> > >
>>>>> > > gpg --verify apache-impala-incubating-2.7.0-rc1.tar.gz.asc
>>>>> > > apache-impala-incubating-2.7.0-rc1.tar.gz
>>>>> > >
>>>>> > > This should happen on a machine you are the sole administrator of and
>>>>> > > that you have physical control of:
>>>>> > >
>>>>> > > http://www.apache.org/dev/release.html#owned-controlled-hardware
>>>>> > >
>>>>> > > 2. Build and test. You can do the testing on another machine - for
>>>>> > > instance, you can upload the RC1 tree to a git repo and point your CI
>>>>> > > tool at that repo.
>>>>> > >
>>>>> > > 3. "verify[ing] that the package meets the requirements of the ASF
>>>>> > > policy on releases"
>>>>> > >
>>>>> > > I suppose that means http://www.apache.org/dev/release.html. I am
>>>>> > > asking for clarification on that from our incubating mentors.
>>>>> > >
>>>>> > > Thank you!
>>>>> > >
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Henry Robinson
>>>>> > Software Engineer
>>>>> > Cloudera
>>>>> > 415-994-6679
>>>>> >
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Todd Lipcon
>>>> Software Engineer, Cloudera
Re: Prepare to vote on release candidate 1
Posted by Tim Armstrong <ta...@cloudera.com>.
One thing is to make sure that you don't have any environment variables set
pointing to a different directory. E.g. I have IMPALA_HOME in my
environment.
On Fri, Sep 9, 2016 at 2:09 AM, Lars Volker <lv...@cloudera.com> wrote:
> I unpacked the tarball, added all files to a fresh git repo and ran
> buildall.sh. It terminated with the following error:
>
> CMake Error at be/src/util/CMakeLists.txt:89 (add_library):
> Cannot find source file:
>
> /home/lv/asf/incubator-impala/be/src/thirdparty/squeasel/squeasel.c
>
> Can we add the steps you used to build and run the tests to the README
> file, so it's easier to validate?
>
> On Fri, Sep 9, 2016 at 11:04 AM, Tom White <to...@cloudera.com> wrote:
>
> > On Thu, Sep 8, 2016 at 5:27 PM, Jim Apple <jb...@cloudera.com> wrote:
> > > Sure, I can tag the tree where I git archive'd my next RC. Just to be
> > > sure I understand you - what exactly do you mean by the "git SHA"?
> >
> > I just mean git hash.
> >
> > >
> > > On Thu, Sep 8, 2016 at 2:24 AM, Tom White <to...@cloudera.com> wrote:
> > >> Jim,
> > >>
> > >> Can you provide a git tag and git SHA for the release when you send
> > >> out the vote so we can check the tarball against the tag.
> > >>
> > >> Thanks!
> > >> Tom
> > >>
> > >> On Thu, Sep 8, 2016 at 1:37 AM, Todd Lipcon <to...@cloudera.com>
> wrote:
> > >>> - agree with Tim that it would be nice if the tarball extracted into
> a
> > >>> directory named apache-impala-incubating-2.7.0 (to match the tarball
> > prefix)
> > >>>
> > >>> - A couple places seem to keep a Cloudera copyright/license - eg
> > >>> common/thrift/beeswax.thrift, FrontEndTestBase.java, etc. (I grepped
> > for
> > >>> Cloudera and found these). Should probably check over these and fix
> > before
> > >>> the "real" RC.
> > >>>
> > >>> - I tried building using 'buildall.sh' but it failed with 'fatal: Not
> > a git
> > >>> repository' when trying to do a git clean. Maybe some tweaks need to
> be
> > >>> made so that Impala can be built from a tarball? I worked around it
> > using
> > >>> 'git init' inside my extracted directory.
> > >>>
> > >>> - would be nice if the version number didn't have 'cdh5' in it (eg
> > >>> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/
> version.info,
> > >>> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
> > >>>
> > >>> - can't seem to use testdata/bin/run-all.sh to start DFS. I get:
> > >>> Starting hdfs (Web UI - http://localhost:5070)
> > >>> Failed to start hdfs-datanode. The end of the log
> > >>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-2/var/log/
> > hdfs-datanode.out)
> > >>> is:
> > >>> Failed to start hdfs-datanode. The end of the log
> > >>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-3/var/log/
> > hdfs-datanode.out)
> > >>> is:
> > >>> Failed to start hdfs-datanode. The end of the log
> > >>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-1/var/log/
> > hdfs-datanode.out)
> > >>> is:
> > >>>
> > >>> That said, it appears that impalad built OK (with build-all.sh
> > -notests)
> > >>>
> > >>>
> > >>> some nits/suggestions (ok to address in a later release:
> > >>> - a few mentions of 'Cloudera Impala' in the various pom files
> > >>> - would be great if buildall.sh could check if there is enough
> > remaining
> > >>> space on the drive before building. I had 20GB free but still ran out
> > of
> > >>> space trying to do the default build (had to rebuild with -notests)
> > >>> - consider setting up a cloudfront distribution in front of the
> > >>> native-toolchain S3 bucket? It downloads pretty slowly for me
> > >>> -- related: consider stripping debug info from some of the built
> deps?
> > eg
> > >>> Kudu is 500+MB which seems unnecessary for a test dependency.
> > >>>
> > >>>
> > >>> On Wed, Sep 7, 2016 at 5:01 PM, Tim Armstrong <
> tarmstrong@cloudera.com
> > >
> > >>> wrote:
> > >>>
> > >>>> I'm in the process of testing this.
> > >>>>
> > >>>> One nit that I think we should fix: apache-impala-incubating-2.7.
> > >>>> 0-rc1.tar.gz
> > >>>> unpacks to incubator-impala. I think we should stick with the normal
> > >>>> convention of unpacking to a directory with the same name as the
> > tarball
> > >>>> (or maybe apache-impala-incubating-2.7.0).
> > >>>>
> > >>>> On Wed, Sep 7, 2016 at 2:16 PM, Henry Robinson <he...@cloudera.com>
> > wrote:
> > >>>>
> > >>>> > Jim, this is great work (by everyone) and the culmination of a ton
> > of
> > >>>> > effort. Thanks for getting us to this stage!
> > >>>> >
> > >>>> > On 7 September 2016 at 14:08, Jim Apple <jb...@cloudera.com>
> > wrote:
> > >>>> >
> > >>>> > > I have put a release candidate, along with checksums and a
> > >>>> > > cryptographic signature, in
> > >>>> > >
> > >>>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
> > >>>> > >
> > >>>> > > I will be calling for a vote from the PPMC soon. This thread is
> > not
> > >>>> > > the vote thread.
> > >>>> > >
> > >>>> > > That vote will only pass, according to our bylaws, if it has 3
> > binding
> > >>>> > > +1 votes and more binding +1 votes than -1 votes. Only the votes
> > of
> > >>>> > > PPMC members are binding, but anyone may vote.
> > >>>> > >
> > >>>> > > If that vote passes, I will ask the Incubator PMC to approve the
> > >>>> > > release candidate, following the rules on
> > >>>> > >
> > >>>> > > http://incubator.apache.org/incubation/Incubation_Policy.htm
> > l#Releases
> > >>>> > >
> > >>>> > > To +1 a release candidate, you will need to verify it. This
> > includes:
> > >>>> > >
> > >>>> > > 1. Verifying the signature. You can import my code-signing
> public
> > key
> > >>>> > > from gpg using the instructions in
> > >>>> > >
> > >>>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
> > >>>> > >
> > >>>> > > You can also find that key at
> > >>>> > >
> > >>>> > > https://pgp.mit.edu/pks/lookup?op=get&search=0x91EE43066850196C
> > >>>> > >
> > >>>> > > or
> > >>>> > >
> > >>>> > > http://home.apache.org/keys/committer/jbapple
> > >>>> > >
> > >>>> > > You will be able to verify the signature by typing
> > >>>> > >
> > >>>> > > gpg --verify apache-impala-incubating-2.7.0-rc1.tar.gz.asc
> > >>>> > > apache-impala-incubating-2.7.0-rc1.tar.gz
> > >>>> > >
> > >>>> > > This should happen on a machine you are the sole administrator
> of
> > and
> > >>>> > > that you have physical control of:
> > >>>> > >
> > >>>> > > http://www.apache.org/dev/release.html#owned-controlled-
> hardware
> > >>>> > >
> > >>>> > > 2. Build and test. You can do the testing on another machine -
> for
> > >>>> > > instance, you can upload the RC1 tree to a git repo and point
> > your CI
> > >>>> > > tool at that repo.
> > >>>> > >
> > >>>> > > 3. "verify[ing] that the package meets the requirements of the
> ASF
> > >>>> > > policy on releases"
> > >>>> > >
> > >>>> > > I suppose that means http://www.apache.org/dev/release.html. I
> am
> > >>>> > > asking for clarification on that from our incubating mentors.
> > >>>> > >
> > >>>> > > Thank you!
> > >>>> > >
> > >>>> >
> > >>>> >
> > >>>> >
> > >>>> > --
> > >>>> > Henry Robinson
> > >>>> > Software Engineer
> > >>>> > Cloudera
> > >>>> > 415-994-6679
> > >>>> >
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Todd Lipcon
> > >>> Software Engineer, Cloudera
> >
>
Re: Prepare to vote on release candidate 1
Posted by Lars Volker <lv...@cloudera.com>.
I unpacked the tarball, added all files to a fresh git repo and ran
buildall.sh. It terminated with the following error:
CMake Error at be/src/util/CMakeLists.txt:89 (add_library):
Cannot find source file:
/home/lv/asf/incubator-impala/be/src/thirdparty/squeasel/squeasel.c
Can we add the steps you used to build and run the tests to the README
file, so it's easier to validate?
On Fri, Sep 9, 2016 at 11:04 AM, Tom White <to...@cloudera.com> wrote:
> On Thu, Sep 8, 2016 at 5:27 PM, Jim Apple <jb...@cloudera.com> wrote:
> > Sure, I can tag the tree where I git archive'd my next RC. Just to be
> > sure I understand you - what exactly do you mean by the "git SHA"?
>
> I just mean git hash.
>
> >
> > On Thu, Sep 8, 2016 at 2:24 AM, Tom White <to...@cloudera.com> wrote:
> >> Jim,
> >>
> >> Can you provide a git tag and git SHA for the release when you send
> >> out the vote so we can check the tarball against the tag.
> >>
> >> Thanks!
> >> Tom
> >>
> >> On Thu, Sep 8, 2016 at 1:37 AM, Todd Lipcon <to...@cloudera.com> wrote:
> >>> - agree with Tim that it would be nice if the tarball extracted into a
> >>> directory named apache-impala-incubating-2.7.0 (to match the tarball
> prefix)
> >>>
> >>> - A couple places seem to keep a Cloudera copyright/license - eg
> >>> common/thrift/beeswax.thrift, FrontEndTestBase.java, etc. (I grepped
> for
> >>> Cloudera and found these). Should probably check over these and fix
> before
> >>> the "real" RC.
> >>>
> >>> - I tried building using 'buildall.sh' but it failed with 'fatal: Not
> a git
> >>> repository' when trying to do a git clean. Maybe some tweaks need to be
> >>> made so that Impala can be built from a tarball? I worked around it
> using
> >>> 'git init' inside my extracted directory.
> >>>
> >>> - would be nice if the version number didn't have 'cdh5' in it (eg
> >>> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
> >>> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
> >>>
> >>> - can't seem to use testdata/bin/run-all.sh to start DFS. I get:
> >>> Starting hdfs (Web UI - http://localhost:5070)
> >>> Failed to start hdfs-datanode. The end of the log
> >>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-2/var/log/
> hdfs-datanode.out)
> >>> is:
> >>> Failed to start hdfs-datanode. The end of the log
> >>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-3/var/log/
> hdfs-datanode.out)
> >>> is:
> >>> Failed to start hdfs-datanode. The end of the log
> >>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-1/var/log/
> hdfs-datanode.out)
> >>> is:
> >>>
> >>> That said, it appears that impalad built OK (with build-all.sh
> -notests)
> >>>
> >>>
> >>> some nits/suggestions (ok to address in a later release:
> >>> - a few mentions of 'Cloudera Impala' in the various pom files
> >>> - would be great if buildall.sh could check if there is enough
> remaining
> >>> space on the drive before building. I had 20GB free but still ran out
> of
> >>> space trying to do the default build (had to rebuild with -notests)
> >>> - consider setting up a cloudfront distribution in front of the
> >>> native-toolchain S3 bucket? It downloads pretty slowly for me
> >>> -- related: consider stripping debug info from some of the built deps?
> eg
> >>> Kudu is 500+MB which seems unnecessary for a test dependency.
> >>>
> >>>
> >>> On Wed, Sep 7, 2016 at 5:01 PM, Tim Armstrong <tarmstrong@cloudera.com
> >
> >>> wrote:
> >>>
> >>>> I'm in the process of testing this.
> >>>>
> >>>> One nit that I think we should fix: apache-impala-incubating-2.7.
> >>>> 0-rc1.tar.gz
> >>>> unpacks to incubator-impala. I think we should stick with the normal
> >>>> convention of unpacking to a directory with the same name as the
> tarball
> >>>> (or maybe apache-impala-incubating-2.7.0).
> >>>>
> >>>> On Wed, Sep 7, 2016 at 2:16 PM, Henry Robinson <he...@cloudera.com>
> wrote:
> >>>>
> >>>> > Jim, this is great work (by everyone) and the culmination of a ton
> of
> >>>> > effort. Thanks for getting us to this stage!
> >>>> >
> >>>> > On 7 September 2016 at 14:08, Jim Apple <jb...@cloudera.com>
> wrote:
> >>>> >
> >>>> > > I have put a release candidate, along with checksums and a
> >>>> > > cryptographic signature, in
> >>>> > >
> >>>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
> >>>> > >
> >>>> > > I will be calling for a vote from the PPMC soon. This thread is
> not
> >>>> > > the vote thread.
> >>>> > >
> >>>> > > That vote will only pass, according to our bylaws, if it has 3
> binding
> >>>> > > +1 votes and more binding +1 votes than -1 votes. Only the votes
> of
> >>>> > > PPMC members are binding, but anyone may vote.
> >>>> > >
> >>>> > > If that vote passes, I will ask the Incubator PMC to approve the
> >>>> > > release candidate, following the rules on
> >>>> > >
> >>>> > > http://incubator.apache.org/incubation/Incubation_Policy.htm
> l#Releases
> >>>> > >
> >>>> > > To +1 a release candidate, you will need to verify it. This
> includes:
> >>>> > >
> >>>> > > 1. Verifying the signature. You can import my code-signing public
> key
> >>>> > > from gpg using the instructions in
> >>>> > >
> >>>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
> >>>> > >
> >>>> > > You can also find that key at
> >>>> > >
> >>>> > > https://pgp.mit.edu/pks/lookup?op=get&search=0x91EE43066850196C
> >>>> > >
> >>>> > > or
> >>>> > >
> >>>> > > http://home.apache.org/keys/committer/jbapple
> >>>> > >
> >>>> > > You will be able to verify the signature by typing
> >>>> > >
> >>>> > > gpg --verify apache-impala-incubating-2.7.0-rc1.tar.gz.asc
> >>>> > > apache-impala-incubating-2.7.0-rc1.tar.gz
> >>>> > >
> >>>> > > This should happen on a machine you are the sole administrator of
> and
> >>>> > > that you have physical control of:
> >>>> > >
> >>>> > > http://www.apache.org/dev/release.html#owned-controlled-hardware
> >>>> > >
> >>>> > > 2. Build and test. You can do the testing on another machine - for
> >>>> > > instance, you can upload the RC1 tree to a git repo and point
> your CI
> >>>> > > tool at that repo.
> >>>> > >
> >>>> > > 3. "verify[ing] that the package meets the requirements of the ASF
> >>>> > > policy on releases"
> >>>> > >
> >>>> > > I suppose that means http://www.apache.org/dev/release.html. I am
> >>>> > > asking for clarification on that from our incubating mentors.
> >>>> > >
> >>>> > > Thank you!
> >>>> > >
> >>>> >
> >>>> >
> >>>> >
> >>>> > --
> >>>> > Henry Robinson
> >>>> > Software Engineer
> >>>> > Cloudera
> >>>> > 415-994-6679
> >>>> >
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Todd Lipcon
> >>> Software Engineer, Cloudera
>
Re: Prepare to vote on release candidate 1
Posted by Tom White <to...@cloudera.com>.
On Thu, Sep 8, 2016 at 5:27 PM, Jim Apple <jb...@cloudera.com> wrote:
> Sure, I can tag the tree where I git archive'd my next RC. Just to be
> sure I understand you - what exactly do you mean by the "git SHA"?
I just mean git hash.
>
> On Thu, Sep 8, 2016 at 2:24 AM, Tom White <to...@cloudera.com> wrote:
>> Jim,
>>
>> Can you provide a git tag and git SHA for the release when you send
>> out the vote so we can check the tarball against the tag.
>>
>> Thanks!
>> Tom
>>
>> On Thu, Sep 8, 2016 at 1:37 AM, Todd Lipcon <to...@cloudera.com> wrote:
>>> - agree with Tim that it would be nice if the tarball extracted into a
>>> directory named apache-impala-incubating-2.7.0 (to match the tarball prefix)
>>>
>>> - A couple places seem to keep a Cloudera copyright/license - eg
>>> common/thrift/beeswax.thrift, FrontEndTestBase.java, etc. (I grepped for
>>> Cloudera and found these). Should probably check over these and fix before
>>> the "real" RC.
>>>
>>> - I tried building using 'buildall.sh' but it failed with 'fatal: Not a git
>>> repository' when trying to do a git clean. Maybe some tweaks need to be
>>> made so that Impala can be built from a tarball? I worked around it using
>>> 'git init' inside my extracted directory.
>>>
>>> - would be nice if the version number didn't have 'cdh5' in it (eg
>>> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
>>> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
>>>
>>> - can't seem to use testdata/bin/run-all.sh to start DFS. I get:
>>> Starting hdfs (Web UI - http://localhost:5070)
>>> Failed to start hdfs-datanode. The end of the log
>>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-2/var/log/hdfs-datanode.out)
>>> is:
>>> Failed to start hdfs-datanode. The end of the log
>>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-3/var/log/hdfs-datanode.out)
>>> is:
>>> Failed to start hdfs-datanode. The end of the log
>>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-1/var/log/hdfs-datanode.out)
>>> is:
>>>
>>> That said, it appears that impalad built OK (with build-all.sh -notests)
>>>
>>>
>>> some nits/suggestions (ok to address in a later release:
>>> - a few mentions of 'Cloudera Impala' in the various pom files
>>> - would be great if buildall.sh could check if there is enough remaining
>>> space on the drive before building. I had 20GB free but still ran out of
>>> space trying to do the default build (had to rebuild with -notests)
>>> - consider setting up a cloudfront distribution in front of the
>>> native-toolchain S3 bucket? It downloads pretty slowly for me
>>> -- related: consider stripping debug info from some of the built deps? eg
>>> Kudu is 500+MB which seems unnecessary for a test dependency.
>>>
>>>
>>> On Wed, Sep 7, 2016 at 5:01 PM, Tim Armstrong <ta...@cloudera.com>
>>> wrote:
>>>
>>>> I'm in the process of testing this.
>>>>
>>>> One nit that I think we should fix: apache-impala-incubating-2.7.
>>>> 0-rc1.tar.gz
>>>> unpacks to incubator-impala. I think we should stick with the normal
>>>> convention of unpacking to a directory with the same name as the tarball
>>>> (or maybe apache-impala-incubating-2.7.0).
>>>>
>>>> On Wed, Sep 7, 2016 at 2:16 PM, Henry Robinson <he...@cloudera.com> wrote:
>>>>
>>>> > Jim, this is great work (by everyone) and the culmination of a ton of
>>>> > effort. Thanks for getting us to this stage!
>>>> >
>>>> > On 7 September 2016 at 14:08, Jim Apple <jb...@cloudera.com> wrote:
>>>> >
>>>> > > I have put a release candidate, along with checksums and a
>>>> > > cryptographic signature, in
>>>> > >
>>>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
>>>> > >
>>>> > > I will be calling for a vote from the PPMC soon. This thread is not
>>>> > > the vote thread.
>>>> > >
>>>> > > That vote will only pass, according to our bylaws, if it has 3 binding
>>>> > > +1 votes and more binding +1 votes than -1 votes. Only the votes of
>>>> > > PPMC members are binding, but anyone may vote.
>>>> > >
>>>> > > If that vote passes, I will ask the Incubator PMC to approve the
>>>> > > release candidate, following the rules on
>>>> > >
>>>> > > http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
>>>> > >
>>>> > > To +1 a release candidate, you will need to verify it. This includes:
>>>> > >
>>>> > > 1. Verifying the signature. You can import my code-signing public key
>>>> > > from gpg using the instructions in
>>>> > >
>>>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
>>>> > >
>>>> > > You can also find that key at
>>>> > >
>>>> > > https://pgp.mit.edu/pks/lookup?op=get&search=0x91EE43066850196C
>>>> > >
>>>> > > or
>>>> > >
>>>> > > http://home.apache.org/keys/committer/jbapple
>>>> > >
>>>> > > You will be able to verify the signature by typing
>>>> > >
>>>> > > gpg --verify apache-impala-incubating-2.7.0-rc1.tar.gz.asc
>>>> > > apache-impala-incubating-2.7.0-rc1.tar.gz
>>>> > >
>>>> > > This should happen on a machine you are the sole administrator of and
>>>> > > that you have physical control of:
>>>> > >
>>>> > > http://www.apache.org/dev/release.html#owned-controlled-hardware
>>>> > >
>>>> > > 2. Build and test. You can do the testing on another machine - for
>>>> > > instance, you can upload the RC1 tree to a git repo and point your CI
>>>> > > tool at that repo.
>>>> > >
>>>> > > 3. "verify[ing] that the package meets the requirements of the ASF
>>>> > > policy on releases"
>>>> > >
>>>> > > I suppose that means http://www.apache.org/dev/release.html. I am
>>>> > > asking for clarification on that from our incubating mentors.
>>>> > >
>>>> > > Thank you!
>>>> > >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Henry Robinson
>>>> > Software Engineer
>>>> > Cloudera
>>>> > 415-994-6679
>>>> >
>>>>
>>>
>>>
>>>
>>> --
>>> Todd Lipcon
>>> Software Engineer, Cloudera
Re: Prepare to vote on release candidate 1
Posted by Jim Apple <jb...@cloudera.com>.
Sure, I can tag the tree where I git archive'd my next RC. Just to be
sure I understand you - what exactly do you mean by the "git SHA"?
On Thu, Sep 8, 2016 at 2:24 AM, Tom White <to...@cloudera.com> wrote:
> Jim,
>
> Can you provide a git tag and git SHA for the release when you send
> out the vote so we can check the tarball against the tag.
>
> Thanks!
> Tom
>
> On Thu, Sep 8, 2016 at 1:37 AM, Todd Lipcon <to...@cloudera.com> wrote:
>> - agree with Tim that it would be nice if the tarball extracted into a
>> directory named apache-impala-incubating-2.7.0 (to match the tarball prefix)
>>
>> - A couple places seem to keep a Cloudera copyright/license - eg
>> common/thrift/beeswax.thrift, FrontEndTestBase.java, etc. (I grepped for
>> Cloudera and found these). Should probably check over these and fix before
>> the "real" RC.
>>
>> - I tried building using 'buildall.sh' but it failed with 'fatal: Not a git
>> repository' when trying to do a git clean. Maybe some tweaks need to be
>> made so that Impala can be built from a tarball? I worked around it using
>> 'git init' inside my extracted directory.
>>
>> - would be nice if the version number didn't have 'cdh5' in it (eg
>> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
>> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
>>
>> - can't seem to use testdata/bin/run-all.sh to start DFS. I get:
>> Starting hdfs (Web UI - http://localhost:5070)
>> Failed to start hdfs-datanode. The end of the log
>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-2/var/log/hdfs-datanode.out)
>> is:
>> Failed to start hdfs-datanode. The end of the log
>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-3/var/log/hdfs-datanode.out)
>> is:
>> Failed to start hdfs-datanode. The end of the log
>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-1/var/log/hdfs-datanode.out)
>> is:
>>
>> That said, it appears that impalad built OK (with build-all.sh -notests)
>>
>>
>> some nits/suggestions (ok to address in a later release:
>> - a few mentions of 'Cloudera Impala' in the various pom files
>> - would be great if buildall.sh could check if there is enough remaining
>> space on the drive before building. I had 20GB free but still ran out of
>> space trying to do the default build (had to rebuild with -notests)
>> - consider setting up a cloudfront distribution in front of the
>> native-toolchain S3 bucket? It downloads pretty slowly for me
>> -- related: consider stripping debug info from some of the built deps? eg
>> Kudu is 500+MB which seems unnecessary for a test dependency.
>>
>>
>> On Wed, Sep 7, 2016 at 5:01 PM, Tim Armstrong <ta...@cloudera.com>
>> wrote:
>>
>>> I'm in the process of testing this.
>>>
>>> One nit that I think we should fix: apache-impala-incubating-2.7.
>>> 0-rc1.tar.gz
>>> unpacks to incubator-impala. I think we should stick with the normal
>>> convention of unpacking to a directory with the same name as the tarball
>>> (or maybe apache-impala-incubating-2.7.0).
>>>
>>> On Wed, Sep 7, 2016 at 2:16 PM, Henry Robinson <he...@cloudera.com> wrote:
>>>
>>> > Jim, this is great work (by everyone) and the culmination of a ton of
>>> > effort. Thanks for getting us to this stage!
>>> >
>>> > On 7 September 2016 at 14:08, Jim Apple <jb...@cloudera.com> wrote:
>>> >
>>> > > I have put a release candidate, along with checksums and a
>>> > > cryptographic signature, in
>>> > >
>>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
>>> > >
>>> > > I will be calling for a vote from the PPMC soon. This thread is not
>>> > > the vote thread.
>>> > >
>>> > > That vote will only pass, according to our bylaws, if it has 3 binding
>>> > > +1 votes and more binding +1 votes than -1 votes. Only the votes of
>>> > > PPMC members are binding, but anyone may vote.
>>> > >
>>> > > If that vote passes, I will ask the Incubator PMC to approve the
>>> > > release candidate, following the rules on
>>> > >
>>> > > http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
>>> > >
>>> > > To +1 a release candidate, you will need to verify it. This includes:
>>> > >
>>> > > 1. Verifying the signature. You can import my code-signing public key
>>> > > from gpg using the instructions in
>>> > >
>>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
>>> > >
>>> > > You can also find that key at
>>> > >
>>> > > https://pgp.mit.edu/pks/lookup?op=get&search=0x91EE43066850196C
>>> > >
>>> > > or
>>> > >
>>> > > http://home.apache.org/keys/committer/jbapple
>>> > >
>>> > > You will be able to verify the signature by typing
>>> > >
>>> > > gpg --verify apache-impala-incubating-2.7.0-rc1.tar.gz.asc
>>> > > apache-impala-incubating-2.7.0-rc1.tar.gz
>>> > >
>>> > > This should happen on a machine you are the sole administrator of and
>>> > > that you have physical control of:
>>> > >
>>> > > http://www.apache.org/dev/release.html#owned-controlled-hardware
>>> > >
>>> > > 2. Build and test. You can do the testing on another machine - for
>>> > > instance, you can upload the RC1 tree to a git repo and point your CI
>>> > > tool at that repo.
>>> > >
>>> > > 3. "verify[ing] that the package meets the requirements of the ASF
>>> > > policy on releases"
>>> > >
>>> > > I suppose that means http://www.apache.org/dev/release.html. I am
>>> > > asking for clarification on that from our incubating mentors.
>>> > >
>>> > > Thank you!
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > Henry Robinson
>>> > Software Engineer
>>> > Cloudera
>>> > 415-994-6679
>>> >
>>>
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
Re: Prepare to vote on release candidate 1
Posted by Tom White <to...@cloudera.com>.
Jim,
Can you provide a git tag and git SHA for the release when you send
out the vote so we can check the tarball against the tag.
Thanks!
Tom
On Thu, Sep 8, 2016 at 1:37 AM, Todd Lipcon <to...@cloudera.com> wrote:
> - agree with Tim that it would be nice if the tarball extracted into a
> directory named apache-impala-incubating-2.7.0 (to match the tarball prefix)
>
> - A couple places seem to keep a Cloudera copyright/license - eg
> common/thrift/beeswax.thrift, FrontEndTestBase.java, etc. (I grepped for
> Cloudera and found these). Should probably check over these and fix before
> the "real" RC.
>
> - I tried building using 'buildall.sh' but it failed with 'fatal: Not a git
> repository' when trying to do a git clean. Maybe some tweaks need to be
> made so that Impala can be built from a tarball? I worked around it using
> 'git init' inside my extracted directory.
>
> - would be nice if the version number didn't have 'cdh5' in it (eg
> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
>
> - can't seem to use testdata/bin/run-all.sh to start DFS. I get:
> Starting hdfs (Web UI - http://localhost:5070)
> Failed to start hdfs-datanode. The end of the log
> (/tmp/incubator-impala/testdata/cluster/cdh5/node-2/var/log/hdfs-datanode.out)
> is:
> Failed to start hdfs-datanode. The end of the log
> (/tmp/incubator-impala/testdata/cluster/cdh5/node-3/var/log/hdfs-datanode.out)
> is:
> Failed to start hdfs-datanode. The end of the log
> (/tmp/incubator-impala/testdata/cluster/cdh5/node-1/var/log/hdfs-datanode.out)
> is:
>
> That said, it appears that impalad built OK (with build-all.sh -notests)
>
>
> some nits/suggestions (ok to address in a later release:
> - a few mentions of 'Cloudera Impala' in the various pom files
> - would be great if buildall.sh could check if there is enough remaining
> space on the drive before building. I had 20GB free but still ran out of
> space trying to do the default build (had to rebuild with -notests)
> - consider setting up a cloudfront distribution in front of the
> native-toolchain S3 bucket? It downloads pretty slowly for me
> -- related: consider stripping debug info from some of the built deps? eg
> Kudu is 500+MB which seems unnecessary for a test dependency.
>
>
> On Wed, Sep 7, 2016 at 5:01 PM, Tim Armstrong <ta...@cloudera.com>
> wrote:
>
>> I'm in the process of testing this.
>>
>> One nit that I think we should fix: apache-impala-incubating-2.7.
>> 0-rc1.tar.gz
>> unpacks to incubator-impala. I think we should stick with the normal
>> convention of unpacking to a directory with the same name as the tarball
>> (or maybe apache-impala-incubating-2.7.0).
>>
>> On Wed, Sep 7, 2016 at 2:16 PM, Henry Robinson <he...@cloudera.com> wrote:
>>
>> > Jim, this is great work (by everyone) and the culmination of a ton of
>> > effort. Thanks for getting us to this stage!
>> >
>> > On 7 September 2016 at 14:08, Jim Apple <jb...@cloudera.com> wrote:
>> >
>> > > I have put a release candidate, along with checksums and a
>> > > cryptographic signature, in
>> > >
>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
>> > >
>> > > I will be calling for a vote from the PPMC soon. This thread is not
>> > > the vote thread.
>> > >
>> > > That vote will only pass, according to our bylaws, if it has 3 binding
>> > > +1 votes and more binding +1 votes than -1 votes. Only the votes of
>> > > PPMC members are binding, but anyone may vote.
>> > >
>> > > If that vote passes, I will ask the Incubator PMC to approve the
>> > > release candidate, following the rules on
>> > >
>> > > http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
>> > >
>> > > To +1 a release candidate, you will need to verify it. This includes:
>> > >
>> > > 1. Verifying the signature. You can import my code-signing public key
>> > > from gpg using the instructions in
>> > >
>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
>> > >
>> > > You can also find that key at
>> > >
>> > > https://pgp.mit.edu/pks/lookup?op=get&search=0x91EE43066850196C
>> > >
>> > > or
>> > >
>> > > http://home.apache.org/keys/committer/jbapple
>> > >
>> > > You will be able to verify the signature by typing
>> > >
>> > > gpg --verify apache-impala-incubating-2.7.0-rc1.tar.gz.asc
>> > > apache-impala-incubating-2.7.0-rc1.tar.gz
>> > >
>> > > This should happen on a machine you are the sole administrator of and
>> > > that you have physical control of:
>> > >
>> > > http://www.apache.org/dev/release.html#owned-controlled-hardware
>> > >
>> > > 2. Build and test. You can do the testing on another machine - for
>> > > instance, you can upload the RC1 tree to a git repo and point your CI
>> > > tool at that repo.
>> > >
>> > > 3. "verify[ing] that the package meets the requirements of the ASF
>> > > policy on releases"
>> > >
>> > > I suppose that means http://www.apache.org/dev/release.html. I am
>> > > asking for clarification on that from our incubating mentors.
>> > >
>> > > Thank you!
>> > >
>> >
>> >
>> >
>> > --
>> > Henry Robinson
>> > Software Engineer
>> > Cloudera
>> > 415-994-6679
>> >
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
Re: Prepare to vote on release candidate 1
Posted by Todd Lipcon <to...@cloudera.com>.
- agree with Tim that it would be nice if the tarball extracted into a
directory named apache-impala-incubating-2.7.0 (to match the tarball prefix)
- A couple places seem to keep a Cloudera copyright/license - eg
common/thrift/beeswax.thrift, FrontEndTestBase.java, etc. (I grepped for
Cloudera and found these). Should probably check over these and fix before
the "real" RC.
- I tried building using 'buildall.sh' but it failed with 'fatal: Not a git
repository' when trying to do a git clean. Maybe some tweaks need to be
made so that Impala can be built from a tarball? I worked around it using
'git init' inside my extracted directory.
- would be nice if the version number didn't have 'cdh5' in it (eg
impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
- can't seem to use testdata/bin/run-all.sh to start DFS. I get:
Starting hdfs (Web UI - http://localhost:5070)
Failed to start hdfs-datanode. The end of the log
(/tmp/incubator-impala/testdata/cluster/cdh5/node-2/var/log/hdfs-datanode.out)
is:
Failed to start hdfs-datanode. The end of the log
(/tmp/incubator-impala/testdata/cluster/cdh5/node-3/var/log/hdfs-datanode.out)
is:
Failed to start hdfs-datanode. The end of the log
(/tmp/incubator-impala/testdata/cluster/cdh5/node-1/var/log/hdfs-datanode.out)
is:
That said, it appears that impalad built OK (with build-all.sh -notests)
some nits/suggestions (ok to address in a later release:
- a few mentions of 'Cloudera Impala' in the various pom files
- would be great if buildall.sh could check if there is enough remaining
space on the drive before building. I had 20GB free but still ran out of
space trying to do the default build (had to rebuild with -notests)
- consider setting up a cloudfront distribution in front of the
native-toolchain S3 bucket? It downloads pretty slowly for me
-- related: consider stripping debug info from some of the built deps? eg
Kudu is 500+MB which seems unnecessary for a test dependency.
On Wed, Sep 7, 2016 at 5:01 PM, Tim Armstrong <ta...@cloudera.com>
wrote:
> I'm in the process of testing this.
>
> One nit that I think we should fix: apache-impala-incubating-2.7.
> 0-rc1.tar.gz
> unpacks to incubator-impala. I think we should stick with the normal
> convention of unpacking to a directory with the same name as the tarball
> (or maybe apache-impala-incubating-2.7.0).
>
> On Wed, Sep 7, 2016 at 2:16 PM, Henry Robinson <he...@cloudera.com> wrote:
>
> > Jim, this is great work (by everyone) and the culmination of a ton of
> > effort. Thanks for getting us to this stage!
> >
> > On 7 September 2016 at 14:08, Jim Apple <jb...@cloudera.com> wrote:
> >
> > > I have put a release candidate, along with checksums and a
> > > cryptographic signature, in
> > >
> > > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
> > >
> > > I will be calling for a vote from the PPMC soon. This thread is not
> > > the vote thread.
> > >
> > > That vote will only pass, according to our bylaws, if it has 3 binding
> > > +1 votes and more binding +1 votes than -1 votes. Only the votes of
> > > PPMC members are binding, but anyone may vote.
> > >
> > > If that vote passes, I will ask the Incubator PMC to approve the
> > > release candidate, following the rules on
> > >
> > > http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
> > >
> > > To +1 a release candidate, you will need to verify it. This includes:
> > >
> > > 1. Verifying the signature. You can import my code-signing public key
> > > from gpg using the instructions in
> > >
> > > https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
> > >
> > > You can also find that key at
> > >
> > > https://pgp.mit.edu/pks/lookup?op=get&search=0x91EE43066850196C
> > >
> > > or
> > >
> > > http://home.apache.org/keys/committer/jbapple
> > >
> > > You will be able to verify the signature by typing
> > >
> > > gpg --verify apache-impala-incubating-2.7.0-rc1.tar.gz.asc
> > > apache-impala-incubating-2.7.0-rc1.tar.gz
> > >
> > > This should happen on a machine you are the sole administrator of and
> > > that you have physical control of:
> > >
> > > http://www.apache.org/dev/release.html#owned-controlled-hardware
> > >
> > > 2. Build and test. You can do the testing on another machine - for
> > > instance, you can upload the RC1 tree to a git repo and point your CI
> > > tool at that repo.
> > >
> > > 3. "verify[ing] that the package meets the requirements of the ASF
> > > policy on releases"
> > >
> > > I suppose that means http://www.apache.org/dev/release.html. I am
> > > asking for clarification on that from our incubating mentors.
> > >
> > > Thank you!
> > >
> >
> >
> >
> > --
> > Henry Robinson
> > Software Engineer
> > Cloudera
> > 415-994-6679
> >
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: Prepare to vote on release candidate 1
Posted by Tim Armstrong <ta...@cloudera.com>.
I'm in the process of testing this.
One nit that I think we should fix: apache-impala-incubating-2.7.0-rc1.tar.gz
unpacks to incubator-impala. I think we should stick with the normal
convention of unpacking to a directory with the same name as the tarball
(or maybe apache-impala-incubating-2.7.0).
On Wed, Sep 7, 2016 at 2:16 PM, Henry Robinson <he...@cloudera.com> wrote:
> Jim, this is great work (by everyone) and the culmination of a ton of
> effort. Thanks for getting us to this stage!
>
> On 7 September 2016 at 14:08, Jim Apple <jb...@cloudera.com> wrote:
>
> > I have put a release candidate, along with checksums and a
> > cryptographic signature, in
> >
> > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
> >
> > I will be calling for a vote from the PPMC soon. This thread is not
> > the vote thread.
> >
> > That vote will only pass, according to our bylaws, if it has 3 binding
> > +1 votes and more binding +1 votes than -1 votes. Only the votes of
> > PPMC members are binding, but anyone may vote.
> >
> > If that vote passes, I will ask the Incubator PMC to approve the
> > release candidate, following the rules on
> >
> > http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
> >
> > To +1 a release candidate, you will need to verify it. This includes:
> >
> > 1. Verifying the signature. You can import my code-signing public key
> > from gpg using the instructions in
> >
> > https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
> >
> > You can also find that key at
> >
> > https://pgp.mit.edu/pks/lookup?op=get&search=0x91EE43066850196C
> >
> > or
> >
> > http://home.apache.org/keys/committer/jbapple
> >
> > You will be able to verify the signature by typing
> >
> > gpg --verify apache-impala-incubating-2.7.0-rc1.tar.gz.asc
> > apache-impala-incubating-2.7.0-rc1.tar.gz
> >
> > This should happen on a machine you are the sole administrator of and
> > that you have physical control of:
> >
> > http://www.apache.org/dev/release.html#owned-controlled-hardware
> >
> > 2. Build and test. You can do the testing on another machine - for
> > instance, you can upload the RC1 tree to a git repo and point your CI
> > tool at that repo.
> >
> > 3. "verify[ing] that the package meets the requirements of the ASF
> > policy on releases"
> >
> > I suppose that means http://www.apache.org/dev/release.html. I am
> > asking for clarification on that from our incubating mentors.
> >
> > Thank you!
> >
>
>
>
> --
> Henry Robinson
> Software Engineer
> Cloudera
> 415-994-6679
>
Re: Prepare to vote on release candidate 1
Posted by Henry Robinson <he...@cloudera.com>.
Jim, this is great work (by everyone) and the culmination of a ton of
effort. Thanks for getting us to this stage!
On 7 September 2016 at 14:08, Jim Apple <jb...@cloudera.com> wrote:
> I have put a release candidate, along with checksums and a
> cryptographic signature, in
>
> https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
>
> I will be calling for a vote from the PPMC soon. This thread is not
> the vote thread.
>
> That vote will only pass, according to our bylaws, if it has 3 binding
> +1 votes and more binding +1 votes than -1 votes. Only the votes of
> PPMC members are binding, but anyone may vote.
>
> If that vote passes, I will ask the Incubator PMC to approve the
> release candidate, following the rules on
>
> http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
>
> To +1 a release candidate, you will need to verify it. This includes:
>
> 1. Verifying the signature. You can import my code-signing public key
> from gpg using the instructions in
>
> https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
>
> You can also find that key at
>
> https://pgp.mit.edu/pks/lookup?op=get&search=0x91EE43066850196C
>
> or
>
> http://home.apache.org/keys/committer/jbapple
>
> You will be able to verify the signature by typing
>
> gpg --verify apache-impala-incubating-2.7.0-rc1.tar.gz.asc
> apache-impala-incubating-2.7.0-rc1.tar.gz
>
> This should happen on a machine you are the sole administrator of and
> that you have physical control of:
>
> http://www.apache.org/dev/release.html#owned-controlled-hardware
>
> 2. Build and test. You can do the testing on another machine - for
> instance, you can upload the RC1 tree to a git repo and point your CI
> tool at that repo.
>
> 3. "verify[ing] that the package meets the requirements of the ASF
> policy on releases"
>
> I suppose that means http://www.apache.org/dev/release.html. I am
> asking for clarification on that from our incubating mentors.
>
> Thank you!
>
--
Henry Robinson
Software Engineer
Cloudera
415-994-6679