You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@impala.apache.org by Matthew Jacobs <mj...@cloudera.com> on 2016/12/11 07:11:50 UTC
Important: new shell & bootstrap required after rebasing, again
Hi all,
We had to bump the Kudu version again so unfortunately you'll have to
refresh your environment again to get the latest Kudu. After fetching
the latest Impala commits (including commit 39017adf) and rebasing,
you'll need to:
1) start a new shell OR make sure to
export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
2) Run buildall.sh, that will:
a) download the new Kudu bits in the toolchain by calling
bootstrap_toolchain.py (or you can do so yourself)
b) restart the minicluster with the new kudu
From Lars:
If you don't want to mess with your pane layout in tmux you can run
tmux respawn-pane -k
to kill and replace the current pane with an entirely fresh shell.
I bind this to a shortcut in my tmux config like so:
bind K respawn-pane -k
Re: Important: new shell & bootstrap required after rebasing, again
Posted by David Knupp <dk...@cloudera.com>.
Ah. OK thanks all for the suggestions.
> Matthew Jacobs <ma...@cloudera.com>
> December 12, 2016 at 2:39 PM
> The build number and kudu version in that string look mismatched to me.
>
> I see build 308-96 with kudu a70.
>
> In AWS is:
> 308-96a4cc516e with Kudu e018a83
> s3://native-toolchain/build/308-96a4cc516e/kudu/e018a83-gcc-4.9.2/
> Matthew Jacobs <ma...@cloudera.com>
> December 11, 2016 at 9:07 AM
> Try just setting the toolchain build id manually:
> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>
> Anyway this is annoying, +1 to Tim's proposal for improved environment
> configs.
> Matthew Jacobs <ma...@cloudera.com>
> December 11, 2016 at 9:04 AM
> The URL is still referencing an old toolchain version 267
> (267-98c642ffcb).
>
> The new version is 308-96a4cc516e ; starting a new shell and sourcing
> bin/impala-config.sh shouldn't leave any opportunities for the old
> version to be in your environment. IMPALA_TOOLCHAIN_BUILD_ID must be
> getting set still.
> Marcel Kornacker <ma...@cloudera.com>
> December 11, 2016 at 8:58 AM
> I'm getting Kudu client download errors fairly reliably now. This is
> from a clean shell.
>
> Downloading kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz to
> /home/marcel/impala/toolchain
> Traceback (most recent call last):
> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 355, in
> <module>
> bootstrap(toolchain_root, packages)
> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 117, in
> bootstrap
> download_package(toolchain_root, pkg_name, pkg_version, compiler)
> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 99, in
> download_package
> wget_and_unpack_package(download_path, file_name, destination, True)
> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 81, in
> wget_and_unpack_package
> sh.wget(download_path, directory_prefix=destination,
> no_clobber=wget_no_clobber)
> File
> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
> line 1021, in __call__
> return RunningCommand(cmd, call_args, stdin, stdout, stderr)
> File
> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
> line 486, in __init__
> self.wait()
> File
> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
> line 500, in wait
> self.handle_command_exit_code(exit_code)
> File
> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
> line 516, in handle_command_exit_code
> raise exc(self.ran, self.process.stdout, self.process.stderr)
> sh.ErrorReturnCode_8:
>
> RAN: '/usr/bin/wget
> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
> --directory-prefix=/home/marcel/impala/toolchain --no-clobber'
>
> STDOUT:
>
>
> STDERR:
> --2016-12-11 08:56:17--
> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
> Resolving native-toolchain.s3.amazonaws.com
> (native-toolchain.s3.amazonaws.com)... 54.231.237.83
> Connecting to native-toolchain.s3.amazonaws.com
> (native-toolchain.s3.amazonaws.com)|54.231.237.83|:443... connected.
> HTTP request sent, awaiting response... 403 Forbidden
> 2016-12-11 08:56:17 ERROR 403: Forbidden.
> Matthew Jacobs <ma...@cloudera.com>
> December 10, 2016 at 11:11 PM
> Hi all,
>
>
> We had to bump the Kudu version again so unfortunately you'll have to
> refresh your environment again to get the latest Kudu. After fetching
> the latest Impala commits (including commit 39017adf) and rebasing,
> you'll need to:
>
>
> 1) start a new shell OR make sure to
> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>
>
> 2) Run buildall.sh, that will:
> a) download the new Kudu bits in the toolchain by calling
> bootstrap_toolchain.py (or you can do so yourself)
> b) restart the minicluster with the new kudu
>
>
>
> From Lars:
> If you don't want to mess with your pane layout in tmux you can run
> tmux respawn-pane -k
> to kill and replace the current pane with an entirely fresh shell.
> I bind this to a shortcut in my tmux config like so:
> bind K respawn-pane -k
--
David Knupp
Software Engineer
Cloudera
415-312-1049
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
Re: Important: new shell & bootstrap required after rebasing, again
Posted by Matthew Jacobs <mj...@cloudera.com>.
The build number and kudu version in that string look mismatched to me.
I see build 308-96 with kudu a70.
In AWS is:
308-96a4cc516e with Kudu e018a83
s3://native-toolchain/build/308-96a4cc516e/kudu/e018a83-gcc-4.9.2/
On Mon, Dec 12, 2016 at 2:34 PM, David Knupp <dk...@cloudera.com> wrote:
> I'm still having trouble building, although I have a slightly different use
> case. I need to build on a Cloudcat instance, so I first log in as root, but
> then su to user "systest" and build in ~/systest.
>
> Even though I've exported the new value for the toolchain build, this still
> occurs:
>
> --2016-12-12 14:26:09--
> https://native-toolchain.s3.amazonaws.com/build/308-96a4cc516e/kudu/a70c905006-gcc-4.9.2/kudu-a70c905006-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
> Resolving native-toolchain.s3.amazonaws.com
> (native-toolchain.s3.amazonaws.com)... 54.231.236.39
> Connecting to native-toolchain.s3.amazonaws.com
> (native-toolchain.s3.amazonaws.com)|54.231.236.39|:443... connected.
> HTTP request sent, awaiting response... 403 Forbidden
> 2016-12-12 14:26:09 ERROR 403: Forbidden.
>
>
> I'll keep poking at this, but just throwing it out in case someone has an
> idea off the top of their heads as to what the issue might be
>
>> Matthew Jacobs <ma...@cloudera.com>
>> December 11, 2016 at 9:07 AM
>> Try just setting the toolchain build id manually:
>> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>>
>> Anyway this is annoying, +1 to Tim's proposal for improved environment
>> configs.
>> Matthew Jacobs <ma...@cloudera.com>
>> December 11, 2016 at 9:04 AM
>> The URL is still referencing an old toolchain version 267
>> (267-98c642ffcb).
>>
>> The new version is 308-96a4cc516e ; starting a new shell and sourcing
>> bin/impala-config.sh shouldn't leave any opportunities for the old
>> version to be in your environment. IMPALA_TOOLCHAIN_BUILD_ID must be
>> getting set still.
>> Marcel Kornacker <ma...@cloudera.com>
>> December 11, 2016 at 8:58 AM
>>
>> I'm getting Kudu client download errors fairly reliably now. This is
>> from a clean shell.
>>
>> Downloading kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz to
>> /home/marcel/impala/toolchain
>> Traceback (most recent call last):
>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 355, in
>> <module>
>> bootstrap(toolchain_root, packages)
>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 117, in
>> bootstrap
>> download_package(toolchain_root, pkg_name, pkg_version, compiler)
>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 99, in
>> download_package
>> wget_and_unpack_package(download_path, file_name, destination, True)
>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 81, in
>> wget_and_unpack_package
>> sh.wget(download_path, directory_prefix=destination,
>> no_clobber=wget_no_clobber)
>> File
>> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>> line 1021, in __call__
>> return RunningCommand(cmd, call_args, stdin, stdout, stderr)
>> File
>> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>> line 486, in __init__
>> self.wait()
>> File
>> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>> line 500, in wait
>> self.handle_command_exit_code(exit_code)
>> File
>> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>> line 516, in handle_command_exit_code
>> raise exc(self.ran, self.process.stdout, self.process.stderr)
>> sh.ErrorReturnCode_8:
>>
>> RAN: '/usr/bin/wget
>>
>> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
>> --directory-prefix=/home/marcel/impala/toolchain --no-clobber'
>>
>> STDOUT:
>>
>>
>> STDERR:
>> --2016-12-11 08:56:17--
>>
>> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
>> Resolving native-toolchain.s3.amazonaws.com
>> (native-toolchain.s3.amazonaws.com)... 54.231.237.83
>> Connecting to native-toolchain.s3.amazonaws.com
>> (native-toolchain.s3.amazonaws.com)|54.231.237.83|:443... connected.
>> HTTP request sent, awaiting response... 403 Forbidden
>> 2016-12-11 08:56:17 ERROR 403: Forbidden.
>> Matthew Jacobs <ma...@cloudera.com>
>> December 10, 2016 at 11:11 PM
>> Hi all,
>>
>>
>> We had to bump the Kudu version again so unfortunately you'll have to
>> refresh your environment again to get the latest Kudu. After fetching
>> the latest Impala commits (including commit 39017adf) and rebasing,
>> you'll need to:
>>
>>
>> 1) start a new shell OR make sure to
>> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>>
>>
>> 2) Run buildall.sh, that will:
>> a) download the new Kudu bits in the toolchain by calling
>> bootstrap_toolchain.py (or you can do so yourself)
>> b) restart the minicluster with the new kudu
>>
>>
>>
>> From Lars:
>> If you don't want to mess with your pane layout in tmux you can run
>> tmux respawn-pane -k
>> to kill and replace the current pane with an entirely fresh shell.
>> I bind this to a shortcut in my tmux config like so:
>> bind K respawn-pane -k
>
>
> --
> David Knupp
> Software Engineer
> Cloudera
> 415-312-1049
> <https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
Re: Important: new shell & bootstrap required after rebasing, again
Posted by Dimitris Tsirogiannis <dt...@cloudera.com>.
Can you check if the issue is Kudu-related or not? For instance, does
it work if you set then env variable KUDU_IS_SUPPORTED to false?
Dimitris
On Mon, Dec 12, 2016 at 2:34 PM, David Knupp <dk...@cloudera.com> wrote:
> I'm still having trouble building, although I have a slightly different use
> case. I need to build on a Cloudcat instance, so I first log in as root, but
> then su to user "systest" and build in ~/systest.
>
> Even though I've exported the new value for the toolchain build, this still
> occurs:
>
> --2016-12-12 14:26:09--
> https://native-toolchain.s3.amazonaws.com/build/308-96a4cc516e/kudu/a70c905006-gcc-4.9.2/kudu-a70c905006-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
> Resolving native-toolchain.s3.amazonaws.com
> (native-toolchain.s3.amazonaws.com)... 54.231.236.39
> Connecting to native-toolchain.s3.amazonaws.com
> (native-toolchain.s3.amazonaws.com)|54.231.236.39|:443... connected.
> HTTP request sent, awaiting response... 403 Forbidden
> 2016-12-12 14:26:09 ERROR 403: Forbidden.
>
>
> I'll keep poking at this, but just throwing it out in case someone has an
> idea off the top of their heads as to what the issue might be
>
>> Matthew Jacobs <ma...@cloudera.com>
>> December 11, 2016 at 9:07 AM
>> Try just setting the toolchain build id manually:
>> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>>
>> Anyway this is annoying, +1 to Tim's proposal for improved environment
>> configs.
>> Matthew Jacobs <ma...@cloudera.com>
>> December 11, 2016 at 9:04 AM
>> The URL is still referencing an old toolchain version 267
>> (267-98c642ffcb).
>>
>> The new version is 308-96a4cc516e ; starting a new shell and sourcing
>> bin/impala-config.sh shouldn't leave any opportunities for the old
>> version to be in your environment. IMPALA_TOOLCHAIN_BUILD_ID must be
>> getting set still.
>> Marcel Kornacker <ma...@cloudera.com>
>> December 11, 2016 at 8:58 AM
>>
>> I'm getting Kudu client download errors fairly reliably now. This is
>> from a clean shell.
>>
>> Downloading kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz to
>> /home/marcel/impala/toolchain
>> Traceback (most recent call last):
>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 355, in
>> <module>
>> bootstrap(toolchain_root, packages)
>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 117, in
>> bootstrap
>> download_package(toolchain_root, pkg_name, pkg_version, compiler)
>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 99, in
>> download_package
>> wget_and_unpack_package(download_path, file_name, destination, True)
>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 81, in
>> wget_and_unpack_package
>> sh.wget(download_path, directory_prefix=destination,
>> no_clobber=wget_no_clobber)
>> File
>> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>> line 1021, in __call__
>> return RunningCommand(cmd, call_args, stdin, stdout, stderr)
>> File
>> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>> line 486, in __init__
>> self.wait()
>> File
>> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>> line 500, in wait
>> self.handle_command_exit_code(exit_code)
>> File
>> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>> line 516, in handle_command_exit_code
>> raise exc(self.ran, self.process.stdout, self.process.stderr)
>> sh.ErrorReturnCode_8:
>>
>> RAN: '/usr/bin/wget
>>
>> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
>> --directory-prefix=/home/marcel/impala/toolchain --no-clobber'
>>
>> STDOUT:
>>
>>
>> STDERR:
>> --2016-12-11 08:56:17--
>>
>> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
>> Resolving native-toolchain.s3.amazonaws.com
>> (native-toolchain.s3.amazonaws.com)... 54.231.237.83
>> Connecting to native-toolchain.s3.amazonaws.com
>> (native-toolchain.s3.amazonaws.com)|54.231.237.83|:443... connected.
>> HTTP request sent, awaiting response... 403 Forbidden
>> 2016-12-11 08:56:17 ERROR 403: Forbidden.
>> Matthew Jacobs <ma...@cloudera.com>
>> December 10, 2016 at 11:11 PM
>> Hi all,
>>
>>
>> We had to bump the Kudu version again so unfortunately you'll have to
>> refresh your environment again to get the latest Kudu. After fetching
>> the latest Impala commits (including commit 39017adf) and rebasing,
>> you'll need to:
>>
>>
>> 1) start a new shell OR make sure to
>> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>>
>>
>> 2) Run buildall.sh, that will:
>> a) download the new Kudu bits in the toolchain by calling
>> bootstrap_toolchain.py (or you can do so yourself)
>> b) restart the minicluster with the new kudu
>>
>>
>>
>> From Lars:
>> If you don't want to mess with your pane layout in tmux you can run
>> tmux respawn-pane -k
>> to kill and replace the current pane with an entirely fresh shell.
>> I bind this to a shortcut in my tmux config like so:
>> bind K respawn-pane -k
>
>
> --
> David Knupp
> Software Engineer
> Cloudera
> 415-312-1049
> <https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
Re: Important: new shell & bootstrap required after rebasing, again
Posted by David Knupp <dk...@cloudera.com>.
I'm still having trouble building, although I have a slightly different
use case. I need to build on a Cloudcat instance, so I first log in as
root, but then su to user "systest" and build in ~/systest.
Even though I've exported the new value for the toolchain build, this
still occurs:
--2016-12-12 14:26:09--
https://native-toolchain.s3.amazonaws.com/build/308-96a4cc516e/kudu/a70c905006-gcc-4.9.2/kudu-a70c905006-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
Resolving native-toolchain.s3.amazonaws.com
(native-toolchain.s3.amazonaws.com)... 54.231.236.39
Connecting to native-toolchain.s3.amazonaws.com
(native-toolchain.s3.amazonaws.com)|54.231.236.39|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2016-12-12 14:26:09 ERROR 403: Forbidden.
I'll keep poking at this, but just throwing it out in case someone has
an idea off the top of their heads as to what the issue might be
> Matthew Jacobs <ma...@cloudera.com>
> December 11, 2016 at 9:07 AM
> Try just setting the toolchain build id manually:
> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>
> Anyway this is annoying, +1 to Tim's proposal for improved environment
> configs.
> Matthew Jacobs <ma...@cloudera.com>
> December 11, 2016 at 9:04 AM
> The URL is still referencing an old toolchain version 267
> (267-98c642ffcb).
>
> The new version is 308-96a4cc516e ; starting a new shell and sourcing
> bin/impala-config.sh shouldn't leave any opportunities for the old
> version to be in your environment. IMPALA_TOOLCHAIN_BUILD_ID must be
> getting set still.
> Marcel Kornacker <ma...@cloudera.com>
> December 11, 2016 at 8:58 AM
> I'm getting Kudu client download errors fairly reliably now. This is
> from a clean shell.
>
> Downloading kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz to
> /home/marcel/impala/toolchain
> Traceback (most recent call last):
> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 355, in
> <module>
> bootstrap(toolchain_root, packages)
> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 117, in
> bootstrap
> download_package(toolchain_root, pkg_name, pkg_version, compiler)
> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 99, in
> download_package
> wget_and_unpack_package(download_path, file_name, destination, True)
> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 81, in
> wget_and_unpack_package
> sh.wget(download_path, directory_prefix=destination,
> no_clobber=wget_no_clobber)
> File
> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
> line 1021, in __call__
> return RunningCommand(cmd, call_args, stdin, stdout, stderr)
> File
> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
> line 486, in __init__
> self.wait()
> File
> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
> line 500, in wait
> self.handle_command_exit_code(exit_code)
> File
> "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
> line 516, in handle_command_exit_code
> raise exc(self.ran, self.process.stdout, self.process.stderr)
> sh.ErrorReturnCode_8:
>
> RAN: '/usr/bin/wget
> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
> --directory-prefix=/home/marcel/impala/toolchain --no-clobber'
>
> STDOUT:
>
>
> STDERR:
> --2016-12-11 08:56:17--
> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
> Resolving native-toolchain.s3.amazonaws.com
> (native-toolchain.s3.amazonaws.com)... 54.231.237.83
> Connecting to native-toolchain.s3.amazonaws.com
> (native-toolchain.s3.amazonaws.com)|54.231.237.83|:443... connected.
> HTTP request sent, awaiting response... 403 Forbidden
> 2016-12-11 08:56:17 ERROR 403: Forbidden.
> Matthew Jacobs <ma...@cloudera.com>
> December 10, 2016 at 11:11 PM
> Hi all,
>
>
> We had to bump the Kudu version again so unfortunately you'll have to
> refresh your environment again to get the latest Kudu. After fetching
> the latest Impala commits (including commit 39017adf) and rebasing,
> you'll need to:
>
>
> 1) start a new shell OR make sure to
> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>
>
> 2) Run buildall.sh, that will:
> a) download the new Kudu bits in the toolchain by calling
> bootstrap_toolchain.py (or you can do so yourself)
> b) restart the minicluster with the new kudu
>
>
>
> From Lars:
> If you don't want to mess with your pane layout in tmux you can run
> tmux respawn-pane -k
> to kill and replace the current pane with an entirely fresh shell.
> I bind this to a shortcut in my tmux config like so:
> bind K respawn-pane -k
--
David Knupp
Software Engineer
Cloudera
415-312-1049
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
Re: Important: new shell & bootstrap required after rebasing, again
Posted by Marcel Kornacker <ma...@cloudera.com>.
Sorry, my mistake.
On Sun, Dec 11, 2016 at 9:07 AM, Matthew Jacobs <mj...@cloudera.com> wrote:
> Try just setting the toolchain build id manually:
> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>
> Anyway this is annoying, +1 to Tim's proposal for improved environment configs.
>
> On Sun, Dec 11, 2016 at 9:04 AM, Matthew Jacobs <mj...@cloudera.com> wrote:
>> The URL is still referencing an old toolchain version 267 (267-98c642ffcb).
>>
>> The new version is 308-96a4cc516e ; starting a new shell and sourcing
>> bin/impala-config.sh shouldn't leave any opportunities for the old
>> version to be in your environment. IMPALA_TOOLCHAIN_BUILD_ID must be
>> getting set still.
>>
>> On Sun, Dec 11, 2016 at 8:58 AM, Marcel Kornacker <ma...@cloudera.com> wrote:
>>> I'm getting Kudu client download errors fairly reliably now. This is
>>> from a clean shell.
>>>
>>> Downloading kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz to
>>> /home/marcel/impala/toolchain
>>> Traceback (most recent call last):
>>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 355, in <module>
>>> bootstrap(toolchain_root, packages)
>>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 117, in bootstrap
>>> download_package(toolchain_root, pkg_name, pkg_version, compiler)
>>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 99, in
>>> download_package
>>> wget_and_unpack_package(download_path, file_name, destination, True)
>>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 81, in
>>> wget_and_unpack_package
>>> sh.wget(download_path, directory_prefix=destination,
>>> no_clobber=wget_no_clobber)
>>> File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>>> line 1021, in __call__
>>> return RunningCommand(cmd, call_args, stdin, stdout, stderr)
>>> File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>>> line 486, in __init__
>>> self.wait()
>>> File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>>> line 500, in wait
>>> self.handle_command_exit_code(exit_code)
>>> File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>>> line 516, in handle_command_exit_code
>>> raise exc(self.ran, self.process.stdout, self.process.stderr)
>>> sh.ErrorReturnCode_8:
>>>
>>> RAN: '/usr/bin/wget
>>> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
>>> --directory-prefix=/home/marcel/impala/toolchain --no-clobber'
>>>
>>> STDOUT:
>>>
>>>
>>> STDERR:
>>> --2016-12-11 08:56:17--
>>> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
>>> Resolving native-toolchain.s3.amazonaws.com
>>> (native-toolchain.s3.amazonaws.com)... 54.231.237.83
>>> Connecting to native-toolchain.s3.amazonaws.com
>>> (native-toolchain.s3.amazonaws.com)|54.231.237.83|:443... connected.
>>> HTTP request sent, awaiting response... 403 Forbidden
>>> 2016-12-11 08:56:17 ERROR 403: Forbidden.
>>>
>>> On Sat, Dec 10, 2016 at 11:11 PM, Matthew Jacobs <mj...@cloudera.com> wrote:
>>>> Hi all,
>>>>
>>>>
>>>> We had to bump the Kudu version again so unfortunately you'll have to
>>>> refresh your environment again to get the latest Kudu. After fetching
>>>> the latest Impala commits (including commit 39017adf) and rebasing,
>>>> you'll need to:
>>>>
>>>>
>>>> 1) start a new shell OR make sure to
>>>> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>>>>
>>>>
>>>> 2) Run buildall.sh, that will:
>>>> a) download the new Kudu bits in the toolchain by calling
>>>> bootstrap_toolchain.py (or you can do so yourself)
>>>> b) restart the minicluster with the new kudu
>>>>
>>>>
>>>>
>>>> From Lars:
>>>> If you don't want to mess with your pane layout in tmux you can run
>>>> tmux respawn-pane -k
>>>> to kill and replace the current pane with an entirely fresh shell.
>>>> I bind this to a shortcut in my tmux config like so:
>>>> bind K respawn-pane -k
Re: Important: new shell & bootstrap required after rebasing, again
Posted by Matthew Jacobs <mj...@cloudera.com>.
Try just setting the toolchain build id manually:
export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
Anyway this is annoying, +1 to Tim's proposal for improved environment configs.
On Sun, Dec 11, 2016 at 9:04 AM, Matthew Jacobs <mj...@cloudera.com> wrote:
> The URL is still referencing an old toolchain version 267 (267-98c642ffcb).
>
> The new version is 308-96a4cc516e ; starting a new shell and sourcing
> bin/impala-config.sh shouldn't leave any opportunities for the old
> version to be in your environment. IMPALA_TOOLCHAIN_BUILD_ID must be
> getting set still.
>
> On Sun, Dec 11, 2016 at 8:58 AM, Marcel Kornacker <ma...@cloudera.com> wrote:
>> I'm getting Kudu client download errors fairly reliably now. This is
>> from a clean shell.
>>
>> Downloading kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz to
>> /home/marcel/impala/toolchain
>> Traceback (most recent call last):
>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 355, in <module>
>> bootstrap(toolchain_root, packages)
>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 117, in bootstrap
>> download_package(toolchain_root, pkg_name, pkg_version, compiler)
>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 99, in
>> download_package
>> wget_and_unpack_package(download_path, file_name, destination, True)
>> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 81, in
>> wget_and_unpack_package
>> sh.wget(download_path, directory_prefix=destination,
>> no_clobber=wget_no_clobber)
>> File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>> line 1021, in __call__
>> return RunningCommand(cmd, call_args, stdin, stdout, stderr)
>> File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>> line 486, in __init__
>> self.wait()
>> File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>> line 500, in wait
>> self.handle_command_exit_code(exit_code)
>> File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
>> line 516, in handle_command_exit_code
>> raise exc(self.ran, self.process.stdout, self.process.stderr)
>> sh.ErrorReturnCode_8:
>>
>> RAN: '/usr/bin/wget
>> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
>> --directory-prefix=/home/marcel/impala/toolchain --no-clobber'
>>
>> STDOUT:
>>
>>
>> STDERR:
>> --2016-12-11 08:56:17--
>> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
>> Resolving native-toolchain.s3.amazonaws.com
>> (native-toolchain.s3.amazonaws.com)... 54.231.237.83
>> Connecting to native-toolchain.s3.amazonaws.com
>> (native-toolchain.s3.amazonaws.com)|54.231.237.83|:443... connected.
>> HTTP request sent, awaiting response... 403 Forbidden
>> 2016-12-11 08:56:17 ERROR 403: Forbidden.
>>
>> On Sat, Dec 10, 2016 at 11:11 PM, Matthew Jacobs <mj...@cloudera.com> wrote:
>>> Hi all,
>>>
>>>
>>> We had to bump the Kudu version again so unfortunately you'll have to
>>> refresh your environment again to get the latest Kudu. After fetching
>>> the latest Impala commits (including commit 39017adf) and rebasing,
>>> you'll need to:
>>>
>>>
>>> 1) start a new shell OR make sure to
>>> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>>>
>>>
>>> 2) Run buildall.sh, that will:
>>> a) download the new Kudu bits in the toolchain by calling
>>> bootstrap_toolchain.py (or you can do so yourself)
>>> b) restart the minicluster with the new kudu
>>>
>>>
>>>
>>> From Lars:
>>> If you don't want to mess with your pane layout in tmux you can run
>>> tmux respawn-pane -k
>>> to kill and replace the current pane with an entirely fresh shell.
>>> I bind this to a shortcut in my tmux config like so:
>>> bind K respawn-pane -k
Re: Important: new shell & bootstrap required after rebasing, again
Posted by Matthew Jacobs <mj...@cloudera.com>.
The URL is still referencing an old toolchain version 267 (267-98c642ffcb).
The new version is 308-96a4cc516e ; starting a new shell and sourcing
bin/impala-config.sh shouldn't leave any opportunities for the old
version to be in your environment. IMPALA_TOOLCHAIN_BUILD_ID must be
getting set still.
On Sun, Dec 11, 2016 at 8:58 AM, Marcel Kornacker <ma...@cloudera.com> wrote:
> I'm getting Kudu client download errors fairly reliably now. This is
> from a clean shell.
>
> Downloading kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz to
> /home/marcel/impala/toolchain
> Traceback (most recent call last):
> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 355, in <module>
> bootstrap(toolchain_root, packages)
> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 117, in bootstrap
> download_package(toolchain_root, pkg_name, pkg_version, compiler)
> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 99, in
> download_package
> wget_and_unpack_package(download_path, file_name, destination, True)
> File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 81, in
> wget_and_unpack_package
> sh.wget(download_path, directory_prefix=destination,
> no_clobber=wget_no_clobber)
> File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
> line 1021, in __call__
> return RunningCommand(cmd, call_args, stdin, stdout, stderr)
> File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
> line 486, in __init__
> self.wait()
> File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
> line 500, in wait
> self.handle_command_exit_code(exit_code)
> File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
> line 516, in handle_command_exit_code
> raise exc(self.ran, self.process.stdout, self.process.stderr)
> sh.ErrorReturnCode_8:
>
> RAN: '/usr/bin/wget
> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
> --directory-prefix=/home/marcel/impala/toolchain --no-clobber'
>
> STDOUT:
>
>
> STDERR:
> --2016-12-11 08:56:17--
> https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
> Resolving native-toolchain.s3.amazonaws.com
> (native-toolchain.s3.amazonaws.com)... 54.231.237.83
> Connecting to native-toolchain.s3.amazonaws.com
> (native-toolchain.s3.amazonaws.com)|54.231.237.83|:443... connected.
> HTTP request sent, awaiting response... 403 Forbidden
> 2016-12-11 08:56:17 ERROR 403: Forbidden.
>
> On Sat, Dec 10, 2016 at 11:11 PM, Matthew Jacobs <mj...@cloudera.com> wrote:
>> Hi all,
>>
>>
>> We had to bump the Kudu version again so unfortunately you'll have to
>> refresh your environment again to get the latest Kudu. After fetching
>> the latest Impala commits (including commit 39017adf) and rebasing,
>> you'll need to:
>>
>>
>> 1) start a new shell OR make sure to
>> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>>
>>
>> 2) Run buildall.sh, that will:
>> a) download the new Kudu bits in the toolchain by calling
>> bootstrap_toolchain.py (or you can do so yourself)
>> b) restart the minicluster with the new kudu
>>
>>
>>
>> From Lars:
>> If you don't want to mess with your pane layout in tmux you can run
>> tmux respawn-pane -k
>> to kill and replace the current pane with an entirely fresh shell.
>> I bind this to a shortcut in my tmux config like so:
>> bind K respawn-pane -k
Re: Important: new shell & bootstrap required after rebasing, again
Posted by Marcel Kornacker <ma...@cloudera.com>.
I'm getting Kudu client download errors fairly reliably now. This is
from a clean shell.
Downloading kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz to
/home/marcel/impala/toolchain
Traceback (most recent call last):
File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 355, in <module>
bootstrap(toolchain_root, packages)
File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 117, in bootstrap
download_package(toolchain_root, pkg_name, pkg_version, compiler)
File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 99, in
download_package
wget_and_unpack_package(download_path, file_name, destination, True)
File "/home/marcel/impala/bin/bootstrap_toolchain.py", line 81, in
wget_and_unpack_package
sh.wget(download_path, directory_prefix=destination,
no_clobber=wget_no_clobber)
File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
line 1021, in __call__
return RunningCommand(cmd, call_args, stdin, stdout, stderr)
File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
line 486, in __init__
self.wait()
File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
line 500, in wait
self.handle_command_exit_code(exit_code)
File "/home/marcel/impala/infra/python/env/local/lib/python2.7/site-packages/sh.py",
line 516, in handle_command_exit_code
raise exc(self.ran, self.process.stdout, self.process.stderr)
sh.ErrorReturnCode_8:
RAN: '/usr/bin/wget
https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
--directory-prefix=/home/marcel/impala/toolchain --no-clobber'
STDOUT:
STDERR:
--2016-12-11 08:56:17--
https://native-toolchain.s3.amazonaws.com/build/267-98c642ffcb/kudu/e018a83-gcc-4.9.2/kudu-e018a83-gcc-4.9.2-ec2-package-ubuntu-14-04.tar.gz
Resolving native-toolchain.s3.amazonaws.com
(native-toolchain.s3.amazonaws.com)... 54.231.237.83
Connecting to native-toolchain.s3.amazonaws.com
(native-toolchain.s3.amazonaws.com)|54.231.237.83|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2016-12-11 08:56:17 ERROR 403: Forbidden.
On Sat, Dec 10, 2016 at 11:11 PM, Matthew Jacobs <mj...@cloudera.com> wrote:
> Hi all,
>
>
> We had to bump the Kudu version again so unfortunately you'll have to
> refresh your environment again to get the latest Kudu. After fetching
> the latest Impala commits (including commit 39017adf) and rebasing,
> you'll need to:
>
>
> 1) start a new shell OR make sure to
> export IMPALA_TOOLCHAIN_BUILD_ID=308-96a4cc516e
>
>
> 2) Run buildall.sh, that will:
> a) download the new Kudu bits in the toolchain by calling
> bootstrap_toolchain.py (or you can do so yourself)
> b) restart the minicluster with the new kudu
>
>
>
> From Lars:
> If you don't want to mess with your pane layout in tmux you can run
> tmux respawn-pane -k
> to kill and replace the current pane with an entirely fresh shell.
> I bind this to a shortcut in my tmux config like so:
> bind K respawn-pane -k