You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@impala.apache.org by Lars Volker <lv...@cloudera.com> on 2016/09/29 09:56:17 UTC

Cannot build with custom toolchain

Hi,

I tried to follow the instructions here:
https://cwiki.apache.org/confluence/display/IMPALA/Building+native-toolchain+from+scratch+and+using+with+Impala

However I keep running into the following error:

SKIP_TOOLCHAIN_BOOTSTRAP is true, skipping download of Python dependencies.
SKIP_TOOLCHAIN_BOOTSTRAP is true, skipping toolchain bootstrap.
cp: target
‘/home/lv/i4/toolchain/cdh_components/hadoop-2.6.0-cdh5.10.0-SNAPSHOT//lib/native’
is not a directory
Error in ./buildall.sh at line 281: cp
"$HADOOP_LZO"/build/native/Linux-*-*/lib/libgplcompression.*
"$HADOOP_HOME/lib/native"

The problem seems to be that CDH component download is disabled by
SKIP_TOOLCHAIN_DOWNLOAD, too.

Is there a way around this?

Thanks, Lars

Re: Cannot build with custom toolchain

Posted by Tim Armstrong <ta...@cloudera.com>.

I don't think there's a combination of options you can set to get the
desired behaviour out of the box. I think the script evolved into this
rather than was designed - the CDH component downloading was added into the
existing script after that option was added.

You can work around this by changing IMPALA_TOOLCHAIN. E.g. running
IMPALA_TOOLCHAIN=/tmp/toolchain ./bin/bootstrap_toolchain.py to download
the toolchain to /tmp.

What I essentially did is did the initial bootstrap, then set
SKIP_TOOLCHAIN_BOOTSTRAP=true and then symlink custom-build toolchain
components into the toolchain dir.

It would probably make sense to add some command line flags
to bootstrap_toolchain.py to control what it downloads since I'm sure other
people will run into this.

On Thu, Sep 29, 2016 at 2:56 AM, Lars Volker <lv...@cloudera.com> wrote:

> Hi,
>
> I tried to follow the instructions here:
> https://cwiki.apache.org/confluence/display/IMPALA/
> Building+native-toolchain+from+scratch+and+using+with+Impala
>
> However I keep running into the following error:
>
> SKIP_TOOLCHAIN_BOOTSTRAP is true, skipping download of Python dependencies.
> SKIP_TOOLCHAIN_BOOTSTRAP is true, skipping toolchain bootstrap.
> cp: target
> ‘/home/lv/i4/toolchain/cdh_components/hadoop-2.6.0-cdh5.
> 10.0-SNAPSHOT//lib/native’
> is not a directory
> Error in ./buildall.sh at line 281: cp
> "$HADOOP_LZO"/build/native/Linux-*-*/lib/libgplcompression.*
> "$HADOOP_HOME/lib/native"
>
> The problem seems to be that CDH component download is disabled by
> SKIP_TOOLCHAIN_DOWNLOAD, too.
>
> Is there a way around this?
>
> Thanks, Lars
>

Re: Cannot build with custom toolchain

Posted by Jim Apple <jb...@cloudera.com>.

Martin noticed this, too:

https://issues.cloudera.org/browse/IMPALA-4067

On Thu, Sep 29, 2016 at 2:56 AM, Lars Volker <lv...@cloudera.com> wrote:
> Hi,
>
> I tried to follow the instructions here:
> https://cwiki.apache.org/confluence/display/IMPALA/Building+native-toolchain+from+scratch+and+using+with+Impala
>
> However I keep running into the following error:
>
> SKIP_TOOLCHAIN_BOOTSTRAP is true, skipping download of Python dependencies.
> SKIP_TOOLCHAIN_BOOTSTRAP is true, skipping toolchain bootstrap.
> cp: target
> ‘/home/lv/i4/toolchain/cdh_components/hadoop-2.6.0-cdh5.10.0-SNAPSHOT//lib/native’
> is not a directory
> Error in ./buildall.sh at line 281: cp
> "$HADOOP_LZO"/build/native/Linux-*-*/lib/libgplcompression.*
> "$HADOOP_HOME/lib/native"
>
> The problem seems to be that CDH component download is disabled by
> SKIP_TOOLCHAIN_DOWNLOAD, too.
>
> Is there a way around this?
>
> Thanks, Lars