[jira] [Closed] (IMPALA-6148) Provide mechanism to specify thirdparty dependency as a URL

["Philip Zeyliger (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Philip Zeyliger closed IMPALA-6148.
-----------------------------------
    Resolution: Fixed

{code}
commit 2212a8897e66e1a5c548132d09a8e33e275a0e3b
Author: Philip Zeyliger <ph...@cloudera.com>
Date:   Thu Nov 2 15:43:10 2017 -0700

    IMPALA-6148: Specifying thirdparty deps as URLs

    If the environment variable $IMPALA_<NAME>_URL is configured in
    impala-config-branch.sh or impala-config-local, for a thirdparty
    dependency, use that to download it instead of the s3://native-toolchain
    bucket. This makes testing against arbitrary versions of the
    dependencies easier.

    I did a little bit of refactoring while here, creating a small class for
    a Package to handle reading the environment variables. I also changed
    bootstrap_toolchain.py to use Python logging, which cleans up the output
    during the multi-threaded downloading.

    I tested this by both with customized URLs and by running the regular
    build (pre-review-test, without most of the slow test suites).

    Change-Id: I4628d86022d4bd8b762313f7056d76416a58b422
    Reviewed-on: http://gerrit.cloudera.org:8080/8456
    Reviewed-by: Joe McDonnell <jo...@cloudera.com>
    Tested-by: Impala Public Jenkins
{code}

> Provide mechanism to specify thirdparty dependency as a URL
> -----------------------------------------------------------
>
>                 Key: IMPALA-6148
>                 URL: https://issues.apache.org/jira/browse/IMPALA-6148
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Infrastructure
>            Reporter: Philip Zeyliger
>            Assignee: Philip Zeyliger
>            Priority: Trivial
>
> To be able to test Impala with different versions of Hadoop (and other dependencies), it's handy to be able to customize where our dependencies are downloaded. Today, our dependencies (native-toolchain and "Hadoop and friends") are typically downloaded from {{s3://native-toolchain/}}. I'm proposing to extend the existing scheme to be able to specify arbitrary URLs as their locations.
> This is coming up in the context of testing Impala against newer versions of Hadoop, Hive, Sentry, and HBase.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)