normalizing spark tarball dependency in Hive build

[Sergey Shelukhin <se...@hortonworks.com>]

Hi.
I was trying to build Hive on a slow connection (or I could have no
connection for that matter), and pulling
"http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.3.0-bin-hadoop2
-without-hive.tgz” was taking forever (I ctrl-c-ed it eventually).
On a good note it did appear to respect “-o” on rebuild attempt (either
that, or whatever was remaining from the canceled build sufficed for the
mvn install -o … build that followed).
Is it possible to get this dependency via some more conventional means
like maven?

Re: normalizing spark tarball dependency in Hive build

[Sergey Shelukhin <se...@hortonworks.com>]

It’s possible to publish binaries to central.
For example, your kit redistributable is published this way:
http://search.maven.org/#browse|928812221


On 15/5/26, 21:35, "Xuefu Zhang" <xz...@cloudera.com> wrote:

>We thought of that, but unfortunate there this is a binary which isn't
>published anywhere in public maven repositories. That's why we hosted it
>at
>cloudfront.
>
>I think this is a general problem for any binaries required by tests. We
>are open to suggestions though.
>
>Thanks,
>Xuefu
>
>On Tue, May 26, 2015 at 1:35 PM, Sergey Shelukhin <se...@hortonworks.com>
>wrote:
>
>> Hi.
>> I was trying to build Hive on a slow connection (or I could have no
>> connection for that matter), and pulling
>> "
>> 
>>http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.3.0-bin-hadoop
>>2
>> -without-hive.tgz” was taking forever (I ctrl-c-ed it eventually).
>> On a good note it did appear to respect “-o” on rebuild attempt (either
>> that, or whatever was remaining from the canceled build sufficed for the
>> mvn install -o … build that followed).
>> Is it possible to get this dependency via some more conventional means
>> like maven?
>>
>>

Re: normalizing spark tarball dependency in Hive build

[Xuefu Zhang <xz...@cloudera.com>]

We thought of that, but unfortunate there this is a binary which isn't
published anywhere in public maven repositories. That's why we hosted it at
cloudfront.

I think this is a general problem for any binaries required by tests. We
are open to suggestions though.

Thanks,
Xuefu

On Tue, May 26, 2015 at 1:35 PM, Sergey Shelukhin <se...@hortonworks.com>
wrote:

> Hi.
> I was trying to build Hive on a slow connection (or I could have no
> connection for that matter), and pulling
> "
> http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.3.0-bin-hadoop2
> -without-hive.tgz” was taking forever (I ctrl-c-ed it eventually).
> On a good note it did appear to respect “-o” on rebuild attempt (either
> that, or whatever was remaining from the canceled build sufficed for the
> mvn install -o … build that followed).
> Is it possible to get this dependency via some more conventional means
> like maven?
>
>