You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@impala.apache.org by LiFu He <he...@gmail.com> on 2020/07/07 12:46:44 UTC
Is it possible to compile Impala on Apache hadoop/hive/hbase/sentry/ranger?
Hi everybody,
I have a question that why Impala relies on the CDH/CDP
distribution(hadoop/hive/hbase/sentry/ranger)? And is it possible to switch
to Apache version? Thanks in advance.
--helifu
Re: Is it possible to compile Impala on Apache hadoop/hive/hbase/sentry/ranger?
Posted by Shant Hovsepian <sh...@superdupershant.com>.
I think it's a great idea to add support for pulling in stock Apache
dependencies. The build configuration is flexible enough to support this,
however there is the burden of making sure all the tests pass and the
system is stable, finding the right combination of versions for various
projects that work well together and have compatible APIs is always quite a
challenge.
It would be best if someone would volunteer to enable an option to build
against these versions and submit a patch.
Cheers!
-Shant
On Tue, Jul 28, 2020 at 11:41 PM Tianhua huang <hu...@gmail.com>
wrote:
> Now we are working on aarch64 supporting for Impala, and plan to integrate
> with ARM CI, but now we have to replace some *.so files, like files under
> cdp_components*/hadoop-3.1.1*/lib/native/*, Hadoop released version 3.3.0 a
> few days ago, it supports aarch64, so I wonder if we can migrate to use
> hadoop 3.3.0 in Impala? Thanks.
>
> On Thu, Jul 9, 2020 at 9:20 AM LiFu He <he...@apache.org> wrote:
>
> > Thank you very much, Jim! However, opaque dependencies are always a worry
> > and it will affect people's choices.
> >
> > On 2020/07/08 14:23:43, Jim Apple <jb...@apache.org> wrote:
> > > Hi LiFu! I'm Jim, and I started contributing to the Impala project in
> > 2015.
> > >
> > > When we were working on learning about Apache and ASF standards back
> when
> > > Impala was incubating, this question came up. The analogy that we
> learned
> > > was that Apache projects are not, themselves, owned by anyone other
> than
> > > the Foundation, they sometimes rely on software projects, such as
> > Microsoft
> > > Windows, that are owned by another corporation.
> > >
> > > While that's the organizational rationale, I think the community would
> > > welcome a build and deploy process that used only officially released
> > > snapshots from the other Apache projects that Impala depends upon. I
> > think
> > > this is a question of someone taking the time to put in the work,
> > similarly
> > > to how Impala has supported only x86-64 for years, but there are new
> > > efforts to make it work on AArch64.
> > >
> > > On Wed, Jul 8, 2020 at 6:38 AM LiFu He <he...@apache.org> wrote:
> > >
> > > > At the same time, I'm curious why an open source project relies on a
> > > > business product, though I understand that there are some historical
> > > > reasons. Could anybody share more info?
> > > >
> > > >
> > > > On 2020/07/07 12:46:44, LiFu He <he...@gmail.com> wrote:
> > > > > Hi everybody,
> > > > >
> > > > > I have a question that why Impala relies on the CDH/CDP
> > > > > distribution(hadoop/hive/hbase/sentry/ranger)? And is it possible
> to
> > > > switch
> > > > > to Apache version? Thanks in advance.
> > > > >
> > > > >
> > > > > --helifu
> > > > >
> > > >
> > >
> >
>
Re: Is it possible to compile Impala on Apache hadoop/hive/hbase/sentry/ranger?
Posted by Tianhua huang <hu...@gmail.com>.
Now we are working on aarch64 supporting for Impala, and plan to integrate
with ARM CI, but now we have to replace some *.so files, like files under
cdp_components*/hadoop-3.1.1*/lib/native/*, Hadoop released version 3.3.0 a
few days ago, it supports aarch64, so I wonder if we can migrate to use
hadoop 3.3.0 in Impala? Thanks.
On Thu, Jul 9, 2020 at 9:20 AM LiFu He <he...@apache.org> wrote:
> Thank you very much, Jim! However, opaque dependencies are always a worry
> and it will affect people's choices.
>
> On 2020/07/08 14:23:43, Jim Apple <jb...@apache.org> wrote:
> > Hi LiFu! I'm Jim, and I started contributing to the Impala project in
> 2015.
> >
> > When we were working on learning about Apache and ASF standards back when
> > Impala was incubating, this question came up. The analogy that we learned
> > was that Apache projects are not, themselves, owned by anyone other than
> > the Foundation, they sometimes rely on software projects, such as
> Microsoft
> > Windows, that are owned by another corporation.
> >
> > While that's the organizational rationale, I think the community would
> > welcome a build and deploy process that used only officially released
> > snapshots from the other Apache projects that Impala depends upon. I
> think
> > this is a question of someone taking the time to put in the work,
> similarly
> > to how Impala has supported only x86-64 for years, but there are new
> > efforts to make it work on AArch64.
> >
> > On Wed, Jul 8, 2020 at 6:38 AM LiFu He <he...@apache.org> wrote:
> >
> > > At the same time, I'm curious why an open source project relies on a
> > > business product, though I understand that there are some historical
> > > reasons. Could anybody share more info?
> > >
> > >
> > > On 2020/07/07 12:46:44, LiFu He <he...@gmail.com> wrote:
> > > > Hi everybody,
> > > >
> > > > I have a question that why Impala relies on the CDH/CDP
> > > > distribution(hadoop/hive/hbase/sentry/ranger)? And is it possible to
> > > switch
> > > > to Apache version? Thanks in advance.
> > > >
> > > >
> > > > --helifu
> > > >
> > >
> >
>
Re: Is it possible to compile Impala on Apache hadoop/hive/hbase/sentry/ranger?
Posted by LiFu He <he...@apache.org>.
Thank you very much, Jim! However, opaque dependencies are always a worry and it will affect people's choices.
On 2020/07/08 14:23:43, Jim Apple <jb...@apache.org> wrote:
> Hi LiFu! I'm Jim, and I started contributing to the Impala project in 2015.
>
> When we were working on learning about Apache and ASF standards back when
> Impala was incubating, this question came up. The analogy that we learned
> was that Apache projects are not, themselves, owned by anyone other than
> the Foundation, they sometimes rely on software projects, such as Microsoft
> Windows, that are owned by another corporation.
>
> While that's the organizational rationale, I think the community would
> welcome a build and deploy process that used only officially released
> snapshots from the other Apache projects that Impala depends upon. I think
> this is a question of someone taking the time to put in the work, similarly
> to how Impala has supported only x86-64 for years, but there are new
> efforts to make it work on AArch64.
>
> On Wed, Jul 8, 2020 at 6:38 AM LiFu He <he...@apache.org> wrote:
>
> > At the same time, I'm curious why an open source project relies on a
> > business product, though I understand that there are some historical
> > reasons. Could anybody share more info?
> >
> >
> > On 2020/07/07 12:46:44, LiFu He <he...@gmail.com> wrote:
> > > Hi everybody,
> > >
> > > I have a question that why Impala relies on the CDH/CDP
> > > distribution(hadoop/hive/hbase/sentry/ranger)? And is it possible to
> > switch
> > > to Apache version? Thanks in advance.
> > >
> > >
> > > --helifu
> > >
> >
>
Re: Is it possible to compile Impala on Apache hadoop/hive/hbase/sentry/ranger?
Posted by Jim Apple <jb...@apache.org>.
Hi LiFu! I'm Jim, and I started contributing to the Impala project in 2015.
When we were working on learning about Apache and ASF standards back when
Impala was incubating, this question came up. The analogy that we learned
was that Apache projects are not, themselves, owned by anyone other than
the Foundation, they sometimes rely on software projects, such as Microsoft
Windows, that are owned by another corporation.
While that's the organizational rationale, I think the community would
welcome a build and deploy process that used only officially released
snapshots from the other Apache projects that Impala depends upon. I think
this is a question of someone taking the time to put in the work, similarly
to how Impala has supported only x86-64 for years, but there are new
efforts to make it work on AArch64.
On Wed, Jul 8, 2020 at 6:38 AM LiFu He <he...@apache.org> wrote:
> At the same time, I'm curious why an open source project relies on a
> business product, though I understand that there are some historical
> reasons. Could anybody share more info?
>
>
> On 2020/07/07 12:46:44, LiFu He <he...@gmail.com> wrote:
> > Hi everybody,
> >
> > I have a question that why Impala relies on the CDH/CDP
> > distribution(hadoop/hive/hbase/sentry/ranger)? And is it possible to
> switch
> > to Apache version? Thanks in advance.
> >
> >
> > --helifu
> >
>
Re: Is it possible to compile Impala on Apache hadoop/hive/hbase/sentry/ranger?
Posted by LiFu He <he...@apache.org>.
At the same time, I'm curious why an open source project relies on a business product, though I understand that there are some historical reasons. Could anybody share more info?
On 2020/07/07 12:46:44, LiFu He <he...@gmail.com> wrote:
> Hi everybody,
>
> I have a question that why Impala relies on the CDH/CDP
> distribution(hadoop/hive/hbase/sentry/ranger)? And is it possible to switch
> to Apache version? Thanks in advance.
>
>
> --helifu
>