You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by Pramod Immaneni <pr...@datatorrent.com> on 2016/10/03 21:02:30 UTC
[Discuss] Hadoop upgrade
It would be good to know the API additions between 2.4.x and 2.6.x that we
think we can use immediately for some feature, to have a more informed vote.
Thanks
On Mon, Oct 3, 2016 at 1:47 PM, David Yan <da...@datatorrent.com> wrote:
> Hi all,
>
> Thomas created this ticket for upgrading our Hadoop dependency version a
> couple weeks ago:
>
> https://issues.apache.org/jira/browse/APEXCORE-536
>
> We'd like to get the ball rolling and would like to take a vote from the
> community which version we would like to upgrade to. We have these choices:
>
> 2.2.0 (no upgrade)
> 2.4.x
> 2.5.x
> 2.6.x
>
> We are not considering 2.7.x because we already know that many Apex users
> are using Hadoop distros that are based on 2.6.
>
> Please note that Apex works with all versions of Hadoop higher or equal to
> the Hadoop version Apex depends on, as long as it's 2.x.x. We are not
> considering Hadoop 3.0.0-alpha yet at this time.
>
> When voting, please keep these in mind:
>
> - The features that are added in 2.4.x, 2.5.x, and 2.6.x respectively, and
> how useful those features are for Apache Apex
> - The Hadoop versions the major distros (Cloudera, Hortonworks, MapR, EMR,
> etc) are supporting
> - The Hadoop versions what typical Apex users are using
>
> Thanks,
>
> David
>
Re: [Discuss] Hadoop upgrade
Posted by Pramod Immaneni <pr...@datatorrent.com>.
Thanks for the information.
On Mon, Oct 3, 2016 at 3:30 PM, David Yan <da...@datatorrent.com> wrote:
> Just from scanning the API, I see the following features additions from 2.4
> to 2.6:
>
> - Ability to get memory-seconds and vcore-seconds
> from ApplicationResourceUsageReport.
> - Node labels
> - Resource reservation
>
> We already know we would like to use the following features that were added
> since 2.2 to 2.4:
> - Application tags
> - API that returns information about Application Attempts
>
> David
>
> On Mon, Oct 3, 2016 at 2:02 PM, Pramod Immaneni <pr...@datatorrent.com>
> wrote:
>
> > It would be good to know the API additions between 2.4.x and 2.6.x that
> we
> > think we can use immediately for some feature, to have a more informed
> > vote.
> >
> > Thanks
> >
> > On Mon, Oct 3, 2016 at 1:47 PM, David Yan <da...@datatorrent.com> wrote:
> >
> > > Hi all,
> > >
> > > Thomas created this ticket for upgrading our Hadoop dependency version
> a
> > > couple weeks ago:
> > >
> > > https://issues.apache.org/jira/browse/APEXCORE-536
> > >
> > > We'd like to get the ball rolling and would like to take a vote from
> the
> > > community which version we would like to upgrade to. We have these
> > choices:
> > >
> > > 2.2.0 (no upgrade)
> > > 2.4.x
> > > 2.5.x
> > > 2.6.x
> > >
> > > We are not considering 2.7.x because we already know that many Apex
> users
> > > are using Hadoop distros that are based on 2.6.
> > >
> > > Please note that Apex works with all versions of Hadoop higher or equal
> > to
> > > the Hadoop version Apex depends on, as long as it's 2.x.x. We are not
> > > considering Hadoop 3.0.0-alpha yet at this time.
> > >
> > > When voting, please keep these in mind:
> > >
> > > - The features that are added in 2.4.x, 2.5.x, and 2.6.x respectively,
> > and
> > > how useful those features are for Apache Apex
> > > - The Hadoop versions the major distros (Cloudera, Hortonworks, MapR,
> > EMR,
> > > etc) are supporting
> > > - The Hadoop versions what typical Apex users are using
> > >
> > > Thanks,
> > >
> > > David
> > >
> >
>
Re: [Discuss] Hadoop upgrade
Posted by David Yan <da...@datatorrent.com>.
Just from scanning the API, I see the following features additions from 2.4
to 2.6:
- Ability to get memory-seconds and vcore-seconds
from ApplicationResourceUsageReport.
- Node labels
- Resource reservation
We already know we would like to use the following features that were added
since 2.2 to 2.4:
- Application tags
- API that returns information about Application Attempts
David
On Mon, Oct 3, 2016 at 2:02 PM, Pramod Immaneni <pr...@datatorrent.com>
wrote:
> It would be good to know the API additions between 2.4.x and 2.6.x that we
> think we can use immediately for some feature, to have a more informed
> vote.
>
> Thanks
>
> On Mon, Oct 3, 2016 at 1:47 PM, David Yan <da...@datatorrent.com> wrote:
>
> > Hi all,
> >
> > Thomas created this ticket for upgrading our Hadoop dependency version a
> > couple weeks ago:
> >
> > https://issues.apache.org/jira/browse/APEXCORE-536
> >
> > We'd like to get the ball rolling and would like to take a vote from the
> > community which version we would like to upgrade to. We have these
> choices:
> >
> > 2.2.0 (no upgrade)
> > 2.4.x
> > 2.5.x
> > 2.6.x
> >
> > We are not considering 2.7.x because we already know that many Apex users
> > are using Hadoop distros that are based on 2.6.
> >
> > Please note that Apex works with all versions of Hadoop higher or equal
> to
> > the Hadoop version Apex depends on, as long as it's 2.x.x. We are not
> > considering Hadoop 3.0.0-alpha yet at this time.
> >
> > When voting, please keep these in mind:
> >
> > - The features that are added in 2.4.x, 2.5.x, and 2.6.x respectively,
> and
> > how useful those features are for Apache Apex
> > - The Hadoop versions the major distros (Cloudera, Hortonworks, MapR,
> EMR,
> > etc) are supporting
> > - The Hadoop versions what typical Apex users are using
> >
> > Thanks,
> >
> > David
> >
>