You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ambari.apache.org by Robin Carnow <ro...@goldbot.com> on 2013/06/13 21:16:07 UTC

Building HDP Repo RPMS

Hello,

I have a question about the set of hadoop rpms that are available in the
HDP and HDP-UTILS repos.
Does anyone know how the rpms are made?
I have a situation where I must use hadoop 1.0.4, but 1.0.3 is the latest
version used by ambari 0.9.

Is there any way to get or create the same set of rpms for hadoop 1.0.4 so
that they will work with Ambari 0.9?  Maybe there is build script that is
run?

I've looked through the rpms and some are just text files which I can use.
Some have binary executables which I guess would not work with a newer
version of hadoop.

I know that Ambari 0.9 is VERY old and I should not be using it.  I do not
have any choice in the matter.  I need a quick fix until I'm able to use
1.2.

Any help would be much appreciated


 Thank you,
Robin

Re: Building HDP Repo RPMS

Posted by Robin Carnow <ro...@goldbot.com>.
Just to report back I was able to successfully build the set of
hadoop-*-1.0.4-1.el5.x86_64.rpm files using bigtop.

It wasn't easy at all but it is possible.





On Fri, Jun 14, 2013 at 10:10 AM, Robin Carnow <ro...@goldbot.com> wrote:

> Thank you very much Paulo.  I will give it a shot.
>
>  ~Robin
>
>
> On Fri, Jun 14, 2013 at 10:01 AM, Paulo Ricardo Paz Vital <
> pvital@linux.vnet.ibm.com> wrote:
>
>> Robin,
>>
>> You can use Bigtop [1] to create new RPM files of Hadoop (use the
>> release 0.3.0 that supports Hadoop-1.X) and set a local repository with
>> the new files to be used by Ambari.
>>
>> [1]http://bigtop.apache.org/
>> [2]
>> http://archive.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/
>>
>> I've been using this approach in some tests with 1.2.X releases, but I
>> guess you can do the same with 0.9 - configure Ambari to use your
>> internal/local repository.
>>
>> Best regards, Paulo.
>>
>> On Fri, 2013-06-14 at 09:29 -0400, Robin Carnow wrote:
>> > Thank you JP for your better articulation of my question although the
>> > rpms I need are different.
>> >
>> > Yusaku I am not familiar with Bigtop.  Are you saying that there is
>> > some way to use Bigtop to take the hadoop-1.0.4-1.x86_64.rpm (or maybe
>> > hadoop-1.0.4.tar.gz) and build the following rpms:
>> >
>> > HDP/repos/centos5/hadoop/hadoop-source-1.0.4-1.el5.x86_64.rpm
>> > HDP/repos/centos5/hadoop/hadoop-jobtracker-1.0.4-1.el5.x86_64.rpm
>> > HDP/repos/centos5/hadoop/hadoop-secondarynamenode-1.0.4-1.el5.x86_64.rpm
>> > HDP/repos/centos5/hadoop/hadoop-sbin-1.0.4-1.el5.x86_64.rpm
>> > HDP/repos/centos5/hadoop/hadoop-debuginfo-1.0.4-1.el5.x86_64.rpm
>> > HDP/repos/centos5/hadoop/hadoop-pipes-1.0.4-1.el5.x86_64.rpm
>> > HDP/repos/centos5/hadoop/hadoop-native-1.0.4-1.el5.x86_64.rpm
>> > HDP/repos/centos5/hadoop/hadoop-tasktracker-1.0.4-1.el5.x86_64.rpm
>> >
>> >
>> > These are the rpms I need to get a local HDP-1.1.1.16 repo up.
>> >
>> >
>> >
>> > Thank you,
>> > Robin
>> >
>> >
>> >
>> > On Fri, Jun 14, 2013 at 7:46 AM, Jp <pa...@gmail.com> wrote:
>> >         The hadoop rpm in HDP repo  has the following rpms...
>> >
>> >         hadoop-sbin-1.2.0.1.3.0.0-107.el6.x86_64
>> >         hadoop-lzo-0.5.0-1.x86_64
>> >         hadoop-lzo-native-0.5.0-1.x86_64
>> >         hadoop-libhdfs-1.2.0.1.3.0.0-107.el6.x86_64
>> >         hadoop-1.2.0.1.3.0.0-107.el6.x86_64
>> >         hadoop-native-1.2.0.1.3.0.0-107.el6.x86_64
>> >         hadoop-pipes-1.2.0.1.3.0.0-107.el6.x86_64
>> >
>> >         whereas the hadoop rpm released by apache has only
>> >
>> >         hadoop-1.0.4-1.x86_64.rpm
>> >
>> >         Is it ok to use apache hadoop rpm instead of HDP hadoop rpm...
>> ll ambari work fine?
>> >
>> >         If not how do we split the hadoop-1.0.4-1.x86_64.rpm  into
>> different rpms as provided by
>> >         HDP....
>> >
>> >         Is it possible to revert to manual configuration after using
>> ambari? If so where do we find the required conf files?
>> >         I dont see them in the statndard folder "/etc/hadoop"
>> >
>> >
>> >
>> >
>> >         On 14-Jun-13 3:23 AM, Yusaku Sako wrote:
>> >
>> >         > AFAIK, HDP is built using Bigtop.
>> >         > Haven't tested but looks like there are RPMs here:
>> >         > https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/
>> >         >
>> >         > Yusaku
>> >         >
>> >         > On Thu, Jun 13, 2013 at 12:16 PM, Robin Carnow <
>> robin@goldbot.com> wrote:
>> >         > > Hello,
>> >         > >
>> >         > > I have a question about the set of hadoop rpms that are
>> available in the HDP
>> >         > > and HDP-UTILS repos.
>> >         > > Does anyone know how the rpms are made?
>> >         > > I have a situation where I must use hadoop 1.0.4, but 1.0.3
>> is the latest
>> >         > > version used by ambari 0.9.
>> >         > >
>> >         > > Is there any way to get or create the same set of rpms for
>> hadoop 1.0.4 so
>> >         > > that they will work with Ambari 0.9?  Maybe there is build
>> script that is
>> >         > > run?
>> >         > >
>> >         > > I've looked through the rpms and some are just text files
>> which I can use.
>> >         > > Some have binary executables which I guess would not work
>> with a newer
>> >         > > version of hadoop.
>> >         > >
>> >         > > I know that Ambari 0.9 is VERY old and I should not be
>> using it.  I do not
>> >         > > have any choice in the matter.  I need a quick fix until
>> I'm able to use
>> >         > > 1.2.
>> >         > >
>> >         > > Any help would be much appreciated
>> >         > >
>> >         > >
>> >         > > Thank you,
>> >         > > Robin
>> >
>> >
>> >
>> >
>>
>>
>>
>

Re: Building HDP Repo RPMS

Posted by Robin Carnow <ro...@goldbot.com>.
Thank you very much Paulo.  I will give it a shot.

~Robin


On Fri, Jun 14, 2013 at 10:01 AM, Paulo Ricardo Paz Vital <
pvital@linux.vnet.ibm.com> wrote:

> Robin,
>
> You can use Bigtop [1] to create new RPM files of Hadoop (use the
> release 0.3.0 that supports Hadoop-1.X) and set a local repository with
> the new files to be used by Ambari.
>
> [1]http://bigtop.apache.org/
> [2]
> http://archive.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/
>
> I've been using this approach in some tests with 1.2.X releases, but I
> guess you can do the same with 0.9 - configure Ambari to use your
> internal/local repository.
>
> Best regards, Paulo.
>
> On Fri, 2013-06-14 at 09:29 -0400, Robin Carnow wrote:
> > Thank you JP for your better articulation of my question although the
> > rpms I need are different.
> >
> > Yusaku I am not familiar with Bigtop.  Are you saying that there is
> > some way to use Bigtop to take the hadoop-1.0.4-1.x86_64.rpm (or maybe
> > hadoop-1.0.4.tar.gz) and build the following rpms:
> >
> > HDP/repos/centos5/hadoop/hadoop-source-1.0.4-1.el5.x86_64.rpm
> > HDP/repos/centos5/hadoop/hadoop-jobtracker-1.0.4-1.el5.x86_64.rpm
> > HDP/repos/centos5/hadoop/hadoop-secondarynamenode-1.0.4-1.el5.x86_64.rpm
> > HDP/repos/centos5/hadoop/hadoop-sbin-1.0.4-1.el5.x86_64.rpm
> > HDP/repos/centos5/hadoop/hadoop-debuginfo-1.0.4-1.el5.x86_64.rpm
> > HDP/repos/centos5/hadoop/hadoop-pipes-1.0.4-1.el5.x86_64.rpm
> > HDP/repos/centos5/hadoop/hadoop-native-1.0.4-1.el5.x86_64.rpm
> > HDP/repos/centos5/hadoop/hadoop-tasktracker-1.0.4-1.el5.x86_64.rpm
> >
> >
> > These are the rpms I need to get a local HDP-1.1.1.16 repo up.
> >
> >
> >
> > Thank you,
> > Robin
> >
> >
> >
> > On Fri, Jun 14, 2013 at 7:46 AM, Jp <pa...@gmail.com> wrote:
> >         The hadoop rpm in HDP repo  has the following rpms...
> >
> >         hadoop-sbin-1.2.0.1.3.0.0-107.el6.x86_64
> >         hadoop-lzo-0.5.0-1.x86_64
> >         hadoop-lzo-native-0.5.0-1.x86_64
> >         hadoop-libhdfs-1.2.0.1.3.0.0-107.el6.x86_64
> >         hadoop-1.2.0.1.3.0.0-107.el6.x86_64
> >         hadoop-native-1.2.0.1.3.0.0-107.el6.x86_64
> >         hadoop-pipes-1.2.0.1.3.0.0-107.el6.x86_64
> >
> >         whereas the hadoop rpm released by apache has only
> >
> >         hadoop-1.0.4-1.x86_64.rpm
> >
> >         Is it ok to use apache hadoop rpm instead of HDP hadoop rpm...
> ll ambari work fine?
> >
> >         If not how do we split the hadoop-1.0.4-1.x86_64.rpm  into
> different rpms as provided by
> >         HDP....
> >
> >         Is it possible to revert to manual configuration after using
> ambari? If so where do we find the required conf files?
> >         I dont see them in the statndard folder "/etc/hadoop"
> >
> >
> >
> >
> >         On 14-Jun-13 3:23 AM, Yusaku Sako wrote:
> >
> >         > AFAIK, HDP is built using Bigtop.
> >         > Haven't tested but looks like there are RPMs here:
> >         > https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/
> >         >
> >         > Yusaku
> >         >
> >         > On Thu, Jun 13, 2013 at 12:16 PM, Robin Carnow <
> robin@goldbot.com> wrote:
> >         > > Hello,
> >         > >
> >         > > I have a question about the set of hadoop rpms that are
> available in the HDP
> >         > > and HDP-UTILS repos.
> >         > > Does anyone know how the rpms are made?
> >         > > I have a situation where I must use hadoop 1.0.4, but 1.0.3
> is the latest
> >         > > version used by ambari 0.9.
> >         > >
> >         > > Is there any way to get or create the same set of rpms for
> hadoop 1.0.4 so
> >         > > that they will work with Ambari 0.9?  Maybe there is build
> script that is
> >         > > run?
> >         > >
> >         > > I've looked through the rpms and some are just text files
> which I can use.
> >         > > Some have binary executables which I guess would not work
> with a newer
> >         > > version of hadoop.
> >         > >
> >         > > I know that Ambari 0.9 is VERY old and I should not be using
> it.  I do not
> >         > > have any choice in the matter.  I need a quick fix until I'm
> able to use
> >         > > 1.2.
> >         > >
> >         > > Any help would be much appreciated
> >         > >
> >         > >
> >         > > Thank you,
> >         > > Robin
> >
> >
> >
> >
>
>
>

Re: Building HDP Repo RPMS

Posted by Paulo Ricardo Paz Vital <pv...@linux.vnet.ibm.com>.
Robin,

You can use Bigtop [1] to create new RPM files of Hadoop (use the
release 0.3.0 that supports Hadoop-1.X) and set a local repository with
the new files to be used by Ambari.

[1]http://bigtop.apache.org/
[2]http://archive.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/

I've been using this approach in some tests with 1.2.X releases, but I
guess you can do the same with 0.9 - configure Ambari to use your
internal/local repository. 

Best regards, Paulo.

On Fri, 2013-06-14 at 09:29 -0400, Robin Carnow wrote:
> Thank you JP for your better articulation of my question although the
> rpms I need are different.
> 
> Yusaku I am not familiar with Bigtop.  Are you saying that there is
> some way to use Bigtop to take the hadoop-1.0.4-1.x86_64.rpm (or maybe
> hadoop-1.0.4.tar.gz) and build the following rpms:
> 
> HDP/repos/centos5/hadoop/hadoop-source-1.0.4-1.el5.x86_64.rpm
> HDP/repos/centos5/hadoop/hadoop-jobtracker-1.0.4-1.el5.x86_64.rpm
> HDP/repos/centos5/hadoop/hadoop-secondarynamenode-1.0.4-1.el5.x86_64.rpm
> HDP/repos/centos5/hadoop/hadoop-sbin-1.0.4-1.el5.x86_64.rpm
> HDP/repos/centos5/hadoop/hadoop-debuginfo-1.0.4-1.el5.x86_64.rpm
> HDP/repos/centos5/hadoop/hadoop-pipes-1.0.4-1.el5.x86_64.rpm
> HDP/repos/centos5/hadoop/hadoop-native-1.0.4-1.el5.x86_64.rpm
> HDP/repos/centos5/hadoop/hadoop-tasktracker-1.0.4-1.el5.x86_64.rpm
> 
> 
> These are the rpms I need to get a local HDP-1.1.1.16 repo up.
> 
> 
> 
> Thank you,
> Robin
> 
> 
> 
> On Fri, Jun 14, 2013 at 7:46 AM, Jp <pa...@gmail.com> wrote:
>         The hadoop rpm in HDP repo  has the following rpms...
>         
>         hadoop-sbin-1.2.0.1.3.0.0-107.el6.x86_64
>         hadoop-lzo-0.5.0-1.x86_64
>         hadoop-lzo-native-0.5.0-1.x86_64
>         hadoop-libhdfs-1.2.0.1.3.0.0-107.el6.x86_64
>         hadoop-1.2.0.1.3.0.0-107.el6.x86_64
>         hadoop-native-1.2.0.1.3.0.0-107.el6.x86_64
>         hadoop-pipes-1.2.0.1.3.0.0-107.el6.x86_64
>         
>         whereas the hadoop rpm released by apache has only 
>         
>         hadoop-1.0.4-1.x86_64.rpm 
>         
>         Is it ok to use apache hadoop rpm instead of HDP hadoop rpm... ll ambari work fine?
>         
>         If not how do we split the hadoop-1.0.4-1.x86_64.rpm  into different rpms as provided by 
>         HDP....
>         
>         Is it possible to revert to manual configuration after using ambari? If so where do we find the required conf files?
>         I dont see them in the statndard folder "/etc/hadoop"
>         
>         
>         
>         
>         On 14-Jun-13 3:23 AM, Yusaku Sako wrote:
>         
>         > AFAIK, HDP is built using Bigtop.
>         > Haven't tested but looks like there are RPMs here:
>         > https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/
>         > 
>         > Yusaku
>         > 
>         > On Thu, Jun 13, 2013 at 12:16 PM, Robin Carnow <ro...@goldbot.com> wrote:
>         > > Hello,
>         > > 
>         > > I have a question about the set of hadoop rpms that are available in the HDP
>         > > and HDP-UTILS repos.
>         > > Does anyone know how the rpms are made?
>         > > I have a situation where I must use hadoop 1.0.4, but 1.0.3 is the latest
>         > > version used by ambari 0.9.
>         > > 
>         > > Is there any way to get or create the same set of rpms for hadoop 1.0.4 so
>         > > that they will work with Ambari 0.9?  Maybe there is build script that is
>         > > run?
>         > > 
>         > > I've looked through the rpms and some are just text files which I can use.
>         > > Some have binary executables which I guess would not work with a newer
>         > > version of hadoop.
>         > > 
>         > > I know that Ambari 0.9 is VERY old and I should not be using it.  I do not
>         > > have any choice in the matter.  I need a quick fix until I'm able to use
>         > > 1.2.
>         > > 
>         > > Any help would be much appreciated
>         > > 
>         > > 
>         > > Thank you,
>         > > Robin
>         
>         
> 
> 



Re: Building HDP Repo RPMS

Posted by Artem Ervits <ar...@nyp.org>.
You can browse the repos on public-hortonworks... Website and pull the 1.0.x rpms. As far as having them work with later version of Ambari I am not so sure. You can also take a look at Apache Bigtop website. 1.0.x releases are part of bigtop 0.3.




Artem Ervits
Data Analyst
New York Presbyterian Hospital

From: Robin Carnow [mailto:robin@goldbot.com]
Sent: Friday, June 14, 2013 09:29 AM
To: ambari-user@incubator.apache.org <am...@incubator.apache.org>
Subject: Re: Building HDP Repo RPMS

Thank you JP for your better articulation of my question although the rpms I need are different.

Yusaku I am not familiar with Bigtop.  Are you saying that there is some way to use Bigtop to take the hadoop-1.0.4-1.x86_64.rpm (or maybe hadoop-1.0.4.tar.gz) and build the following rpms:

HDP/repos/centos5/hadoop/hadoop-source-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-jobtracker-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-secondarynamenode-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-sbin-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-debuginfo-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-pipes-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-native-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-tasktracker-1.0.4-1.el5.x86_64.rpm

These are the rpms I need to get a local HDP-1.1.1.16 repo up.


Thank you,
Robin


On Fri, Jun 14, 2013 at 7:46 AM, Jp <pa...@gmail.com>> wrote:
The hadoop rpm in HDP repo  has the following rpms...

hadoop-sbin-1.2.0.1.3.0.0-107.el6.x86_64
hadoop-lzo-0.5.0-1.x86_64
hadoop-lzo-native-0.5.0-1.x86_64
hadoop-libhdfs-1.2.0.1.3.0.0-107.el6.x86_64
hadoop-1.2.0.1.3.0.0-107.el6.x86_64
hadoop-native-1.2.0.1.3.0.0-107.el6.x86_64
hadoop-pipes-1.2.0.1.3.0.0-107.el6.x86_64

whereas the hadoop rpm released by apache has only


hadoop-1.0.4-1.x86_64.rpm<https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/hadoop-1.0.4-1.x86_64.rpm>

Is it ok to use apache hadoop rpm instead of HDP hadoop rpm... ll ambari work fine?

If not how do we split the hadoop-1.0.4-1.x86_64.rpm<https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/hadoop-1.0.4-1.x86_64.rpm>  into different rpms as provided by
HDP....

Is it possible to revert to manual configuration after using ambari? If so where do we find the required conf files?
I dont see them in the statndard folder "/etc/hadoop"





On 14-Jun-13 3:23 AM, Yusaku Sako wrote:

AFAIK, HDP is built using Bigtop.
Haven't tested but looks like there are RPMs here:
https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/

Yusaku

On Thu, Jun 13, 2013 at 12:16 PM, Robin Carnow <ro...@goldbot.com> wrote:


Hello,

I have a question about the set of hadoop rpms that are available in the HDP
and HDP-UTILS repos.
Does anyone know how the rpms are made?
I have a situation where I must use hadoop 1.0.4, but 1.0.3 is the latest
version used by ambari 0.9.

Is there any way to get or create the same set of rpms for hadoop 1.0.4 so
that they will work with Ambari 0.9?  Maybe there is build script that is
run?

I've looked through the rpms and some are just text files which I can use.
Some have binary executables which I guess would not work with a newer
version of hadoop.

I know that Ambari 0.9 is VERY old and I should not be using it.  I do not
have any choice in the matter.  I need a quick fix until I'm able to use
1.2.

Any help would be much appreciated


Thank you,
Robin




This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

Re: Building HDP Repo RPMS

Posted by Robin Carnow <ro...@goldbot.com>.
Thank you JP for your better articulation of my question although the rpms
I need are different.

Yusaku I am not familiar with Bigtop.  Are you saying that there is some
way to use Bigtop to take the hadoop-1.0.4-1.x86_64.rpm (or maybe
hadoop-1.0.4.tar.gz) and build the following rpms:

HDP/repos/centos5/hadoop/hadoop-source-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-jobtracker-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-secondarynamenode-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-sbin-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-debuginfo-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-pipes-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-native-1.0.4-1.el5.x86_64.rpm
HDP/repos/centos5/hadoop/hadoop-tasktracker-1.0.4-1.el5.x86_64.rpm

These are the rpms I need to get a local HDP-1.1.1.16 repo up.


Thank you,
Robin


On Fri, Jun 14, 2013 at 7:46 AM, Jp <pa...@gmail.com> wrote:

>  The hadoop rpm in HDP repo  has the following rpms...
>
> hadoop-sbin-1.2.0.1.3.0.0-107.el6.x86_64
> hadoop-lzo-0.5.0-1.x86_64
> hadoop-lzo-native-0.5.0-1.x86_64
> hadoop-libhdfs-1.2.0.1.3.0.0-107.el6.x86_64
> hadoop-1.2.0.1.3.0.0-107.el6.x86_64
> hadoop-native-1.2.0.1.3.0.0-107.el6.x86_64
> hadoop-pipes-1.2.0.1.3.0.0-107.el6.x86_64
>
> whereas the hadoop rpm released by apache has only
>
> hadoop-1.0.4-1.x86_64.rpm <https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/hadoop-1.0.4-1.x86_64.rpm>
>
> Is it ok to use apache hadoop rpm instead of HDP hadoop rpm... ll ambari work fine?
>
> If not how do we split the hadoop-1.0.4-1.x86_64.rpm <https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/hadoop-1.0.4-1.x86_64.rpm>  into different rpms as provided by
> HDP....
>
> Is it possible to revert to manual configuration after using ambari? If so where do we find the required conf files?
> I dont see them in the statndard folder "/etc/hadoop"
>
>
>
>
>
> On 14-Jun-13 3:23 AM, Yusaku Sako wrote:
>
> AFAIK, HDP is built using Bigtop.
> Haven't tested but looks like there are RPMs here:https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/
>
> Yusaku
>
> On Thu, Jun 13, 2013 at 12:16 PM, Robin Carnow <ro...@goldbot.com> <ro...@goldbot.com> wrote:
>
>  Hello,
>
> I have a question about the set of hadoop rpms that are available in the HDP
> and HDP-UTILS repos.
> Does anyone know how the rpms are made?
> I have a situation where I must use hadoop 1.0.4, but 1.0.3 is the latest
> version used by ambari 0.9.
>
> Is there any way to get or create the same set of rpms for hadoop 1.0.4 so
> that they will work with Ambari 0.9?  Maybe there is build script that is
> run?
>
> I've looked through the rpms and some are just text files which I can use.
> Some have binary executables which I guess would not work with a newer
> version of hadoop.
>
> I know that Ambari 0.9 is VERY old and I should not be using it.  I do not
> have any choice in the matter.  I need a quick fix until I'm able to use
> 1.2.
>
> Any help would be much appreciated
>
>
> Thank you,
> Robin
>
>
>

Re: Building HDP Repo RPMS

Posted by Jp <pa...@gmail.com>.
The hadoop rpm in HDP repo  has the following rpms...

hadoop-sbin-1.2.0.1.3.0.0-107.el6.x86_64
hadoop-lzo-0.5.0-1.x86_64
hadoop-lzo-native-0.5.0-1.x86_64
hadoop-libhdfs-1.2.0.1.3.0.0-107.el6.x86_64
hadoop-1.2.0.1.3.0.0-107.el6.x86_64
hadoop-native-1.2.0.1.3.0.0-107.el6.x86_64
hadoop-pipes-1.2.0.1.3.0.0-107.el6.x86_64

whereas the hadoop rpm released by apache has only

hadoop-1.0.4-1.x86_64.rpm  <https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/hadoop-1.0.4-1.x86_64.rpm>  

Is it ok to use apache hadoop rpm instead of HDP hadoop rpm... ll ambari work fine?

If not how do we split thehadoop-1.0.4-1.x86_64.rpm  <https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/hadoop-1.0.4-1.x86_64.rpm>   into different rpms as provided by
HDP....

Is it possible to revert to manual configuration after using ambari? If so where do we find the required conf files?
I dont see them in the statndard folder "/etc/hadoop"




On 14-Jun-13 3:23 AM, Yusaku Sako wrote:
> AFAIK, HDP is built using Bigtop.
> Haven't tested but looks like there are RPMs here:
> https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/
>
> Yusaku
>
> On Thu, Jun 13, 2013 at 12:16 PM, Robin Carnow <ro...@goldbot.com> wrote:
>> Hello,
>>
>> I have a question about the set of hadoop rpms that are available in the HDP
>> and HDP-UTILS repos.
>> Does anyone know how the rpms are made?
>> I have a situation where I must use hadoop 1.0.4, but 1.0.3 is the latest
>> version used by ambari 0.9.
>>
>> Is there any way to get or create the same set of rpms for hadoop 1.0.4 so
>> that they will work with Ambari 0.9?  Maybe there is build script that is
>> run?
>>
>> I've looked through the rpms and some are just text files which I can use.
>> Some have binary executables which I guess would not work with a newer
>> version of hadoop.
>>
>> I know that Ambari 0.9 is VERY old and I should not be using it.  I do not
>> have any choice in the matter.  I need a quick fix until I'm able to use
>> 1.2.
>>
>> Any help would be much appreciated
>>
>>
>> Thank you,
>> Robin


Re: Building HDP Repo RPMS

Posted by Yusaku Sako <yu...@hortonworks.com>.
AFAIK, HDP is built using Bigtop.
Haven't tested but looks like there are RPMs here:
https://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/

Yusaku

On Thu, Jun 13, 2013 at 12:16 PM, Robin Carnow <ro...@goldbot.com> wrote:
> Hello,
>
> I have a question about the set of hadoop rpms that are available in the HDP
> and HDP-UTILS repos.
> Does anyone know how the rpms are made?
> I have a situation where I must use hadoop 1.0.4, but 1.0.3 is the latest
> version used by ambari 0.9.
>
> Is there any way to get or create the same set of rpms for hadoop 1.0.4 so
> that they will work with Ambari 0.9?  Maybe there is build script that is
> run?
>
> I've looked through the rpms and some are just text files which I can use.
> Some have binary executables which I guess would not work with a newer
> version of hadoop.
>
> I know that Ambari 0.9 is VERY old and I should not be using it.  I do not
> have any choice in the matter.  I need a quick fix until I'm able to use
> 1.2.
>
> Any help would be much appreciated
>
>
> Thank you,
> Robin