You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@bigtop.apache.org by Philip Herron <ph...@wandisco.com> on 2013/03/27 14:27:34 UTC
Bigtop Component Tutorial
Hey all,
I forgot about this but i had written some documentation and didn't
finish but this might be useful to put somewhere:
To make rpm components within bigtop its not written 100% but something
is better than nothing.
--Phil
# Re sending this as i was not subscribed to the user mailing list and
thing it could be useful there too.
Re: Bigtop Component Tutorial
Posted by Konstantin Boudnik <co...@apache.org>.
Actually, Wiki is even better than a JIRA for docs - more alive.
Cos
On Thu, Mar 28, 2013 at 09:57AM, Roman Shaposhnik wrote:
> On Wed, Mar 27, 2013 at 10:38 AM, Konstantin Boudnik <co...@apache.org> wrote:
> > Thanks Phil for taking a stab at it.
> > I believe this is a very good start for the collection of docs - development
> > and otherwise - which should belong to docs/ folder in the source tree.
>
> Indeed a really good start! As far as documents-in-progress go we've had
> reasonable success with our Bigtop wiki:
> https://cwiki.apache.org/confluence/display/BIGTOP/Index
>
> Feel free to wikify your draft and send us a link. Then we can all participate
> in polishing it, etc.
>
> Thanks,
> Roman.
>
> P.S. Some folks may not know it but ASF wiki supports notifications -- you can
> subscribe to receive updates to a page. Personally I find it extremely useful.
Re: Bigtop Component Tutorial
Posted by Konstantin Boudnik <co...@apache.org>.
Actually, Wiki is even better than a JIRA for docs - more alive.
Cos
On Thu, Mar 28, 2013 at 09:57AM, Roman Shaposhnik wrote:
> On Wed, Mar 27, 2013 at 10:38 AM, Konstantin Boudnik <co...@apache.org> wrote:
> > Thanks Phil for taking a stab at it.
> > I believe this is a very good start for the collection of docs - development
> > and otherwise - which should belong to docs/ folder in the source tree.
>
> Indeed a really good start! As far as documents-in-progress go we've had
> reasonable success with our Bigtop wiki:
> https://cwiki.apache.org/confluence/display/BIGTOP/Index
>
> Feel free to wikify your draft and send us a link. Then we can all participate
> in polishing it, etc.
>
> Thanks,
> Roman.
>
> P.S. Some folks may not know it but ASF wiki supports notifications -- you can
> subscribe to receive updates to a page. Personally I find it extremely useful.
Re: Bigtop Component Tutorial
Posted by Roman Shaposhnik <rv...@apache.org>.
On Wed, Mar 27, 2013 at 10:38 AM, Konstantin Boudnik <co...@apache.org> wrote:
> Thanks Phil for taking a stab at it.
> I believe this is a very good start for the collection of docs - development
> and otherwise - which should belong to docs/ folder in the source tree.
Indeed a really good start! As far as documents-in-progress go we've had
reasonable success with our Bigtop wiki:
https://cwiki.apache.org/confluence/display/BIGTOP/Index
Feel free to wikify your draft and send us a link. Then we can all participate
in polishing it, etc.
Thanks,
Roman.
P.S. Some folks may not know it but ASF wiki supports notifications -- you can
subscribe to receive updates to a page. Personally I find it extremely useful.
Re: Bigtop Component Tutorial
Posted by Roman Shaposhnik <rv...@apache.org>.
On Wed, Mar 27, 2013 at 10:38 AM, Konstantin Boudnik <co...@apache.org> wrote:
> Thanks Phil for taking a stab at it.
> I believe this is a very good start for the collection of docs - development
> and otherwise - which should belong to docs/ folder in the source tree.
Indeed a really good start! As far as documents-in-progress go we've had
reasonable success with our Bigtop wiki:
https://cwiki.apache.org/confluence/display/BIGTOP/Index
Feel free to wikify your draft and send us a link. Then we can all participate
in polishing it, etc.
Thanks,
Roman.
P.S. Some folks may not know it but ASF wiki supports notifications -- you can
subscribe to receive updates to a page. Personally I find it extremely useful.
Re: Bigtop Component Tutorial
Posted by Konstantin Boudnik <co...@apache.org>.
Thanks Phil for taking a stab at it.
I believe this is a very good start for the collection of docs - development
and otherwise - which should belong to docs/ folder in the source tree.
Would you mind opening a JIRA for documentation improvement and submitting
this is as a patch?
Thanks,
Cos
On Wed, Mar 27, 2013 at 01:27PM, Philip Herron wrote:
> Hey all,
>
> I forgot about this but i had written some documentation and didn't
> finish but this might be useful to put somewhere:
>
> To make rpm components within bigtop its not written 100% but something
> is better than nothing.
>
> --Phil
>
> # Re sending this as i was not subscribed to the user mailing list and
> thing it could be useful there too.
> make-component
> ==============
>
> Author: Philip Herron <ph...@wandisco.com>
> Date: 2013-03-27 13:09:35 GMT
>
>
> Table of Contents
> =================
> 1 Bigtop Component Tutorial
> 1.1 RPM
>
>
> 1 Bigtop Component Tutorial
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> 1.1 RPM
> ========
> Creating new Components for bigtop is very simple lets say your package is called MY_HADOOP_COMPONENT
>
> # add your archive for your component
> $ $EDITOR Makefile
> APACHE_MIRROR ?= http://apache.osuosl.org
> # ..
> MY_ARCHIVE ?= http://myarchive.com/dist/
>
> $ $EDITOR bigtop.mk
> # add this to your BOM file bigtop.mk
> MYHADCOMP_NAME=MY_HADOOP_COMPONENT
> MYHADCOMP_RELNOTES_NAME=My Vedor string notes
> MYHADCOMP_BASE_VERSION=1.0-SNAPSHOT
> MYHADCOMP_PKG_VERSION=0.2+myvendorstring
> MYHADCOMP_RELEASE_VERSION=1
> MYHADCOMP_TARBALL_DST=MY_HADDOP_COMPONENT-$(MYHADCOMP_BASE_VERSION).tar.gz
> MYHADCOMP_TARBALL_SRC=$(MYHADCOMP_TARBALL_DST)
> MYHADCOMP_DOWNLOAD_PATH=/$(HDFS_TARBALL_SRC)
> MYHADCOMP_SITE=$(MY_ARCHIVE)$(MYHADCOMP_DOWNLOAD_PATH)
> MYHADCOMP_ARCHIVE=$(MYHADCOMP_SITE)
> $(eval $(call PACKAGE,MY_HADOOP_COMPONENT,MYHADCOMP))
>
>
> This BOM config will be looking for:
>
> [http://myarchive.com/dist/MY\_HADOOP\_COMPONENT-1.0-SNAPSHOT.tar.gz]
> And will download to the DST specified. This is useful because one you have this done to upgrade
> versions all you do is edit the bom file and specify the new version and bigtop takes care of the rest
>
> Now you need to add in all your work to make the packages build. The idiom in bigtop is to have a {common,rpm,deb}
> with this you have a common dir with files affecting both rpm and deb builds. We usualy had a do-component-build
>
> So lets create the dir structure you need for MY_HADOOP_COMPONENT
>
> $ mkdir bigtop-packages/src/{common,rpm}/MY_HADOOP_COMPONENT
> $ $EDITOR bigtop-packages/src/common/do-component-build
> # this is just bash within this file so it might simply be:
>
> #!/bin/bash
> set -ex
> . `dirname $0`/bigtop.bom
>
> ant -Dhadoop.version=${HADOOP_VERSION} dist
> EOF
>
> $ mkdir bigtop-packages/src/rpm/MY_HADOOP_COMPONENT/SPECS
> $ $EDITOR bigtop-packages/src/rpm/MY_HADOOP_COMPONENT/SPECS/MY_HADOOP_COMPONENT.spec
> Name: MY_HADOOP_COMPONENT
> Version: %{MY_HADOOP_COMPONENT_version}
> Release: %{MY_HADOOP_COMPONENT_release}
> Summary: summary of what this is
>
> Group: Development/Libraries
> License: Your license
> URL: http://wandisco.com
> Source0: MY_HADOOP_COMPONENT-%{MY_HADOOP_COMPONENT_base_version}.tar.gz
> Source1: do-component-build
> BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
>
> #BuildRequires:
> Requires: hadoop-hdfs.....
>
> AutoReq: no
>
> %define lib_hadoop_dirname /usr/lib
> %define lib_hdfs %{lib_hadoop_dirname}/hadoop-hdfs
>
> %description
> longer description of your package
>
> %prep
> %setup -n MY_HADOOP_COMPONENT-%{MY_HADOOP_COMPONENT_base_version}
>
> %build
> bash %{SOURCE1}
>
> %install
> rm -rf $RPM_BUILD_ROOT
>
> %__install -d -m 0755 $RPM_BUILD_ROOT/%{lib_hdfs}
> %__install -m 0755 dist/lib/MY_HADOOP_COMPONENT-%{MY_HADOOP_COMPONENT_base_version}.jar $RPM_BUILD_ROOT/%{lib_hdfs}
> %__install -m 0644 %{SOURCE2} $RPM_BUILD_ROOT/%{l
>
> %clean
> rm -rf $RPM_BUILD_ROOT
>
> %files
> %defattr(-,root,root,-)
> %{lib_hdfs}
>
> %changelog
>
> EOF
>
> # Make sure you keep the naming proper as bigtop will be looking for the same name as specified within MYHADCOMP_NAME in your BOM file.
>
>
> Now you should be able to run:
>
> make MY_HADOOP_COMPONENT-rpm
>
>
> Done!
>
> [http://myarchive.com/dist/MY\_HADOOP\_COMPONENT-1.0-SNAPSHOT.tar.gz]: http://myarchive.com/dist/MY_HADOOP_COMPONENT-1.0-SNAPSHOT.tar.gz
>
Re: Bigtop Component Tutorial
Posted by Konstantin Boudnik <co...@apache.org>.
Thanks Phil for taking a stab at it.
I believe this is a very good start for the collection of docs - development
and otherwise - which should belong to docs/ folder in the source tree.
Would you mind opening a JIRA for documentation improvement and submitting
this is as a patch?
Thanks,
Cos
On Wed, Mar 27, 2013 at 01:27PM, Philip Herron wrote:
> Hey all,
>
> I forgot about this but i had written some documentation and didn't
> finish but this might be useful to put somewhere:
>
> To make rpm components within bigtop its not written 100% but something
> is better than nothing.
>
> --Phil
>
> # Re sending this as i was not subscribed to the user mailing list and
> thing it could be useful there too.
> make-component
> ==============
>
> Author: Philip Herron <ph...@wandisco.com>
> Date: 2013-03-27 13:09:35 GMT
>
>
> Table of Contents
> =================
> 1 Bigtop Component Tutorial
> 1.1 RPM
>
>
> 1 Bigtop Component Tutorial
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> 1.1 RPM
> ========
> Creating new Components for bigtop is very simple lets say your package is called MY_HADOOP_COMPONENT
>
> # add your archive for your component
> $ $EDITOR Makefile
> APACHE_MIRROR ?= http://apache.osuosl.org
> # ..
> MY_ARCHIVE ?= http://myarchive.com/dist/
>
> $ $EDITOR bigtop.mk
> # add this to your BOM file bigtop.mk
> MYHADCOMP_NAME=MY_HADOOP_COMPONENT
> MYHADCOMP_RELNOTES_NAME=My Vedor string notes
> MYHADCOMP_BASE_VERSION=1.0-SNAPSHOT
> MYHADCOMP_PKG_VERSION=0.2+myvendorstring
> MYHADCOMP_RELEASE_VERSION=1
> MYHADCOMP_TARBALL_DST=MY_HADDOP_COMPONENT-$(MYHADCOMP_BASE_VERSION).tar.gz
> MYHADCOMP_TARBALL_SRC=$(MYHADCOMP_TARBALL_DST)
> MYHADCOMP_DOWNLOAD_PATH=/$(HDFS_TARBALL_SRC)
> MYHADCOMP_SITE=$(MY_ARCHIVE)$(MYHADCOMP_DOWNLOAD_PATH)
> MYHADCOMP_ARCHIVE=$(MYHADCOMP_SITE)
> $(eval $(call PACKAGE,MY_HADOOP_COMPONENT,MYHADCOMP))
>
>
> This BOM config will be looking for:
>
> [http://myarchive.com/dist/MY\_HADOOP\_COMPONENT-1.0-SNAPSHOT.tar.gz]
> And will download to the DST specified. This is useful because one you have this done to upgrade
> versions all you do is edit the bom file and specify the new version and bigtop takes care of the rest
>
> Now you need to add in all your work to make the packages build. The idiom in bigtop is to have a {common,rpm,deb}
> with this you have a common dir with files affecting both rpm and deb builds. We usualy had a do-component-build
>
> So lets create the dir structure you need for MY_HADOOP_COMPONENT
>
> $ mkdir bigtop-packages/src/{common,rpm}/MY_HADOOP_COMPONENT
> $ $EDITOR bigtop-packages/src/common/do-component-build
> # this is just bash within this file so it might simply be:
>
> #!/bin/bash
> set -ex
> . `dirname $0`/bigtop.bom
>
> ant -Dhadoop.version=${HADOOP_VERSION} dist
> EOF
>
> $ mkdir bigtop-packages/src/rpm/MY_HADOOP_COMPONENT/SPECS
> $ $EDITOR bigtop-packages/src/rpm/MY_HADOOP_COMPONENT/SPECS/MY_HADOOP_COMPONENT.spec
> Name: MY_HADOOP_COMPONENT
> Version: %{MY_HADOOP_COMPONENT_version}
> Release: %{MY_HADOOP_COMPONENT_release}
> Summary: summary of what this is
>
> Group: Development/Libraries
> License: Your license
> URL: http://wandisco.com
> Source0: MY_HADOOP_COMPONENT-%{MY_HADOOP_COMPONENT_base_version}.tar.gz
> Source1: do-component-build
> BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
>
> #BuildRequires:
> Requires: hadoop-hdfs.....
>
> AutoReq: no
>
> %define lib_hadoop_dirname /usr/lib
> %define lib_hdfs %{lib_hadoop_dirname}/hadoop-hdfs
>
> %description
> longer description of your package
>
> %prep
> %setup -n MY_HADOOP_COMPONENT-%{MY_HADOOP_COMPONENT_base_version}
>
> %build
> bash %{SOURCE1}
>
> %install
> rm -rf $RPM_BUILD_ROOT
>
> %__install -d -m 0755 $RPM_BUILD_ROOT/%{lib_hdfs}
> %__install -m 0755 dist/lib/MY_HADOOP_COMPONENT-%{MY_HADOOP_COMPONENT_base_version}.jar $RPM_BUILD_ROOT/%{lib_hdfs}
> %__install -m 0644 %{SOURCE2} $RPM_BUILD_ROOT/%{l
>
> %clean
> rm -rf $RPM_BUILD_ROOT
>
> %files
> %defattr(-,root,root,-)
> %{lib_hdfs}
>
> %changelog
>
> EOF
>
> # Make sure you keep the naming proper as bigtop will be looking for the same name as specified within MYHADCOMP_NAME in your BOM file.
>
>
> Now you should be able to run:
>
> make MY_HADOOP_COMPONENT-rpm
>
>
> Done!
>
> [http://myarchive.com/dist/MY\_HADOOP\_COMPONENT-1.0-SNAPSHOT.tar.gz]: http://myarchive.com/dist/MY_HADOOP_COMPONENT-1.0-SNAPSHOT.tar.gz
>