You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@bigtop.apache.org by Fabio <fa...@pirola.org> on 2015/08/12 00:49:57 UTC
Questions about debian build and packages
Hi all,
I am a new bigtop user and I am trying to create my docker image from
**scratch** based on debian-testing and bigtop (I see that you have
publish docker images but I prefer to create a new one from my own).
I wrote a simple wiki in order to create a debian-testing docker image
from scratch.
Please find:
- in attachment debug log from my docker image;
- a little wiki in the bottom of the email.
I have two questions:
1. How can I create debian packages? I searched with "find . -type f
-name *.deb" but I have not found anything
2. I found warnings in the build log file:
Warning: Waiting for output; will sleep 0.0 seconds
Warning: Waiting for output; will sleep 0.5 seconds
Warning: Could not get output
Is it something strange happened or is it okay?
Thanks for your work
Regards
Fabio
***
# Instructions
## Create docker image
1. Install Linux container
# apt-get install lxc
2. Install a Debian base system into debian-testing subdirectory
# debootstrap testing debian-testing/ http://http.debian.net/debian/
Note: to locate a mirror near you, see the list of [Debian worldwide
mirror sites](https://www.debian.org/mirror/list).
Note-2: it is possible to include packages using debootstrap but I found
an error with the following command (I will analyze&open a bug report to
debian project)
# debootstrap
--include=aptitude,htop,iotop,sudo,puppet,curl,gradle,git,maven,openjdk-7-jdk,openjdk-7-jre,libxslt1-dev,libkrb5-dev,libldap2-dev,libmysqlclient-dev,libsasl2-dev,libsqlite3-dev,libxml2-dev,python-dev,python-setuptools,liblzo2-dev,libzip-dev,libfuse-dev,libssl-dev,build-essential,dh-make,debhelper,devscripts,reprepro
testing debian-testing/ http://http.debian.net/debian/
3. Create tar archive
# cd debian-testing && tar cf ../debian-testing.tar .
4. Create an empty filesystem image and import the contents of the tarball
# cd .. && docker import - debian:testing-bigtop < debian-testing.tar
5. Check image has been correctly imported
$ docker images
REPOSITORY TAG IMAGE ID CREATED
VIRTUAL SIZE
debian testing-bigtop 907580971b7e 9 seconds
ago 283.2 MB
6. A simple run test
$ docker run -i -t debian:testing-bigtop /bin/bash
root@a855d3b5e5b9:/# uname -a
Linux a855d3b5e5b9 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1
(2015-05-24) x86_64 GNU/Linux
## Install Bigtop on docker image
1. Open docker image
$ docker run -i -t debian:testing-bigtop /bin/bash
2. Resynchronize the package index files && install the newest versions
of all packages currently installed on the system
root@6f1894cdcbee:/# apt-get update && apt-get upgrade
3. Install all dependencies
root@6f1894cdcbee:/# apt-get install aptitude htop iotop sudo puppet
curl gradle git maven openjdk-7-jdk openjdk-7-jre libxslt1-dev
libkrb5-dev libldap2-dev libmysqlclient-dev libsasl2-dev libsqlite3-dev
libxml2-dev python-dev python-setuptools liblzo2-dev libzip-dev
libfuse-dev libssl-dev build-essential dh-make debhelper devscripts reprepro
4. Open root directory
root@6f1894cdcbee:/# cd root
5. Download latest version of Bigtop
root@6f1894cdcbee:/# git clone
https://git-wip-us.apache.org/repos/asf/bigtop.git && cd bigtop
6. Create and install packages
root@6f1894cdcbee:/# ./gradlew toolchain --debug
&>/root/build_bigtop_`date +%Y%m%d`"_"`date +%H%M%S`.log
***
Some useful links:
[Debian - A brief introduction to using
docker](https://www.debian-administration.org/article/696/A_brief_introduction_to_using_docker#a2)
[Debian - Debootstrap](https://wiki.debian.org/Debootstrap)
Re: Questions about debian build and packages
Posted by Fabio <fa...@pirola.org>.
Hi Olaf,
Il 19 agosto 2015 21:09:26 CEST, Olaf Flebbe <of...@oflebbe.de> ha scritto:
>
>The recommended way to install is ./gradlew toolchain
>
>Best Regards,
> Olaf
Thanks for your answers
Best Regards
Fabio
Re: Questions about debian build and packages
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi Fabio,
> Il 12/08/2015 15:59, Olaf Flebbe ha scritto:
>> Please refer to
>> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop
> Thanks for the link.
> I found a page "Building Bigtop on Ubuntu"
> (https://cwiki.apache.org/confluence/display/BIGTOP/Building+Bigtop+on+Ubuntu)
> that probably it needs some update.
The Header of this page reads "Outdated and deprecated. "
> For example:
> - advice to use http://www.apache.org/dist/bigtop/bigtop-1.0.0/repos/
> (refer to
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop
> - probably also this page need some update)
> - change from "make" to "./gradlew" .
This page describes Bigtop 0.3.0 , I changed the wiki topic and the link from the cwiki home page)
>
>> The packaging.gradle assembles sdeb files with are used with debuild .
> I use the following commands to assemble deb files (see
> https://cwiki.apache.org/confluence/display/BIGTOP/Building+Bigtop+on+Ubuntu):
> ./gradlew realclean
> ./gradlew bigtop-utils-deb # build this project first
> ./gradlew hadoop-deb # to build just for hadoop first
> ./gradlew deb # build all the rest
> Is there a better way to assemble sdeb files?
./gradlew clean hadoop-deb bigtop-utils-deb
should work too
>
>>> root@6f1894cdcbee:/# apt-get install aptitude htop iotop sudo puppet
>>> curl gradle git maven openjdk-7-jdkopenjdk-7-jre libxslt1-dev
>>> libkrb5-dev libldap2-dev libmysqlclient-dev libsasl2-dev libsqlite3-dev
>>> libxml2-dev python-dev python-setuptools liblzo2-dev libzip-dev
>>> libfuse-dev libssl-dev build-essential dh-make debhelper devscripts reprepro
>>
>> It should be sufficient to install curl and openjdk-7-jre-headless (the gradle dependencies).
> I used a dependencies list on bigtop
> homepage but now the page has been updated for the new release.
>
>
The recommended way to install is ./gradlew toolchain
Best Regards,
Olaf
Re: Questions about debian build and packages
Posted by Fabio <fa...@pirola.org>.
Hi Olaf,
Il 12/08/2015 15:59, Olaf Flebbe ha scritto:
> Please refer to
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop
Thanks for the link.
I found a page "Building Bigtop on Ubuntu"
(https://cwiki.apache.org/confluence/display/BIGTOP/Building+Bigtop+on+Ubuntu)
that probably it needs some update.
For example:
- advice to use http://www.apache.org/dist/bigtop/bigtop-1.0.0/repos/
(refer to
https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop
- probably also this page need some update)
- change from "make" to "./gradlew" .
> The packaging.gradle assembles sdeb files with are used with debuild .
I use the following commands to assemble deb files (see
https://cwiki.apache.org/confluence/display/BIGTOP/Building+Bigtop+on+Ubuntu):
./gradlew realclean
./gradlew bigtop-utils-deb # build this project first
./gradlew hadoop-deb # to build just for hadoop first
./gradlew deb # build all the rest
Is there a better way to assemble sdeb files?
>> root@6f1894cdcbee:/# apt-get install aptitude htop iotop sudo puppet
>> curl gradle git maven openjdk-7-jdkopenjdk-7-jre libxslt1-dev
>> libkrb5-dev libldap2-dev libmysqlclient-dev libsasl2-dev libsqlite3-dev
>> libxml2-dev python-dev python-setuptools liblzo2-dev libzip-dev
>> libfuse-dev libssl-dev build-essential dh-make debhelper devscripts reprepro
>
> It should be sufficient to install curl and openjdk-7-jre-headless (the gradle dependencies).
I used a dependencies list on bigtop
homepage but now the page has been updated for the new release.
Best Regards
Fabio
Re: Questions about debian build and packages
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi,
Please refer to
https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop
The packaging is not debian conforming since we need a special environment installed with "gradlew toolchain."
The Source DEB files will be assembled by packages.gradle
> Am 12.08.2015 um 00:49 schrieb Fabio <fa...@pirola.org>:
>
> Hi all,
>
> I am a new bigtop user and I am trying to create my docker image from
> **scratch** based on debian-testing and bigtop (I see that you have
> publish docker images but I prefer to create a new one from my own).
> I wrote a simple wiki in order to create a debian-testing docker image
> from scratch.
> Please find:
> - in attachment debug log from my docker image;
> - a little wiki in the bottom of the email.
>
> I have two questions:
>
> 1. How can I create debian packages? I searched with "find . -type f
> -name *.deb" but I have not found anything
The packaging.gradle assembles sdeb files with are used with debuild .
>
> 2. I found warnings in the build log file:
> Warning: Waiting for output; will sleep 0.0 seconds
> Warning: Waiting for output; will sleep 0.5 seconds
> Warning: Could not get output
> Is it something strange happened or is it okay?
>
>
There are a few place where this can happen but only for tenth's of seconds, since invoked commands will sometimes work a bit longer.
> Thanks for your work
> Regards
>
> Fabio
>
>
> ***
> # Instructions
>
> ## Create docker image
> 1. Install Linux container
> # apt-get install lxc
>
> 2. Install a Debian base system into debian-testing subdirectory
> # debootstrap testing debian-testing/ http://http.debian.net/debian/
> Note: to locate a mirror near you, see the list of [Debian worldwide
> mirror sites](https://www.debian.org/mirror/list).
> Note-2: it is possible to include packages using debootstrap but I found
> an error with the following command (I will analyze&open a bug report to
> debian project)
> # debootstrap
> --include=aptitude,htop,iotop,sudo,puppet,curl,gradle,git,maven,openjdk-7-jdk,openjdk-7-jre,libxslt1-dev,libkrb5-dev,libldap2-dev,libmysqlclient-dev,libsasl2-dev,libsqlite3-dev,libxml2-dev,python-dev,python-setuptools,liblzo2-dev,libzip-dev,libfuse-dev,libssl-dev,build-essential,dh-make,debhelper,devscripts,reprepro
> testing debian-testing/ http://http.debian.net/debian/
>
> 3. Create tar archive
> # cd debian-testing && tar cf ../debian-testing.tar .
>
> 4. Create an empty filesystem image and import the contents of the tarball
> # cd .. && docker import - debian:testing-bigtop < debian-testing.tar
>
> 5. Check image has been correctly imported
> $ docker images
> REPOSITORY TAG IMAGE ID CREATED
> VIRTUAL SIZE
> debian testing-bigtop 907580971b7e 9 seconds
> ago 283.2 MB
>
> 6. A simple run test
> $ docker run -i -t debian:testing-bigtop /bin/bash
> root@a855d3b5e5b9:/# uname -a
> Linux a855d3b5e5b9 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1
> (2015-05-24) x86_64 GNU/Linux
>
>
> ## Install Bigtop on docker image
> 1. Open docker image
> $ docker run -i -t debian:testing-bigtop /bin/bash
>
> 2. Resynchronize the package index files && install the newest versions
> of all packages currently installed on the system
> root@6f1894cdcbee:/# apt-get update && apt-get upgrade
>
> 3. Install all dependencies
> root@6f1894cdcbee:/# apt-get install aptitude htop iotop sudo puppet
> curl gradle git maven openjdk-7-jdkopenjdk-7-jre libxslt1-dev
> libkrb5-dev libldap2-dev libmysqlclient-dev libsasl2-dev libsqlite3-dev
> libxml2-dev python-dev python-setuptools liblzo2-dev libzip-dev
> libfuse-dev libssl-dev build-essential dh-make debhelper devscripts reprepro
It should be sufficient to install curl and openjdk-7-jre-headless (the gradle dependencies).
>
> 4. Open root directory
> root@6f1894cdcbee:/# cd root
>
> 5. Download latest version of Bigtop
> root@6f1894cdcbee:/# git clone
> https://git-wip-us.apache.org/repos/asf/bigtop.git && cd bigtop
>
> 6. Create and install packages
> root@6f1894cdcbee:/# ./gradlew toolchain --debug
> &>/root/build_bigtop_`date +%Y%m%d`"_"`date +%H%M%S`.log
>
>
> ***
> Some useful links:
> [Debian - A brief introduction to using
> docker](https://www.debian-administration.org/article/696/A_brief_introduction_to_using_docker#a2)
> [Debian - Debootstrap](https://wiki.debian.org/Debootstrap)
>
>
> <build_bigtop_20150811_212148.log.tar.gz>