Re: Bigtop Docker image with Dockerfile.

[Olaf Flebbe <o....@science-computing.de>]

hi 

a jenkins user is created by the toolchain installer. see bigtop_toolchain/readme.md

Von meinem iPhone gesendet

Am 27.02.2015 um 01:22 schrieb Christian Tzolov <christian.tzolov@gmail.com <ma...@gmail.com> >:

@Peter, it is clear why Hue needs not-root user :) The point is that a non-root, sudo, passwordless user is not provided OOTB in the core Docker image (at least not in centos6). So one has to take care to create one in order to prepare the bigtop environment. 
My goal is to create a BigTop environment with a single command. So if the creation of non-root user is not done in the Dockerfile then it needs to be taken care of in the toolchain gradle script i guess. 

@Roman, thanks for comment. I'll ping you on the messenger sometime tomorrow. 

Cheers, Chris


On Fri, Feb 27, 2015 at 12:18 AM, jay vyas <jayunit100.apache@gmail.com <ma...@gmail.com> > wrote:
Thanks peter !  If its a bad thing, then lets disable the possibility of building the packs as root. 

Do you think it Would be a good JIRA to have (confirm that all packages build only as non-root).  

On Thu, Feb 26, 2015 at 6:07 PM, Peter Linnell <plinnell@apache.org <ma...@apache.org> > wrote:
On Wed, 25 Feb 2015 18:57:02 +0100
Christian Tzolov <christian.tzolov@gmail.com <ma...@gmail.com> > wrote:

> Hi Konstantin,
>
> Below are some of my findings. Note that so far i've played only with
> CentOS6.
>
> - Missing python-setuptools and asciidoc packages. Don't remember
> which project needed them.
> - To build spark-rpm you need install scala and set the SCALA_HOME
> env.
> - Due to https://issues.apache.org/jira/browse/CRUNCH-500 you can't
> use Maven > 3.1. I've fixed this in the Crunch project but will not be
> available until Crunch 0.12 release. For the time being can create a
> patch for Crunch's do-component-build? In the Dockerfile I've
> downgraded maven to 3.0.5
> - Hue (hue-ui) requires non root user to build the rpm.

Hue wanting a non-root user to build is a feature not a bug :)

I've learned the hard way in packaging: *Never* ever build packages as
root.


>
> ----
>
> I've tried
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop
> using the centos image. It seems like the /etc/profile.d/bigtop.sh is
> missing? The 'toolchain' requires java to be installed first (and
> JAVA_HOME to be set). This in turn requires 'sudo' package. For Spark
> you need to install Scala and set SCALA_HOME. And for Hue you need to
> create sudo, passwordless user...
> Maybe with the ubunto image is better (haven't tried it) but for me
> the above requires too many steps to prepare the environment.
>
> Hope this helps.
>
> Cheers,
> Christian
>
>
>
>
>
> On Wed, Feb 25, 2015 at 4:12 AM, Konstantin Boudnik <cos@apache.org <ma...@apache.org> >
> wrote:
>
> > Hi Christian.
> >
> > On Tue, Feb 24, 2015 at 09:24AM, Christian Tzolov wrote:
> > > Hi Konstantin,
> > >
> > > I see your point. My goal however was to generate RPMs for some
> > > projects (Spark particularly) and experiment with different
> > > versions and configurations.
> > >
> > > I could not find a proper doc explaining how to bootstrap with
> > > Bigtop.
> > The
> > > wiki pages like (
> > >
> > https://cwiki.apache.org/confluence/display/BIGTOP/Building+Bigtop+on+RHEL7
> > )
> > > are outdated, missing libraries and stil refering to the 'make'
> > > tool.
> >
> > Yup, it is - I just recently figured this out and filed the JIRA
> >    https://issues.apache.org/jira/browse/BIGTOP-1665
> > which you're welcome to take, if you want.
> >
> > > I've tried the './gradlew toolchain' to build an environment but
> > > it still requires some libs to be preinstalled so i've decided to
> > > keep everything
> > in
> > > a single Dockerfile.
> >
> > Could you please share what libs were missing? It'd be great to get
> > it fixed.
> > The issue here is the we are moving forward quite fast and
> > sometimes devs env.
> > isn't exactly pristine, so think like missing packages might go
> > unnoticed.
> >
> > > Concerning only the packaging tasks I guess 2 environments (for
> > > RPM and DEB) are enough?
> >
> > Yes, that's all Bigtop does on the packaging side.
> >
> > I have put together an initial draft for new build procedure
> >   https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop
> >
> > and you're welcome to update/add to it. Hope it helps.
> >
> >   Cos
> >
> > > On Mon, Feb 23, 2015 at 8:39 PM, Konstantin Boudnik
> > > <cos@apache.org <ma...@apache.org> >
> > wrote:
> > >
> > > > Thanks Christian.
> > > >
> > > > We have considered this on a multiple occasions and as you
> > > > found for yourself
> > > > the Dockerfile file is pretty simple, so the consensus was to
> > > > just check-in a
> > > > Gradle code for its generation, hence making containers
> > > > preparation as stateless as possible.
> > > >
> > > > Roman should have this patch somewhere, not sure if he's ready
> > > > to
> > commit
> > > > yet.
> > > > But we always can use your approach is a backup plan, although
> > > > we'd
> > need
> > > > Dockerfiles for all supported Linuxes.
> > > >
> > > >   Cos
> > > >
> > > > On Mon, Feb 23, 2015 at 07:45PM, Christian Tzolov wrote:
> > > > >    I've put together a Docker container, equipped with all
> > > > > necessary
> > > > tools
> > > > >    and libraries and ready to generate RPM packages. Here are
> > > > > the Dockerhub/Github references:
> > > > >    https://registry.hub.docker.com/u/tzolov/bigtop-centos
> > > > >    https://github.com/tzolov/bigtop-centos
> > > > >    There are a couple of existing bigtop images in Dockerhub
> > > > > but
> > neither
> > > > of
> > > > >    them provides neither description nor a Dockerfile.
> > > > >
> > > > >    For simplicity I've decided to keep it as as single image
> > > > > for now. Cheers,
> > > > >    Christian
> > > >
> >
> >

Hope that helps,
Peter



-- 
jay vyas

-- 
Vorstandsvorsitzender/Chairman of the board of management:
Gerd-Lothar Leonhart
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Michael Heinrichs, Dr. Arno Steitz
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Philippe Miltin
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196