[jira] [Commented] (BIGTOP-3343) Add Debian 10 and Ubuntu 18.04 support to the Docker provisioner

["Luca Toscano (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17088878#comment-17088878 ] 

Luca Toscano commented on BIGTOP-3343:
--------------------------------------

Really nice testing results! I can confirm point 4., the only openjdk available is 11 and it is not really an option for Hadoop (we, as Wikimedia, backported openjdk-8 to Buster rebuilding it manually, but adopted jdk should be fine as well).

> Add Debian 10 and Ubuntu 18.04 support to the Docker provisioner
> ----------------------------------------------------------------
>
>                 Key: BIGTOP-3343
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-3343
>             Project: Bigtop
>          Issue Type: Sub-task
>          Components: debian, docker, provisioner
>            Reporter: Kengo Seki
>            Assignee: Kengo Seki
>            Priority: Major
>
> I tried to use bigtop/puppet:trunk-debian-10 and bigtop/puppet:trunk-ubuntu-18.04 with the Docker provisioner and found some problems:
> 1. The systemd-sysv package has to be installed to ubuntu-18.04 in bigtop_toolchain/bin/puppetize.sh, otherwise the provisioner fails as follows:
> {code}
> $ ./docker-hadoop.sh -c 1 -C config_ubuntu-18.04.yaml 
> (snip)
> Creating 20200421223222r15703_bigtop_1 ... error
> ERROR: for 20200421223222r15703_bigtop_1  Cannot start service bigtop: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/sbin/init\": stat /sbin/init: no such file or directory": unknown
> ERROR: for bigtop  Cannot start service bigtop: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/sbin/init\": stat /sbin/init: no such file or directory": unknown
> ERROR: Encountered errors while bringing up the project.
> {code}
> 2. {{APT::Get::AllowUnauthenticated 1;}} in bigtop-deploy/puppet/manifests/bigtop_repo.pp doesn't seem to work on these distros. It seems to be replaced with {{Acquire::AllowInsecureRepositories "true";}}.
> 3. The gnupg package has to be installed to debian-10 in bigtop_toolchain/bin/puppetize.sh, otherwise the provisioner fails as follows:
> {code}
> $ ./docker-hadoop.sh -c 1 -C config_debian-10.yaml 
> (snip)
> Error: Could not find a suitable provider for apt_key
> Notice: Applied catalog in 0.39 seconds
> [LOG] Failed to provision container a1eddad59e7f0ff1cbfd56d09ea9d78503676fbdff157ad85d95c6819b03e3fc with exit code 6
> {code}
> 4. The existing puppet manifest (bigtop-deploy/puppet/manifests/jdk.pp) tries to install openjdk-8-jdk as follows, but it's not provided by the distro on debian-10. Instead, we have to install adoptopenjdk 8 just like bigtop_toolchain/manifests/jdk.pp.
> {code}
> $ ./docker-hadoop.sh -c 1 -C config_debian-10.yaml 
> (snip)
> Notice: /Stage[main]/Jdk/Apt::Pin[backports_jdk]/Apt::Setting[pref-backports_jdk]/File[/etc/apt/preferences.d/backports_jdk.pref]/ensure: defined content as '{md5}98444fb15be80221672742e79a046130'
> Notice: /Stage[main]/Jdk/Apt::Pin[backports_ca]/Apt::Setting[pref-backports_ca]/File[/etc/apt/preferences.d/backports_ca.pref]/ensure: defined content as '{md5}610a41d22559e3f19685788014d7d9df'
> Notice: /Stage[main]/Jdk/Exec[own_update]/returns: executed successfully
> Error: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install openjdk-8-jdk' returned 100: Reading package lists...
> Building dependency tree...
> Reading state information...
> E: Unable to locate package openjdk-8-jdk
> Error: /Stage[main]/Jdk/Package[jdk]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install openjdk-8-jdk' returned 100: Reading package lists...
> Building dependency tree...
> Reading state information...
> E: Unable to locate package openjdk-8-jdk
> Notice: /Stage[main]/Hadoop::Common/Package[hadoop]: Dependency Package[jdk] has failures: true
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)