[jira] [Updated] (MESOS-3462) Containerization issues with mesos running on CoreOS

["haosdent (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MESOS-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

haosdent updated MESOS-3462:
----------------------------
    Assignee:     (was: haosdent)

> Containerization issues with mesos running on CoreOS
> ----------------------------------------------------
>
>                 Key: MESOS-3462
>                 URL: https://issues.apache.org/jira/browse/MESOS-3462
>             Project: Mesos
>          Issue Type: Bug
>          Components: containerization
>    Affects Versions: 0.24.0
>         Environment: CoreOS 801.0.0 64-bit
>            Reporter: Francis Chuang
>
> These are the steps to I used to build mesos 0.24.0 on Ubuntu 15.04 64-bit:
> wget http://www.apache.org/dist/mesos/0.24.0/mesos-0.24.0.tar.gz
> wget http://mirror.ventraip.net.au/apache/apr/apr-1.5.2.tar.gz
> wget http://mirror.ventraip.net.au/apache/apr/apr-util-1.5.4.tar.gz
> wget http://mirror.ventraip.net.au/apache/subversion/subversion-1.9.0.tar.gz
> wget http://www.sqlite.org/sqlite-amalgamation-3071501.zip
> wget ftp://ftp.cyrusimap.org/cyrus-sasl/cyrus-sasl-2.1.26.tar.gz
> mkdir /tmp/mesos-build
> cd /tmp/mesos-build
> - Build apr
> tar zxf apr-$APR_VERSION.tar.gz
> cd apr-$APR_VERSION
> ./configure CC=gcc-4.8 --prefix=/tmp/mesos-build/apr
> make
> make install
> cd ..
> - Build apr-util
> tar zxf apr-util-$APR_UTIL_VERSION.tar.gz
> cd apr-util-$APR_UTIL_VERSION
> ./configure CC=gcc-4.8 --prefix=/tmp/mesos-build/apr-util --with-apr=/tmp/mesos-build/apr
> make
> make install
> cd ..
> - Build libsasl2
> tar zxf cyrus-sasl-$SASL_VERSION.tar.gz
> cd cyrus-sasl-$SASL_VERSION
> ./configure CC=gcc-4.8 CPPFLAGS=-I/usr/include/openssl --prefix=/tmp/mesos-build/sasl2 --enable-cram
> make
> make install
> cd ..
> - Build subversion
> tar zxf subversion-$SVN_VERSION.tar.gz
> unzip sqlite-amalgamation-$SQLITE_AMALGATION_VERSION.zip
> mv sqlite-amalgamation-$SQLITE_AMALGATION_VERSION/ subversion-$SVN_VERSION/sqlite-amalgamation/
> cd subversion-$SVN_VERSION
> ./configure CC=gcc-4.8 CXX=g++-4.8 --prefix=/tmp/mesos-build/svn --with-apr=/tmp/mesos-build/apr --with-apr-util=/tmp/mesos-build/apr-util --with-sasl=/tmp/mesos-build/sasl2
> make
> make install
> cd ..
> - Build curl
> tar zxf curl-$CURL_VERSION.tar.gz
> cd curl-$CURL_VERSION
> ./configure CC=gcc-4.8 --prefix=/tmp/mesos-build/curl
> make
> make install
> cd ..
> - Build mesos
> tar zxf mesos-$MESOS_VERSION.tar.gz
> cd mesos-$MESOS_VERSION
> mkdir build
> cd build
> ../configure CC=gcc-4.8 CXX=g++-4.8 LD_LIBRARY_PATH=/tmp/mesos-build/sasl2/lib SASL_PATH=/tmp/mesos-build/sasl2/lib/sasl2 --prefix=/tmp/mesos-build/mesos --with-svn=/tmp/mesos-build/svn --with-apr=/tmp/mesos-build/apr --with-sasl=/tmp/mesos-build/sasl2/ --with-curl=/tmp/mesos-build/curl
> make
> make install
> cd ..
> cd ..
> - Copy shared objects into mesos build
> cp apr/lib/libapr-1.so.0.5.2 mesos/lib/libapr-1.so.0
> cp apr-util/lib/libaprutil-1.so.0.5.4 mesos/lib/libaprutil-1.so.0
> cp sasl2/lib/libsasl2.so.3.0.0 mesos/lib/libsasl2.so.3
> cp svn/lib/libsvn_delta-1.so.0.0.0 mesos/lib/libsvn_delta-1.so.0
> cp svn/lib/libsvn_subr-1.so.0.0.0 mesos/lib/libsvn_subr-1.so.0
> I then compress the build into an archive and distributed it onto my CoreOS nodes.
> Once I have the archive extracted on each node, I start the master and slaves:
> /opt/mesos/sbin/mesos-master --zk=zk://192.168.33.10/mesos --quorum=1 --hostname=192.168.33.10 --ip=192.168.33.10 --webui_dir=/opt/mesos/share/mesos/webui --cluster=mesos
> /opt/mesos/sbin/mesos-slave --hostname=192.168.33.11 --ip=192.168.33.11 --master=zk://192.168.33.10/mesos --executor_environment_variables='{"LD_LIBRARY_PATH": "/opt/mesos/lib", "PATH": "/opt/java/bin:/usr/sbin:/usr/bin"}' --containerizers=docker,mesos --executor_registration_timeout=60mins --launcher_dir=/opt/mesos/libexec/mesos/
> In addition, the following environment variables are set:
> LD_LIBRARY_PATH=/opt/mesos/lib/
> JAVA_HOME=/opt/java
> MESOS_NATIVE_JAVA_LIBRARY=/opt/mesos/lib/libmesos.so
> I am finding that when I run meso-hdfs from https://github.com/mesosphere/hdfs, the scheduler starts properly and launches the executors. However, the executors will fail and terminate without writing any error to stderr and stdout.
> I have reproduced the same problem with mesos 0.24, 0.23 and 0.22.1
> If I install mesos onto a Ubuntu machine (tried 14.04 and 15.04 64-bit) using the apt-repositories, this problem does not happen.
> I am not well-versed with mesos internals, but it was pointed out that it's most likely a containerization issue. This issue on github documents the process of trying to get mesos-hdfs to work on my compiled mesos binaries: https://github.com/mesosphere/hdfs/issues/194



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)