You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@mesos.apache.org by Mark Hammons <ma...@inaf.cnrs-gif.fr> on 2016/10/17 12:21:49 UTC

Problems with shared libraries with mesos containerization....

Hi all,

I've been working with the mesos containerizer to handle my docker containers 
recently. I created a docker container that requires libtiff.so.5, and the 
application within it runs well. But when I try to run it within the mesos 
containerizer I get an error saying libtiff.so.5 doesn't exist.

The application is being launched via java's process mechanism from inside a 
java thread in a custom java executor if that makes a difference. 

Any idea what could be causing this change in behavior? As you can see in the 
attached log file, I check /usr/lib, and a symbolic link to /usr/lib/x86..../
libtiff.so.5 exists in /usr/lib so the program should be able to find and load 
that....
----
Mark Edgar Hammons II | +33 06 03 69 56 56

Re: Problems with shared libraries with mesos containerization....

Posted by Mark Hammons <ma...@inaf.cnrs-gif.fr>.

I've created the issue and attached a log file with all the info you asked for 
to it.

https://issues.apache.org/jira/browse/MESOS-6404

On Monday, October 17, 2016 8:42:13 AM CEST Avinash Sridharan wrote:
> Does look like the symlink exists:
> /usr/lib/libtiff.so.5
> 
> I am assuming you have checked the realpath exists as well (that's why was
> asking for `ls -al`) ?
> 
> Don't see that you have volume mounts that might obfuscate the path. Could
> you create a JIRA and if possible point give access to your docker image
> for us to try? Do mention the exact version of Mesos and the Distro you are
> trying to run this on.
> 
> -Avinash
> 
> On Mon, Oct 17, 2016 at 8:35 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> > wrote:
> > 
> > No, it's a regular linux log. I've reattached it.
> > 
> > On Monday, October 17, 2016 8:32:07 AM CEST Avinash Sridharan wrote:
> > > can't seem to open the attached logs, is it gzip?
> > > 
> > > On Mon, Oct 17, 2016 at 8:14 AM, Mark Hammons <
> > 
> > mark.hammons@inaf.cnrs-gif.fr
> > 
> > > > wrote:
> > > > 
> > > > Adding LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu doesn't fix the
> > 
> > error.
> > 
> > > > On Monday, October 17, 2016 5:06:34 PM CEST Mark Hammons wrote:
> > > > > As I've shown in the original logs, the symbolic link libtiff.so.5
> > 
> > is in
> > 
> > > > > /usr/ lib. I used LD_PRELOAD just to try to force
> > 
> > /usr/lib/libtiff.so.5
> > 
> > > > to
> > > > 
> > > > > be loaded since it wasn't for some reason.
> > > > > 
> > > > > On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan wrote:
> > > > > > Can you prepend your command with `ls -al /usr/lib` and check in
> > > > 
> > > > stdout if
> > > > 
> > > > > > you are seeing the shared object? By the way why are you using
> > > > 
> > > > LD_PRELOAD
> > > > 
> > > > > > instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to
> > 
> > load
> > 
> > > > your
> > > > 
> > > > > > library before any other libraries. It's usually used to override
> > > > 
> > > > system
> > > > 
> > > > > > libraries so found a bit odd that you are using it here?
> > > > > > 
> > > > > > On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons
> > > > > > <ma...@inaf.cnrs-gif.fr>
> > > > > > 
> > > > > > > wrote:
> > > > > > > 
> > > > > > > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for the
> > > > 
> > > > process
> > > > 
> > > > > > > environment it says it can't load /usr/lib/libtiff.so.5.
> > > > > > > 
> > > > > > > On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan
> > 
> > wrote:
> > > > > > > > Are you setting the env in the dockerfile?
> > > > > > > > 
> > > > > > > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
> > > > > > > 
> > > > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > > > 
> > > > > > > > > wrote:
> > > > > > > > > 
> > > > > > > > > Here's the code I define my executor to mesos with:
> > > > > > > > >         val iuwtURI = CommandInfo.URI.newBuilder().
> > > > 
> > > > setValue("http://
> > > > 
> > > > > > > ***/
> > > > > > > 
> > > > > > > > > IUWT.tar.gz").setExtract(true).setCache(false).build()
> > > > > > > > > 
> > > > > > > > >         val iuwtjURI =
> > > > > > > > >         CommandInfo.URI.newBuilder().setValue("http://
> > > > > > > 
> > > > > > > ***/
> > > > > > > 
> > > > > > > > > iuwtExecutor-assembly-0.1-
> > > > > > > > > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
> > > > > > > > > 
> > > > > > > > >         val iuwtExec = "java -jar
> > > > > > > > >         iuwtExecutor-assembly-0.1-SNAPSHOT.jar
> > > > > > > 
> > > > > > > -
> > > > > > > 
> > > > > > > > > Xmx1024M -Xmx128M"
> > > > > > > > > 
> > > > > > > > >         val iuwtCommand =
> > > > > > > > > 
> > > > > > > > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(
> > > > 
> > > > List(iuwtjURI,
> > > > 
> > > > > > > > > iuwtURI).asJava).setShell(true).build()
> > > > > > > > > 
> > > > > > > > >         val iuwtImageInfo =
> > > > > > > > > 
> > > > > > > > > Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> > > > > > > > > Image.Docker.newBuilder.setName("ubuntu-
> > > > > > > > > mesos:0.11-17102016-IUWT").build()).build()
> > > > > > > > > 
> > > > > > > > >         val iuwtContInfo =
> > > > > > > > > 
> > > > > > > > > ContainerInfo.MesosInfo.newBuilder().setImage(
> > > > 
> > > > iuwtImageInfo).build()
> > > > 
> > > > > > > > >         val iuwtContainer = ContainerInfo.newBuilder()
> > > > > > > > >         
> > > > > > > > >                 .setMesos(iuwtContInfo)
> > > > > > > > >           
> > > > > > > > >           .setType(ContainerInfo.Type.MESOS)
> > > > > > > > >           .build()
> > > > > > > > >         
> > > > > > > > >         val iuwtExecutor = ExecutorInfo.newBuilder()
> > > > > > > > >         
> > > > > > > > >                 .setCommand(iuwtCommand)
> > > > > > > > >                 .setContainer(iuwtContainer)
> > > > > > > > >                 .setExecutorId(ExecutorID.
> > > > 
> > > > newBuilder().setValue("iuw
> > > > 
> > > > > > > > >                 t-
> > > > > > > > > 
> > > > > > > > > executor"))
> > > > > > > > > 
> > > > > > > > >                 .setName("iuwt-executor").build()
> > > > > > > > > 
> > > > > > > > > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted
> > 
> > docker
> > 
> > > > > > > > > image.
> > > > > > > > > IUWT is
> > > > > > > > > the program I'm trying to run, and it runs perfectly fine in
> > 
> > the
> > 
> > > > > > > > > aforementioned docker container when running on docker. The
> > > > > > > 
> > > > > > > libtiff.so.5
> > > > > > > 
> > > > > > > > > problem only manifests when I'm using mesos' containerizer.
> > > > > > > > > 
> > > > > > > > > On Monday, October 17, 2016 6:52:17 AM CEST Avinash
> > > > > > > > > Sridharan
> > > > 
> > > > wrote:
> > > > > > > > > > You are running a container with its own image right? So
> > > > > > > > > > is
> > > > > > > > > 
> > > > > > > > > /usr/lib/x86_64
> > > > > > > > > 
> > > > > > > > > > in the container's file system or the host file system?
> > > > > > > > > > 
> > > > > > > > > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> > > > > > > > > 
> > > > > > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > > > > > 
> > > > > > > > > > > wrote:
> > > > > > > > > > > 
> > > > > > > > > > > Yes, it's installed under /usr/lib/x86_64 or whatever
> > > > > > > > > > > the
> > > > > > > > > > > multilib
> > > > > > > > > 
> > > > > > > > > path is
> > > > > > > > > 
> > > > > > > > > > > in
> > > > > > > > > > > debian. It seems files under this path are not
> > 
> > accessible.
> > 
> > > > > > > > > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to
> > 
> > force the
> > 
> > > > > > > symlink
> > > > > > > 
> > > > > > > > > to
> > > > > > > > > 
> > > > > > > > > > > load
> > > > > > > > > > > but it refused to load it. I think the mesos
> > 
> > containerizer
> > 
> > > > > > > > > > > is
> > > > > > > > > 
> > > > > > > > > preventing
> > > > > > > > > 
> > > > > > > > > > > the
> > > > > > > > > > > program from accessing anything in a directory under
> > > > 
> > > > /usr/lib/
> > > > 
> > > > > > > > > > > for
> > > > > > > > > > > some
> > > > > > > > > > > reason, as the same program runs fine in the same
> > 
> > container
> > 
> > > > > > > > > > > running
> > > > > > > > > 
> > > > > > > > > under
> > > > > > > > > 
> > > > > > > > > > > docker.
> > > > > > > > > > > 
> > > > > > > > > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash
> > > > > > > > > > > Sridharan
> > > > > > > 
> > > > > > > wrote:
> > > > > > > > > > > > Is the library part of the image that you are running?
> > > > > > > > > > > > Also
> > > > > > > > > > > > you
> > > > > > > > > > > > might
> > > > > > > > > > > 
> > > > > > > > > > > need
> > > > > > > > > > > 
> > > > > > > > > > > > to setup LD_LIBRARY_PATH in your env while launching
> > 
> > the
> > 
> > > > image
> > > > 
> > > > > > > > > > > > so
> > > > > > > > > 
> > > > > > > > > that
> > > > > > > > > 
> > > > > > > > > > > the
> > > > > > > > > > > 
> > > > > > > > > > > > container process knows where to look for the shared
> > > > 
> > > > object.
> > > > 
> > > > > > > > > > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > > > > > > > > > > 
> > > > > > > > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > > > > > > > 
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > 
> > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > 
> > > > > > > > > > > > > I've been working with the mesos containerizer to
> > 
> > handle
> > 
> > > > my
> > > > 
> > > > > > > docker
> > > > > > > 
> > > > > > > > > > > > > containers
> > > > > > > > > > > > > recently. I created a docker container that requires
> > > > > > > 
> > > > > > > libtiff.so.5,
> > > > > > > 
> > > > > > > > > and
> > > > > > > > > 
> > > > > > > > > > > the
> > > > > > > > > > > 
> > > > > > > > > > > > > application within it runs well. But when I try to
> > 
> > run
> > 
> > > > > > > > > > > > > it
> > > > > > > 
> > > > > > > within
> > > > > > > 
> > > > > > > > > the
> > > > > > > > > 
> > > > > > > > > > > mesos
> > > > > > > > > > > 
> > > > > > > > > > > > > containerizer I get an error saying libtiff.so.5
> > 
> > doesn't
> > 
> > > > > > > > > > > > > exist.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > The application is being launched via java's process
> > > > > > > > > > > > > mechanism
> > > > > > > > > > > > > from
> > > > > > > > > > > 
> > > > > > > > > > > inside
> > > > > > > > > > > 
> > > > > > > > > > > > > a
> > > > > > > > > > > > > java thread in a custom java executor if that makes
> > > > > > > > > > > > > a
> > > > > > > 
> > > > > > > difference.
> > > > > > > 
> > > > > > > > > > > > > Any idea what could be causing this change in
> > 
> > behavior?
> > 
> > > > As
> > > > 
> > > > > > > > > > > > > you
> > > > > > > 
> > > > > > > can
> > > > > > > 
> > > > > > > > > see
> > > > > > > > > 
> > > > > > > > > > > in
> > > > > > > > > > > 
> > > > > > > > > > > > > the
> > > > > > > > > > > > > attached log file, I check /usr/lib, and a symbolic
> > 
> > link
> > 
> > > > to
> > > > 
> > > > > > > > > > > > > /usr/lib/x86..../
> > > > > > > > > > > > > libtiff.so.5 exists in /usr/lib so the program
> > 
> > should be
> > 
> > > > > > > > > > > > > able
> > > > > > > 
> > > > > > > to
> > > > > > > 
> > > > > > > > > find
> > > > > > > > > 
> > > > > > > > > > > and
> > > > > > > > > > > 
> > > > > > > > > > > > > load
> > > > > > > > > > > > > that....
> > > > > > > > > > > > > ----
> > > > > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > > > > > > > 
> > > > > > > > > > > --
> > > > > > > > > > > ----
> > > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > > > > > 
> > > > > > > > > --
> > > > > > > > > ----
> > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > > > 
> > > > > > > --
> > > > > > > ----
> > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > 
> > > > --
> > > > ----
> > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > 
> > --
> > ----
> > Mark Edgar Hammons II | +33 06 03 69 56 56


-- 
----
Mark Edgar Hammons II | +33 06 03 69 56 56

Re: Problems with shared libraries with mesos containerization....

Posted by Avinash Sridharan <av...@mesosphere.io>.

Maybe just attach the docker file and make sure layers are available in
docker hub?
On Mon, Oct 17, 2016 at 9:07 AM Mark Hammons <ma...@inaf.cnrs-gif.fr>
wrote:

> Oh, the container image itself is very large. Do you have some place I can
> upload it to?
>
> On Monday, October 17, 2016 8:43:10 AM CEST Avinash Sridharan wrote:
> > Also would be good to see the env variables (LD_LIBRARY_PATH) being seen
> by
> > the container when you start it, just to make sure the env are getting
> set
> > correctly.
> >
> > On Mon, Oct 17, 2016 at 8:42 AM, Avinash Sridharan <
> avinash@mesosphere.io>
> >
> > wrote:
> > > Does look like the symlink exists:
> > > /usr/lib/libtiff.so.5
> > >
> > > I am assuming you have checked the realpath exists as well (that's why
> was
> > > asking for `ls -al`) ?
> > >
> > > Don't see that you have volume mounts that might obfuscate the path.
> Could
> > > you create a JIRA and if possible point give access to your docker
> image
> > > for us to try? Do mention the exact version of Mesos and the Distro you
> > > are
> > > trying to run this on.
> > >
> > > -Avinash
> > >
> > > On Mon, Oct 17, 2016 at 8:35 AM, Mark Hammons <
> > >
> > > mark.hammons@inaf.cnrs-gif.fr> wrote:
> > >> No, it's a regular linux log. I've reattached it.
> > >>
> > >> On Monday, October 17, 2016 8:32:07 AM CEST Avinash Sridharan wrote:
> > >> > can't seem to open the attached logs, is it gzip?
> > >> >
> > >> > On Mon, Oct 17, 2016 at 8:14 AM, Mark Hammons <
> > >>
> > >> mark.hammons@inaf.cnrs-gif.fr
> > >>
> > >> > > wrote:
> > >> > >
> > >> > > Adding LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu doesn't fix the
> > >>
> > >> error.
> > >>
> > >> > > On Monday, October 17, 2016 5:06:34 PM CEST Mark Hammons wrote:
> > >> > > > As I've shown in the original logs, the symbolic link
> libtiff.so.5
> > >>
> > >> is in
> > >>
> > >> > > > /usr/ lib. I used LD_PRELOAD just to try to force
> > >>
> > >> /usr/lib/libtiff.so.5
> > >>
> > >> > > to
> > >> > >
> > >> > > > be loaded since it wasn't for some reason.
> > >> > > >
> > >> > > > On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan
> wrote:
> > >> > > > > Can you prepend your command with `ls -al /usr/lib` and check
> in
> > >> > >
> > >> > > stdout if
> > >> > >
> > >> > > > > you are seeing the shared object? By the way why are you using
> > >> > >
> > >> > > LD_PRELOAD
> > >> > >
> > >> > > > > instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker
> to
> > >>
> > >> load
> > >>
> > >> > > your
> > >> > >
> > >> > > > > library before any other libraries. It's usually used to
> override
> > >> > >
> > >> > > system
> > >> > >
> > >> > > > > libraries so found a bit odd that you are using it here?
> > >> > > > >
> > >> > > > > On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons
> > >> > > > > <ma...@inaf.cnrs-gif.fr>
> > >> > > > >
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for
> > >>
> > >> the
> > >>
> > >> > > process
> > >> > >
> > >> > > > > > environment it says it can't load /usr/lib/libtiff.so.5.
> > >> > > > > >
> > >> > > > > > On Monday, October 17, 2016 7:52:16 AM CEST Avinash
> Sridharan
> > >>
> > >> wrote:
> > >> > > > > > > Are you setting the env in the dockerfile?
> > >> > > > > > >
> > >> > > > > > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
> > >> > > > > >
> > >> > > > > > mark.hammons@inaf.cnrs-gif.fr
> > >> > > > > >
> > >> > > > > > > > wrote:
> > >> > > > > > > >
> > >> > > > > > > > Here's the code I define my executor to mesos with:
> > >> > > > > > > >         val iuwtURI = CommandInfo.URI.newBuilder().
> > >> > >
> > >> > > setValue("http://
> > >> > >
> > >> > > > > > ***/
> > >> > > > > >
> > >> > > > > > > > IUWT.tar.gz").setExtract(true).setCache(false).build()
> > >> > > > > > > >
> > >> > > > > > > >         val iuwtjURI =
> > >> > > > > > > >         CommandInfo.URI.newBuilder().setValue("http://
> > >> > > > > >
> > >> > > > > > ***/
> > >> > > > > >
> > >> > > > > > > > iuwtExecutor-assembly-0.1-
> > >> > > > > > > >
> SNAPSHOT.jar").setExecutable(false).setCache(false).build()
> > >> > > > > > > >
> > >> > > > > > > >         val iuwtExec = "java -jar
> > >> > > > > > > >         iuwtExecutor-assembly-0.1-SNAPSHOT.jar
> > >> > > > > >
> > >> > > > > > -
> > >> > > > > >
> > >> > > > > > > > Xmx1024M -Xmx128M"
> > >> > > > > > > >
> > >> > > > > > > >         val iuwtCommand =
> > >> > > > > > > >
> > >> > > > > > > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(
> > >> > >
> > >> > > List(iuwtjURI,
> > >> > >
> > >> > > > > > > > iuwtURI).asJava).setShell(true).build()
> > >> > > > > > > >
> > >> > > > > > > >         val iuwtImageInfo =
> > >> > > > > > > >
> > >> > > > > > > > Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> > >> > > > > > > > Image.Docker.newBuilder.setName("ubuntu-
> > >> > > > > > > > mesos:0.11-17102016-IUWT").build()).build()
> > >> > > > > > > >
> > >> > > > > > > >         val iuwtContInfo =
> > >> > > > > > > >
> > >> > > > > > > > ContainerInfo.MesosInfo.newBuilder().setImage(
> > >> > >
> > >> > > iuwtImageInfo).build()
> > >> > >
> > >> > > > > > > >         val iuwtContainer = ContainerInfo.newBuilder()
> > >> > > > > > > >
> > >> > > > > > > >                 .setMesos(iuwtContInfo)
> > >> > > > > > > >
> > >> > > > > > > >           .setType(ContainerInfo.Type.MESOS)
> > >> > > > > > > >           .build()
> > >> > > > > > > >
> > >> > > > > > > >         val iuwtExecutor = ExecutorInfo.newBuilder()
> > >> > > > > > > >
> > >> > > > > > > >                 .setCommand(iuwtCommand)
> > >> > > > > > > >                 .setContainer(iuwtContainer)
> > >> > > > > > > >                 .setExecutorId(ExecutorID.
> > >> > >
> > >> > > newBuilder().setValue("iuw
> > >> > >
> > >> > > > > > > >                 t-
> > >> > > > > > > >
> > >> > > > > > > > executor"))
> > >> > > > > > > >
> > >> > > > > > > >                 .setName("iuwt-executor").build()
> > >> > > > > > > >
> > >> > > > > > > > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted
> > >>
> > >> docker
> > >>
> > >> > > > > > > > image.
> > >> > > > > > > > IUWT is
> > >> > > > > > > > the program I'm trying to run, and it runs perfectly
> fine
> > >>
> > >> in the
> > >>
> > >> > > > > > > > aforementioned docker container when running on docker.
> The
> > >> > > > > >
> > >> > > > > > libtiff.so.5
> > >> > > > > >
> > >> > > > > > > > problem only manifests when I'm using mesos'
> containerizer.
> > >> > > > > > > >
> > >> > > > > > > > On Monday, October 17, 2016 6:52:17 AM CEST Avinash
> > >>
> > >> Sridharan
> > >>
> > >> > > wrote:
> > >> > > > > > > > > You are running a container with its own image right?
> So
> > >>
> > >> is
> > >>
> > >> > > > > > > > /usr/lib/x86_64
> > >> > > > > > > >
> > >> > > > > > > > > in the container's file system or the host file
> system?
> > >> > > > > > > > >
> > >> > > > > > > > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> > >> > > > > > > >
> > >> > > > > > > > mark.hammons@inaf.cnrs-gif.fr
> > >> > > > > > > >
> > >> > > > > > > > > > wrote:
> > >> > > > > > > > > >
> > >> > > > > > > > > > Yes, it's installed under /usr/lib/x86_64 or
> whatever
> > >>
> > >> the
> > >>
> > >> > > > > > > > > > multilib
> > >> > > > > > > >
> > >> > > > > > > > path is
> > >> > > > > > > >
> > >> > > > > > > > > > in
> > >> > > > > > > > > > debian. It seems files under this path are not
> > >>
> > >> accessible.
> > >>
> > >> > > > > > > > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to
> > >>
> > >> force the
> > >>
> > >> > > > > > symlink
> > >> > > > > >
> > >> > > > > > > > to
> > >> > > > > > > >
> > >> > > > > > > > > > load
> > >> > > > > > > > > > but it refused to load it. I think the mesos
> > >>
> > >> containerizer
> > >>
> > >> > > > > > > > > > is
> > >> > > > > > > >
> > >> > > > > > > > preventing
> > >> > > > > > > >
> > >> > > > > > > > > > the
> > >> > > > > > > > > > program from accessing anything in a directory under
> > >> > >
> > >> > > /usr/lib/
> > >> > >
> > >> > > > > > > > > > for
> > >> > > > > > > > > > some
> > >> > > > > > > > > > reason, as the same program runs fine in the same
> > >>
> > >> container
> > >>
> > >> > > > > > > > > > running
> > >> > > > > > > >
> > >> > > > > > > > under
> > >> > > > > > > >
> > >> > > > > > > > > > docker.
> > >> > > > > > > > > >
> > >> > > > > > > > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash
> > >> > > > > > > > > > Sridharan
> > >> > > > > >
> > >> > > > > > wrote:
> > >> > > > > > > > > > > Is the library part of the image that you are
> > >> > > > > > > > > > > running?
> > >> > > > > > > > > > > Also
> > >> > > > > > > > > > > you
> > >> > > > > > > > > > > might
> > >> > > > > > > > > >
> > >> > > > > > > > > > need
> > >> > > > > > > > > >
> > >> > > > > > > > > > > to setup LD_LIBRARY_PATH in your env while
> launching
> > >>
> > >> the
> > >>
> > >> > > image
> > >> > >
> > >> > > > > > > > > > > so
> > >> > > > > > > >
> > >> > > > > > > > that
> > >> > > > > > > >
> > >> > > > > > > > > > the
> > >> > > > > > > > > >
> > >> > > > > > > > > > > container process knows where to look for the
> shared
> > >> > >
> > >> > > object.
> > >> > >
> > >> > > > > > > > > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > >> > > > > > > > > >
> > >> > > > > > > > > > mark.hammons@inaf.cnrs-gif.fr
> > >> > > > > > > > > >
> > >> > > > > > > > > > > > wrote:
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > Hi all,
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > I've been working with the mesos containerizer
> to
> > >>
> > >> handle
> > >>
> > >> > > my
> > >> > >
> > >> > > > > > docker
> > >> > > > > >
> > >> > > > > > > > > > > > containers
> > >> > > > > > > > > > > > recently. I created a docker container that
> > >> > > > > > > > > > > > requires
> > >> > > > > >
> > >> > > > > > libtiff.so.5,
> > >> > > > > >
> > >> > > > > > > > and
> > >> > > > > > > >
> > >> > > > > > > > > > the
> > >> > > > > > > > > >
> > >> > > > > > > > > > > > application within it runs well. But when I try
> to
> > >>
> > >> run
> > >>
> > >> > > > > > > > > > > > it
> > >> > > > > >
> > >> > > > > > within
> > >> > > > > >
> > >> > > > > > > > the
> > >> > > > > > > >
> > >> > > > > > > > > > mesos
> > >> > > > > > > > > >
> > >> > > > > > > > > > > > containerizer I get an error saying libtiff.so.5
> > >>
> > >> doesn't
> > >>
> > >> > > > > > > > > > > > exist.
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > The application is being launched via java's
> > >> > > > > > > > > > > > process
> > >> > > > > > > > > > > > mechanism
> > >> > > > > > > > > > > > from
> > >> > > > > > > > > >
> > >> > > > > > > > > > inside
> > >> > > > > > > > > >
> > >> > > > > > > > > > > > a
> > >> > > > > > > > > > > > java thread in a custom java executor if that
> makes
> > >>
> > >> a
> > >>
> > >> > > > > > difference.
> > >> > > > > >
> > >> > > > > > > > > > > > Any idea what could be causing this change in
> > >>
> > >> behavior?
> > >>
> > >> > > As
> > >> > >
> > >> > > > > > > > > > > > you
> > >> > > > > >
> > >> > > > > > can
> > >> > > > > >
> > >> > > > > > > > see
> > >> > > > > > > >
> > >> > > > > > > > > > in
> > >> > > > > > > > > >
> > >> > > > > > > > > > > > the
> > >> > > > > > > > > > > > attached log file, I check /usr/lib, and a
> symbolic
> > >>
> > >> link
> > >>
> > >> > > to
> > >> > >
> > >> > > > > > > > > > > > /usr/lib/x86..../
> > >> > > > > > > > > > > > libtiff.so.5 exists in /usr/lib so the program
> > >>
> > >> should be
> > >>
> > >> > > > > > > > > > > > able
> > >> > > > > >
> > >> > > > > > to
> > >> > > > > >
> > >> > > > > > > > find
> > >> > > > > > > >
> > >> > > > > > > > > > and
> > >> > > > > > > > > >
> > >> > > > > > > > > > > > load
> > >> > > > > > > > > > > > that....
> > >> > > > > > > > > > > > ----
> > >> > > > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > >> > > > > > > > > >
> > >> > > > > > > > > > --
> > >> > > > > > > > > > ----
> > >> > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > >> > > > > > > >
> > >> > > > > > > > --
> > >> > > > > > > > ----
> > >> > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > >> > > > > >
> > >> > > > > > --
> > >> > > > > > ----
> > >> > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > >> > >
> > >> > > --
> > >> > > ----
> > >> > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > >>
> > >> --
> > >> ----
> > >> Mark Edgar Hammons II | +33 06 03 69 56 56
> > >
> > > --
> > > Avinash Sridharan, Mesosphere
> > > +1 (323) 702 5245
>
>
> --
> ----
> Mark Edgar Hammons II | +33 06 03 69 56 56

Re: Problems with shared libraries with mesos containerization....

Posted by Mark Hammons <ma...@inaf.cnrs-gif.fr>.

Oh, the container image itself is very large. Do you have some place I can 
upload it to?

On Monday, October 17, 2016 8:43:10 AM CEST Avinash Sridharan wrote:
> Also would be good to see the env variables (LD_LIBRARY_PATH) being seen by
> the container when you start it, just to make sure the env are getting set
> correctly.
> 
> On Mon, Oct 17, 2016 at 8:42 AM, Avinash Sridharan <av...@mesosphere.io>
> 
> wrote:
> > Does look like the symlink exists:
> > /usr/lib/libtiff.so.5
> > 
> > I am assuming you have checked the realpath exists as well (that's why was
> > asking for `ls -al`) ?
> > 
> > Don't see that you have volume mounts that might obfuscate the path. Could
> > you create a JIRA and if possible point give access to your docker image
> > for us to try? Do mention the exact version of Mesos and the Distro you
> > are
> > trying to run this on.
> > 
> > -Avinash
> > 
> > On Mon, Oct 17, 2016 at 8:35 AM, Mark Hammons <
> > 
> > mark.hammons@inaf.cnrs-gif.fr> wrote:
> >> No, it's a regular linux log. I've reattached it.
> >> 
> >> On Monday, October 17, 2016 8:32:07 AM CEST Avinash Sridharan wrote:
> >> > can't seem to open the attached logs, is it gzip?
> >> > 
> >> > On Mon, Oct 17, 2016 at 8:14 AM, Mark Hammons <
> >> 
> >> mark.hammons@inaf.cnrs-gif.fr
> >> 
> >> > > wrote:
> >> > > 
> >> > > Adding LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu doesn't fix the
> >> 
> >> error.
> >> 
> >> > > On Monday, October 17, 2016 5:06:34 PM CEST Mark Hammons wrote:
> >> > > > As I've shown in the original logs, the symbolic link libtiff.so.5
> >> 
> >> is in
> >> 
> >> > > > /usr/ lib. I used LD_PRELOAD just to try to force
> >> 
> >> /usr/lib/libtiff.so.5
> >> 
> >> > > to
> >> > > 
> >> > > > be loaded since it wasn't for some reason.
> >> > > > 
> >> > > > On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan 
wrote:
> >> > > > > Can you prepend your command with `ls -al /usr/lib` and check in
> >> > > 
> >> > > stdout if
> >> > > 
> >> > > > > you are seeing the shared object? By the way why are you using
> >> > > 
> >> > > LD_PRELOAD
> >> > > 
> >> > > > > instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to
> >> 
> >> load
> >> 
> >> > > your
> >> > > 
> >> > > > > library before any other libraries. It's usually used to override
> >> > > 
> >> > > system
> >> > > 
> >> > > > > libraries so found a bit odd that you are using it here?
> >> > > > > 
> >> > > > > On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons
> >> > > > > <ma...@inaf.cnrs-gif.fr>
> >> > > > > 
> >> > > > > > wrote:
> >> > > > > > 
> >> > > > > > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for
> >> 
> >> the
> >> 
> >> > > process
> >> > > 
> >> > > > > > environment it says it can't load /usr/lib/libtiff.so.5.
> >> > > > > > 
> >> > > > > > On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan
> >> 
> >> wrote:
> >> > > > > > > Are you setting the env in the dockerfile?
> >> > > > > > > 
> >> > > > > > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
> >> > > > > > 
> >> > > > > > mark.hammons@inaf.cnrs-gif.fr
> >> > > > > > 
> >> > > > > > > > wrote:
> >> > > > > > > > 
> >> > > > > > > > Here's the code I define my executor to mesos with:
> >> > > > > > > >         val iuwtURI = CommandInfo.URI.newBuilder().
> >> > > 
> >> > > setValue("http://
> >> > > 
> >> > > > > > ***/
> >> > > > > > 
> >> > > > > > > > IUWT.tar.gz").setExtract(true).setCache(false).build()
> >> > > > > > > > 
> >> > > > > > > >         val iuwtjURI =
> >> > > > > > > >         CommandInfo.URI.newBuilder().setValue("http://
> >> > > > > > 
> >> > > > > > ***/
> >> > > > > > 
> >> > > > > > > > iuwtExecutor-assembly-0.1-
> >> > > > > > > > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
> >> > > > > > > > 
> >> > > > > > > >         val iuwtExec = "java -jar
> >> > > > > > > >         iuwtExecutor-assembly-0.1-SNAPSHOT.jar
> >> > > > > > 
> >> > > > > > -
> >> > > > > > 
> >> > > > > > > > Xmx1024M -Xmx128M"
> >> > > > > > > > 
> >> > > > > > > >         val iuwtCommand =
> >> > > > > > > > 
> >> > > > > > > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(
> >> > > 
> >> > > List(iuwtjURI,
> >> > > 
> >> > > > > > > > iuwtURI).asJava).setShell(true).build()
> >> > > > > > > > 
> >> > > > > > > >         val iuwtImageInfo =
> >> > > > > > > > 
> >> > > > > > > > Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> >> > > > > > > > Image.Docker.newBuilder.setName("ubuntu-
> >> > > > > > > > mesos:0.11-17102016-IUWT").build()).build()
> >> > > > > > > > 
> >> > > > > > > >         val iuwtContInfo =
> >> > > > > > > > 
> >> > > > > > > > ContainerInfo.MesosInfo.newBuilder().setImage(
> >> > > 
> >> > > iuwtImageInfo).build()
> >> > > 
> >> > > > > > > >         val iuwtContainer = ContainerInfo.newBuilder()
> >> > > > > > > >         
> >> > > > > > > >                 .setMesos(iuwtContInfo)
> >> > > > > > > >           
> >> > > > > > > >           .setType(ContainerInfo.Type.MESOS)
> >> > > > > > > >           .build()
> >> > > > > > > >         
> >> > > > > > > >         val iuwtExecutor = ExecutorInfo.newBuilder()
> >> > > > > > > >         
> >> > > > > > > >                 .setCommand(iuwtCommand)
> >> > > > > > > >                 .setContainer(iuwtContainer)
> >> > > > > > > >                 .setExecutorId(ExecutorID.
> >> > > 
> >> > > newBuilder().setValue("iuw
> >> > > 
> >> > > > > > > >                 t-
> >> > > > > > > > 
> >> > > > > > > > executor"))
> >> > > > > > > > 
> >> > > > > > > >                 .setName("iuwt-executor").build()
> >> > > > > > > > 
> >> > > > > > > > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted
> >> 
> >> docker
> >> 
> >> > > > > > > > image.
> >> > > > > > > > IUWT is
> >> > > > > > > > the program I'm trying to run, and it runs perfectly fine
> >> 
> >> in the
> >> 
> >> > > > > > > > aforementioned docker container when running on docker. 
The
> >> > > > > > 
> >> > > > > > libtiff.so.5
> >> > > > > > 
> >> > > > > > > > problem only manifests when I'm using mesos' containerizer.
> >> > > > > > > > 
> >> > > > > > > > On Monday, October 17, 2016 6:52:17 AM CEST Avinash
> >> 
> >> Sridharan
> >> 
> >> > > wrote:
> >> > > > > > > > > You are running a container with its own image right? So
> >> 
> >> is
> >> 
> >> > > > > > > > /usr/lib/x86_64
> >> > > > > > > > 
> >> > > > > > > > > in the container's file system or the host file system?
> >> > > > > > > > > 
> >> > > > > > > > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> >> > > > > > > > 
> >> > > > > > > > mark.hammons@inaf.cnrs-gif.fr
> >> > > > > > > > 
> >> > > > > > > > > > wrote:
> >> > > > > > > > > > 
> >> > > > > > > > > > Yes, it's installed under /usr/lib/x86_64 or whatever
> >> 
> >> the
> >> 
> >> > > > > > > > > > multilib
> >> > > > > > > > 
> >> > > > > > > > path is
> >> > > > > > > > 
> >> > > > > > > > > > in
> >> > > > > > > > > > debian. It seems files under this path are not
> >> 
> >> accessible.
> >> 
> >> > > > > > > > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to
> >> 
> >> force the
> >> 
> >> > > > > > symlink
> >> > > > > > 
> >> > > > > > > > to
> >> > > > > > > > 
> >> > > > > > > > > > load
> >> > > > > > > > > > but it refused to load it. I think the mesos
> >> 
> >> containerizer
> >> 
> >> > > > > > > > > > is
> >> > > > > > > > 
> >> > > > > > > > preventing
> >> > > > > > > > 
> >> > > > > > > > > > the
> >> > > > > > > > > > program from accessing anything in a directory under
> >> > > 
> >> > > /usr/lib/
> >> > > 
> >> > > > > > > > > > for
> >> > > > > > > > > > some
> >> > > > > > > > > > reason, as the same program runs fine in the same
> >> 
> >> container
> >> 
> >> > > > > > > > > > running
> >> > > > > > > > 
> >> > > > > > > > under
> >> > > > > > > > 
> >> > > > > > > > > > docker.
> >> > > > > > > > > > 
> >> > > > > > > > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash
> >> > > > > > > > > > Sridharan
> >> > > > > > 
> >> > > > > > wrote:
> >> > > > > > > > > > > Is the library part of the image that you are
> >> > > > > > > > > > > running?
> >> > > > > > > > > > > Also
> >> > > > > > > > > > > you
> >> > > > > > > > > > > might
> >> > > > > > > > > > 
> >> > > > > > > > > > need
> >> > > > > > > > > > 
> >> > > > > > > > > > > to setup LD_LIBRARY_PATH in your env while launching
> >> 
> >> the
> >> 
> >> > > image
> >> > > 
> >> > > > > > > > > > > so
> >> > > > > > > > 
> >> > > > > > > > that
> >> > > > > > > > 
> >> > > > > > > > > > the
> >> > > > > > > > > > 
> >> > > > > > > > > > > container process knows where to look for the shared
> >> > > 
> >> > > object.
> >> > > 
> >> > > > > > > > > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> >> > > > > > > > > > 
> >> > > > > > > > > > mark.hammons@inaf.cnrs-gif.fr
> >> > > > > > > > > > 
> >> > > > > > > > > > > > wrote:
> >> > > > > > > > > > > > 
> >> > > > > > > > > > > > Hi all,
> >> > > > > > > > > > > > 
> >> > > > > > > > > > > > I've been working with the mesos containerizer to
> >> 
> >> handle
> >> 
> >> > > my
> >> > > 
> >> > > > > > docker
> >> > > > > > 
> >> > > > > > > > > > > > containers
> >> > > > > > > > > > > > recently. I created a docker container that
> >> > > > > > > > > > > > requires
> >> > > > > > 
> >> > > > > > libtiff.so.5,
> >> > > > > > 
> >> > > > > > > > and
> >> > > > > > > > 
> >> > > > > > > > > > the
> >> > > > > > > > > > 
> >> > > > > > > > > > > > application within it runs well. But when I try to
> >> 
> >> run
> >> 
> >> > > > > > > > > > > > it
> >> > > > > > 
> >> > > > > > within
> >> > > > > > 
> >> > > > > > > > the
> >> > > > > > > > 
> >> > > > > > > > > > mesos
> >> > > > > > > > > > 
> >> > > > > > > > > > > > containerizer I get an error saying libtiff.so.5
> >> 
> >> doesn't
> >> 
> >> > > > > > > > > > > > exist.
> >> > > > > > > > > > > > 
> >> > > > > > > > > > > > The application is being launched via java's
> >> > > > > > > > > > > > process
> >> > > > > > > > > > > > mechanism
> >> > > > > > > > > > > > from
> >> > > > > > > > > > 
> >> > > > > > > > > > inside
> >> > > > > > > > > > 
> >> > > > > > > > > > > > a
> >> > > > > > > > > > > > java thread in a custom java executor if that makes
> >> 
> >> a
> >> 
> >> > > > > > difference.
> >> > > > > > 
> >> > > > > > > > > > > > Any idea what could be causing this change in
> >> 
> >> behavior?
> >> 
> >> > > As
> >> > > 
> >> > > > > > > > > > > > you
> >> > > > > > 
> >> > > > > > can
> >> > > > > > 
> >> > > > > > > > see
> >> > > > > > > > 
> >> > > > > > > > > > in
> >> > > > > > > > > > 
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > attached log file, I check /usr/lib, and a symbolic
> >> 
> >> link
> >> 
> >> > > to
> >> > > 
> >> > > > > > > > > > > > /usr/lib/x86..../
> >> > > > > > > > > > > > libtiff.so.5 exists in /usr/lib so the program
> >> 
> >> should be
> >> 
> >> > > > > > > > > > > > able
> >> > > > > > 
> >> > > > > > to
> >> > > > > > 
> >> > > > > > > > find
> >> > > > > > > > 
> >> > > > > > > > > > and
> >> > > > > > > > > > 
> >> > > > > > > > > > > > load
> >> > > > > > > > > > > > that....
> >> > > > > > > > > > > > ----
> >> > > > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> >> > > > > > > > > > 
> >> > > > > > > > > > --
> >> > > > > > > > > > ----
> >> > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> >> > > > > > > > 
> >> > > > > > > > --
> >> > > > > > > > ----
> >> > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> >> > > > > > 
> >> > > > > > --
> >> > > > > > ----
> >> > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> >> > > 
> >> > > --
> >> > > ----
> >> > > Mark Edgar Hammons II | +33 06 03 69 56 56
> >> 
> >> --
> >> ----
> >> Mark Edgar Hammons II | +33 06 03 69 56 56
> > 
> > --
> > Avinash Sridharan, Mesosphere
> > +1 (323) 702 5245


-- 
----
Mark Edgar Hammons II | +33 06 03 69 56 56

Re: Problems with shared libraries with mesos containerization....

Posted by Avinash Sridharan <av...@mesosphere.io>.

Also would be good to see the env variables (LD_LIBRARY_PATH) being seen by
the container when you start it, just to make sure the env are getting set
correctly.

On Mon, Oct 17, 2016 at 8:42 AM, Avinash Sridharan <av...@mesosphere.io>
wrote:

> Does look like the symlink exists:
> /usr/lib/libtiff.so.5
>
> I am assuming you have checked the realpath exists as well (that's why was
> asking for `ls -al`) ?
>
> Don't see that you have volume mounts that might obfuscate the path. Could
> you create a JIRA and if possible point give access to your docker image
> for us to try? Do mention the exact version of Mesos and the Distro you are
> trying to run this on.
>
> -Avinash
>
> On Mon, Oct 17, 2016 at 8:35 AM, Mark Hammons <
> mark.hammons@inaf.cnrs-gif.fr> wrote:
>
>> No, it's a regular linux log. I've reattached it.
>>
>> On Monday, October 17, 2016 8:32:07 AM CEST Avinash Sridharan wrote:
>> > can't seem to open the attached logs, is it gzip?
>> >
>> > On Mon, Oct 17, 2016 at 8:14 AM, Mark Hammons <
>> mark.hammons@inaf.cnrs-gif.fr
>> > > wrote:
>> > >
>> > > Adding LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu doesn't fix the
>> error.
>> > >
>> > > On Monday, October 17, 2016 5:06:34 PM CEST Mark Hammons wrote:
>> > > > As I've shown in the original logs, the symbolic link libtiff.so.5
>> is in
>> > > > /usr/ lib. I used LD_PRELOAD just to try to force
>> /usr/lib/libtiff.so.5
>> > >
>> > > to
>> > >
>> > > > be loaded since it wasn't for some reason.
>> > > >
>> > > > On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan wrote:
>> > > > > Can you prepend your command with `ls -al /usr/lib` and check in
>> > >
>> > > stdout if
>> > >
>> > > > > you are seeing the shared object? By the way why are you using
>> > >
>> > > LD_PRELOAD
>> > >
>> > > > > instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to
>> load
>> > >
>> > > your
>> > >
>> > > > > library before any other libraries. It's usually used to override
>> > >
>> > > system
>> > >
>> > > > > libraries so found a bit odd that you are using it here?
>> > > > >
>> > > > > On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons
>> > > > > <ma...@inaf.cnrs-gif.fr>
>> > > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for
>> the
>> > >
>> > > process
>> > >
>> > > > > > environment it says it can't load /usr/lib/libtiff.so.5.
>> > > > > >
>> > > > > > On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan
>> wrote:
>> > > > > > > Are you setting the env in the dockerfile?
>> > > > > > >
>> > > > > > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
>> > > > > >
>> > > > > > mark.hammons@inaf.cnrs-gif.fr
>> > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > Here's the code I define my executor to mesos with:
>> > > > > > > >         val iuwtURI = CommandInfo.URI.newBuilder().
>> > >
>> > > setValue("http://
>> > >
>> > > > > > ***/
>> > > > > >
>> > > > > > > > IUWT.tar.gz").setExtract(true).setCache(false).build()
>> > > > > > > >
>> > > > > > > >         val iuwtjURI =
>> > > > > > > >         CommandInfo.URI.newBuilder().setValue("http://
>> > > > > >
>> > > > > > ***/
>> > > > > >
>> > > > > > > > iuwtExecutor-assembly-0.1-
>> > > > > > > > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
>> > > > > > > >
>> > > > > > > >         val iuwtExec = "java -jar
>> > > > > > > >         iuwtExecutor-assembly-0.1-SNAPSHOT.jar
>> > > > > >
>> > > > > > -
>> > > > > >
>> > > > > > > > Xmx1024M -Xmx128M"
>> > > > > > > >
>> > > > > > > >         val iuwtCommand =
>> > > > > > > >
>> > > > > > > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(
>> > >
>> > > List(iuwtjURI,
>> > >
>> > > > > > > > iuwtURI).asJava).setShell(true).build()
>> > > > > > > >
>> > > > > > > >         val iuwtImageInfo =
>> > > > > > > >
>> > > > > > > > Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
>> > > > > > > > Image.Docker.newBuilder.setName("ubuntu-
>> > > > > > > > mesos:0.11-17102016-IUWT").build()).build()
>> > > > > > > >
>> > > > > > > >         val iuwtContInfo =
>> > > > > > > >
>> > > > > > > > ContainerInfo.MesosInfo.newBuilder().setImage(
>> > >
>> > > iuwtImageInfo).build()
>> > >
>> > > > > > > >         val iuwtContainer = ContainerInfo.newBuilder()
>> > > > > > > >
>> > > > > > > >                 .setMesos(iuwtContInfo)
>> > > > > > > >
>> > > > > > > >           .setType(ContainerInfo.Type.MESOS)
>> > > > > > > >           .build()
>> > > > > > > >
>> > > > > > > >         val iuwtExecutor = ExecutorInfo.newBuilder()
>> > > > > > > >
>> > > > > > > >                 .setCommand(iuwtCommand)
>> > > > > > > >                 .setContainer(iuwtContainer)
>> > > > > > > >                 .setExecutorId(ExecutorID.
>> > >
>> > > newBuilder().setValue("iuw
>> > >
>> > > > > > > >                 t-
>> > > > > > > >
>> > > > > > > > executor"))
>> > > > > > > >
>> > > > > > > >                 .setName("iuwt-executor").build()
>> > > > > > > >
>> > > > > > > > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted
>> docker
>> > > > > > > > image.
>> > > > > > > > IUWT is
>> > > > > > > > the program I'm trying to run, and it runs perfectly fine
>> in the
>> > > > > > > > aforementioned docker container when running on docker. The
>> > > > > >
>> > > > > > libtiff.so.5
>> > > > > >
>> > > > > > > > problem only manifests when I'm using mesos' containerizer.
>> > > > > > > >
>> > > > > > > > On Monday, October 17, 2016 6:52:17 AM CEST Avinash
>> Sridharan
>> > >
>> > > wrote:
>> > > > > > > > > You are running a container with its own image right? So
>> is
>> > > > > > > >
>> > > > > > > > /usr/lib/x86_64
>> > > > > > > >
>> > > > > > > > > in the container's file system or the host file system?
>> > > > > > > > >
>> > > > > > > > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
>> > > > > > > >
>> > > > > > > > mark.hammons@inaf.cnrs-gif.fr
>> > > > > > > >
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > Yes, it's installed under /usr/lib/x86_64 or whatever
>> the
>> > > > > > > > > > multilib
>> > > > > > > >
>> > > > > > > > path is
>> > > > > > > >
>> > > > > > > > > > in
>> > > > > > > > > > debian. It seems files under this path are not
>> accessible.
>> > > > > > > > > >
>> > > > > > > > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to
>> force the
>> > > > > >
>> > > > > > symlink
>> > > > > >
>> > > > > > > > to
>> > > > > > > >
>> > > > > > > > > > load
>> > > > > > > > > > but it refused to load it. I think the mesos
>> containerizer
>> > > > > > > > > > is
>> > > > > > > >
>> > > > > > > > preventing
>> > > > > > > >
>> > > > > > > > > > the
>> > > > > > > > > > program from accessing anything in a directory under
>> > >
>> > > /usr/lib/
>> > >
>> > > > > > > > > > for
>> > > > > > > > > > some
>> > > > > > > > > > reason, as the same program runs fine in the same
>> container
>> > > > > > > > > > running
>> > > > > > > >
>> > > > > > > > under
>> > > > > > > >
>> > > > > > > > > > docker.
>> > > > > > > > > >
>> > > > > > > > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash
>> > > > > > > > > > Sridharan
>> > > > > >
>> > > > > > wrote:
>> > > > > > > > > > > Is the library part of the image that you are running?
>> > > > > > > > > > > Also
>> > > > > > > > > > > you
>> > > > > > > > > > > might
>> > > > > > > > > >
>> > > > > > > > > > need
>> > > > > > > > > >
>> > > > > > > > > > > to setup LD_LIBRARY_PATH in your env while launching
>> the
>> > >
>> > > image
>> > >
>> > > > > > > > > > > so
>> > > > > > > >
>> > > > > > > > that
>> > > > > > > >
>> > > > > > > > > > the
>> > > > > > > > > >
>> > > > > > > > > > > container process knows where to look for the shared
>> > >
>> > > object.
>> > >
>> > > > > > > > > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
>> > > > > > > > > >
>> > > > > > > > > > mark.hammons@inaf.cnrs-gif.fr
>> > > > > > > > > >
>> > > > > > > > > > > > wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > > Hi all,
>> > > > > > > > > > > >
>> > > > > > > > > > > > I've been working with the mesos containerizer to
>> handle
>> > >
>> > > my
>> > >
>> > > > > > docker
>> > > > > >
>> > > > > > > > > > > > containers
>> > > > > > > > > > > > recently. I created a docker container that requires
>> > > > > >
>> > > > > > libtiff.so.5,
>> > > > > >
>> > > > > > > > and
>> > > > > > > >
>> > > > > > > > > > the
>> > > > > > > > > >
>> > > > > > > > > > > > application within it runs well. But when I try to
>> run
>> > > > > > > > > > > > it
>> > > > > >
>> > > > > > within
>> > > > > >
>> > > > > > > > the
>> > > > > > > >
>> > > > > > > > > > mesos
>> > > > > > > > > >
>> > > > > > > > > > > > containerizer I get an error saying libtiff.so.5
>> doesn't
>> > > > > > > > > > > > exist.
>> > > > > > > > > > > >
>> > > > > > > > > > > > The application is being launched via java's process
>> > > > > > > > > > > > mechanism
>> > > > > > > > > > > > from
>> > > > > > > > > >
>> > > > > > > > > > inside
>> > > > > > > > > >
>> > > > > > > > > > > > a
>> > > > > > > > > > > > java thread in a custom java executor if that makes
>> a
>> > > > > >
>> > > > > > difference.
>> > > > > >
>> > > > > > > > > > > > Any idea what could be causing this change in
>> behavior?
>> > >
>> > > As
>> > >
>> > > > > > > > > > > > you
>> > > > > >
>> > > > > > can
>> > > > > >
>> > > > > > > > see
>> > > > > > > >
>> > > > > > > > > > in
>> > > > > > > > > >
>> > > > > > > > > > > > the
>> > > > > > > > > > > > attached log file, I check /usr/lib, and a symbolic
>> link
>> > >
>> > > to
>> > >
>> > > > > > > > > > > > /usr/lib/x86..../
>> > > > > > > > > > > > libtiff.so.5 exists in /usr/lib so the program
>> should be
>> > > > > > > > > > > > able
>> > > > > >
>> > > > > > to
>> > > > > >
>> > > > > > > > find
>> > > > > > > >
>> > > > > > > > > > and
>> > > > > > > > > >
>> > > > > > > > > > > > load
>> > > > > > > > > > > > that....
>> > > > > > > > > > > > ----
>> > > > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
>> > > > > > > > > >
>> > > > > > > > > > --
>> > > > > > > > > > ----
>> > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
>> > > > > > > >
>> > > > > > > > --
>> > > > > > > > ----
>> > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
>> > > > > >
>> > > > > > --
>> > > > > > ----
>> > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
>> > >
>> > > --
>> > > ----
>> > > Mark Edgar Hammons II | +33 06 03 69 56 56
>>
>>
>> --
>> ----
>> Mark Edgar Hammons II | +33 06 03 69 56 56
>>
>
>
>
> --
> Avinash Sridharan, Mesosphere
> +1 (323) 702 5245
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245

Re: Problems with shared libraries with mesos containerization....

Posted by Avinash Sridharan <av...@mesosphere.io>.

Does look like the symlink exists:
/usr/lib/libtiff.so.5

I am assuming you have checked the realpath exists as well (that's why was
asking for `ls -al`) ?

Don't see that you have volume mounts that might obfuscate the path. Could
you create a JIRA and if possible point give access to your docker image
for us to try? Do mention the exact version of Mesos and the Distro you are
trying to run this on.

-Avinash

On Mon, Oct 17, 2016 at 8:35 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> wrote:

> No, it's a regular linux log. I've reattached it.
>
> On Monday, October 17, 2016 8:32:07 AM CEST Avinash Sridharan wrote:
> > can't seem to open the attached logs, is it gzip?
> >
> > On Mon, Oct 17, 2016 at 8:14 AM, Mark Hammons <
> mark.hammons@inaf.cnrs-gif.fr
> > > wrote:
> > >
> > > Adding LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu doesn't fix the
> error.
> > >
> > > On Monday, October 17, 2016 5:06:34 PM CEST Mark Hammons wrote:
> > > > As I've shown in the original logs, the symbolic link libtiff.so.5
> is in
> > > > /usr/ lib. I used LD_PRELOAD just to try to force
> /usr/lib/libtiff.so.5
> > >
> > > to
> > >
> > > > be loaded since it wasn't for some reason.
> > > >
> > > > On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan wrote:
> > > > > Can you prepend your command with `ls -al /usr/lib` and check in
> > >
> > > stdout if
> > >
> > > > > you are seeing the shared object? By the way why are you using
> > >
> > > LD_PRELOAD
> > >
> > > > > instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to
> load
> > >
> > > your
> > >
> > > > > library before any other libraries. It's usually used to override
> > >
> > > system
> > >
> > > > > libraries so found a bit odd that you are using it here?
> > > > >
> > > > > On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons
> > > > > <ma...@inaf.cnrs-gif.fr>
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for the
> > >
> > > process
> > >
> > > > > > environment it says it can't load /usr/lib/libtiff.so.5.
> > > > > >
> > > > > > On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan
> wrote:
> > > > > > > Are you setting the env in the dockerfile?
> > > > > > >
> > > > > > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
> > > > > >
> > > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > Here's the code I define my executor to mesos with:
> > > > > > > >         val iuwtURI = CommandInfo.URI.newBuilder().
> > >
> > > setValue("http://
> > >
> > > > > > ***/
> > > > > >
> > > > > > > > IUWT.tar.gz").setExtract(true).setCache(false).build()
> > > > > > > >
> > > > > > > >         val iuwtjURI =
> > > > > > > >         CommandInfo.URI.newBuilder().setValue("http://
> > > > > >
> > > > > > ***/
> > > > > >
> > > > > > > > iuwtExecutor-assembly-0.1-
> > > > > > > > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
> > > > > > > >
> > > > > > > >         val iuwtExec = "java -jar
> > > > > > > >         iuwtExecutor-assembly-0.1-SNAPSHOT.jar
> > > > > >
> > > > > > -
> > > > > >
> > > > > > > > Xmx1024M -Xmx128M"
> > > > > > > >
> > > > > > > >         val iuwtCommand =
> > > > > > > >
> > > > > > > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(
> > >
> > > List(iuwtjURI,
> > >
> > > > > > > > iuwtURI).asJava).setShell(true).build()
> > > > > > > >
> > > > > > > >         val iuwtImageInfo =
> > > > > > > >
> > > > > > > > Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> > > > > > > > Image.Docker.newBuilder.setName("ubuntu-
> > > > > > > > mesos:0.11-17102016-IUWT").build()).build()
> > > > > > > >
> > > > > > > >         val iuwtContInfo =
> > > > > > > >
> > > > > > > > ContainerInfo.MesosInfo.newBuilder().setImage(
> > >
> > > iuwtImageInfo).build()
> > >
> > > > > > > >         val iuwtContainer = ContainerInfo.newBuilder()
> > > > > > > >
> > > > > > > >                 .setMesos(iuwtContInfo)
> > > > > > > >
> > > > > > > >           .setType(ContainerInfo.Type.MESOS)
> > > > > > > >           .build()
> > > > > > > >
> > > > > > > >         val iuwtExecutor = ExecutorInfo.newBuilder()
> > > > > > > >
> > > > > > > >                 .setCommand(iuwtCommand)
> > > > > > > >                 .setContainer(iuwtContainer)
> > > > > > > >                 .setExecutorId(ExecutorID.
> > >
> > > newBuilder().setValue("iuw
> > >
> > > > > > > >                 t-
> > > > > > > >
> > > > > > > > executor"))
> > > > > > > >
> > > > > > > >                 .setName("iuwt-executor").build()
> > > > > > > >
> > > > > > > > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted
> docker
> > > > > > > > image.
> > > > > > > > IUWT is
> > > > > > > > the program I'm trying to run, and it runs perfectly fine in
> the
> > > > > > > > aforementioned docker container when running on docker. The
> > > > > >
> > > > > > libtiff.so.5
> > > > > >
> > > > > > > > problem only manifests when I'm using mesos' containerizer.
> > > > > > > >
> > > > > > > > On Monday, October 17, 2016 6:52:17 AM CEST Avinash Sridharan
> > >
> > > wrote:
> > > > > > > > > You are running a container with its own image right? So is
> > > > > > > >
> > > > > > > > /usr/lib/x86_64
> > > > > > > >
> > > > > > > > > in the container's file system or the host file system?
> > > > > > > > >
> > > > > > > > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> > > > > > > >
> > > > > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Yes, it's installed under /usr/lib/x86_64 or whatever the
> > > > > > > > > > multilib
> > > > > > > >
> > > > > > > > path is
> > > > > > > >
> > > > > > > > > > in
> > > > > > > > > > debian. It seems files under this path are not
> accessible.
> > > > > > > > > >
> > > > > > > > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to
> force the
> > > > > >
> > > > > > symlink
> > > > > >
> > > > > > > > to
> > > > > > > >
> > > > > > > > > > load
> > > > > > > > > > but it refused to load it. I think the mesos
> containerizer
> > > > > > > > > > is
> > > > > > > >
> > > > > > > > preventing
> > > > > > > >
> > > > > > > > > > the
> > > > > > > > > > program from accessing anything in a directory under
> > >
> > > /usr/lib/
> > >
> > > > > > > > > > for
> > > > > > > > > > some
> > > > > > > > > > reason, as the same program runs fine in the same
> container
> > > > > > > > > > running
> > > > > > > >
> > > > > > > > under
> > > > > > > >
> > > > > > > > > > docker.
> > > > > > > > > >
> > > > > > > > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash
> > > > > > > > > > Sridharan
> > > > > >
> > > > > > wrote:
> > > > > > > > > > > Is the library part of the image that you are running?
> > > > > > > > > > > Also
> > > > > > > > > > > you
> > > > > > > > > > > might
> > > > > > > > > >
> > > > > > > > > > need
> > > > > > > > > >
> > > > > > > > > > > to setup LD_LIBRARY_PATH in your env while launching
> the
> > >
> > > image
> > >
> > > > > > > > > > > so
> > > > > > > >
> > > > > > > > that
> > > > > > > >
> > > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > container process knows where to look for the shared
> > >
> > > object.
> > >
> > > > > > > > > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > > > > > > > > >
> > > > > > > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi all,
> > > > > > > > > > > >
> > > > > > > > > > > > I've been working with the mesos containerizer to
> handle
> > >
> > > my
> > >
> > > > > > docker
> > > > > >
> > > > > > > > > > > > containers
> > > > > > > > > > > > recently. I created a docker container that requires
> > > > > >
> > > > > > libtiff.so.5,
> > > > > >
> > > > > > > > and
> > > > > > > >
> > > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > application within it runs well. But when I try to
> run
> > > > > > > > > > > > it
> > > > > >
> > > > > > within
> > > > > >
> > > > > > > > the
> > > > > > > >
> > > > > > > > > > mesos
> > > > > > > > > >
> > > > > > > > > > > > containerizer I get an error saying libtiff.so.5
> doesn't
> > > > > > > > > > > > exist.
> > > > > > > > > > > >
> > > > > > > > > > > > The application is being launched via java's process
> > > > > > > > > > > > mechanism
> > > > > > > > > > > > from
> > > > > > > > > >
> > > > > > > > > > inside
> > > > > > > > > >
> > > > > > > > > > > > a
> > > > > > > > > > > > java thread in a custom java executor if that makes a
> > > > > >
> > > > > > difference.
> > > > > >
> > > > > > > > > > > > Any idea what could be causing this change in
> behavior?
> > >
> > > As
> > >
> > > > > > > > > > > > you
> > > > > >
> > > > > > can
> > > > > >
> > > > > > > > see
> > > > > > > >
> > > > > > > > > > in
> > > > > > > > > >
> > > > > > > > > > > > the
> > > > > > > > > > > > attached log file, I check /usr/lib, and a symbolic
> link
> > >
> > > to
> > >
> > > > > > > > > > > > /usr/lib/x86..../
> > > > > > > > > > > > libtiff.so.5 exists in /usr/lib so the program
> should be
> > > > > > > > > > > > able
> > > > > >
> > > > > > to
> > > > > >
> > > > > > > > find
> > > > > > > >
> > > > > > > > > > and
> > > > > > > > > >
> > > > > > > > > > > > load
> > > > > > > > > > > > that....
> > > > > > > > > > > > ----
> > > > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > ----
> > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > > > >
> > > > > > > > --
> > > > > > > > ----
> > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > >
> > > > > > --
> > > > > > ----
> > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > >
> > > --
> > > ----
> > > Mark Edgar Hammons II | +33 06 03 69 56 56
>
>
> --
> ----
> Mark Edgar Hammons II | +33 06 03 69 56 56
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245

Re: Problems with shared libraries with mesos containerization....

Posted by Mark Hammons <ma...@inaf.cnrs-gif.fr>.

No, it's a regular linux log. I've reattached it.

On Monday, October 17, 2016 8:32:07 AM CEST Avinash Sridharan wrote:
> can't seem to open the attached logs, is it gzip?
> 
> On Mon, Oct 17, 2016 at 8:14 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> > wrote:
> > 
> > Adding LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu doesn't fix the error.
> > 
> > On Monday, October 17, 2016 5:06:34 PM CEST Mark Hammons wrote:
> > > As I've shown in the original logs, the symbolic link libtiff.so.5 is in
> > > /usr/ lib. I used LD_PRELOAD just to try to force /usr/lib/libtiff.so.5
> > 
> > to
> > 
> > > be loaded since it wasn't for some reason.
> > > 
> > > On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan wrote:
> > > > Can you prepend your command with `ls -al /usr/lib` and check in
> > 
> > stdout if
> > 
> > > > you are seeing the shared object? By the way why are you using
> > 
> > LD_PRELOAD
> > 
> > > > instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to load
> > 
> > your
> > 
> > > > library before any other libraries. It's usually used to override
> > 
> > system
> > 
> > > > libraries so found a bit odd that you are using it here?
> > > > 
> > > > On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons
> > > > <ma...@inaf.cnrs-gif.fr>
> > > > 
> > > > > wrote:
> > > > > 
> > > > > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for the
> > 
> > process
> > 
> > > > > environment it says it can't load /usr/lib/libtiff.so.5.
> > > > > 
> > > > > On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan wrote:
> > > > > > Are you setting the env in the dockerfile?
> > > > > > 
> > > > > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
> > > > > 
> > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > 
> > > > > > > wrote:
> > > > > > > 
> > > > > > > Here's the code I define my executor to mesos with:
> > > > > > >         val iuwtURI = CommandInfo.URI.newBuilder().
> > 
> > setValue("http://
> > 
> > > > > ***/
> > > > > 
> > > > > > > IUWT.tar.gz").setExtract(true).setCache(false).build()
> > > > > > > 
> > > > > > >         val iuwtjURI =
> > > > > > >         CommandInfo.URI.newBuilder().setValue("http://
> > > > > 
> > > > > ***/
> > > > > 
> > > > > > > iuwtExecutor-assembly-0.1-
> > > > > > > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
> > > > > > > 
> > > > > > >         val iuwtExec = "java -jar
> > > > > > >         iuwtExecutor-assembly-0.1-SNAPSHOT.jar
> > > > > 
> > > > > -
> > > > > 
> > > > > > > Xmx1024M -Xmx128M"
> > > > > > > 
> > > > > > >         val iuwtCommand =
> > > > > > > 
> > > > > > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(
> > 
> > List(iuwtjURI,
> > 
> > > > > > > iuwtURI).asJava).setShell(true).build()
> > > > > > > 
> > > > > > >         val iuwtImageInfo =
> > > > > > > 
> > > > > > > Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> > > > > > > Image.Docker.newBuilder.setName("ubuntu-
> > > > > > > mesos:0.11-17102016-IUWT").build()).build()
> > > > > > > 
> > > > > > >         val iuwtContInfo =
> > > > > > > 
> > > > > > > ContainerInfo.MesosInfo.newBuilder().setImage(
> > 
> > iuwtImageInfo).build()
> > 
> > > > > > >         val iuwtContainer = ContainerInfo.newBuilder()
> > > > > > >         
> > > > > > >                 .setMesos(iuwtContInfo)
> > > > > > >           
> > > > > > >           .setType(ContainerInfo.Type.MESOS)
> > > > > > >           .build()
> > > > > > >         
> > > > > > >         val iuwtExecutor = ExecutorInfo.newBuilder()
> > > > > > >         
> > > > > > >                 .setCommand(iuwtCommand)
> > > > > > >                 .setContainer(iuwtContainer)
> > > > > > >                 .setExecutorId(ExecutorID.
> > 
> > newBuilder().setValue("iuw
> > 
> > > > > > >                 t-
> > > > > > > 
> > > > > > > executor"))
> > > > > > > 
> > > > > > >                 .setName("iuwt-executor").build()
> > > > > > > 
> > > > > > > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted docker
> > > > > > > image.
> > > > > > > IUWT is
> > > > > > > the program I'm trying to run, and it runs perfectly fine in the
> > > > > > > aforementioned docker container when running on docker. The
> > > > > 
> > > > > libtiff.so.5
> > > > > 
> > > > > > > problem only manifests when I'm using mesos' containerizer.
> > > > > > > 
> > > > > > > On Monday, October 17, 2016 6:52:17 AM CEST Avinash Sridharan
> > 
> > wrote:
> > > > > > > > You are running a container with its own image right? So is
> > > > > > > 
> > > > > > > /usr/lib/x86_64
> > > > > > > 
> > > > > > > > in the container's file system or the host file system?
> > > > > > > > 
> > > > > > > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> > > > > > > 
> > > > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > > > 
> > > > > > > > > wrote:
> > > > > > > > > 
> > > > > > > > > Yes, it's installed under /usr/lib/x86_64 or whatever the
> > > > > > > > > multilib
> > > > > > > 
> > > > > > > path is
> > > > > > > 
> > > > > > > > > in
> > > > > > > > > debian. It seems files under this path are not accessible.
> > > > > > > > > 
> > > > > > > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the
> > > > > 
> > > > > symlink
> > > > > 
> > > > > > > to
> > > > > > > 
> > > > > > > > > load
> > > > > > > > > but it refused to load it. I think the mesos containerizer
> > > > > > > > > is
> > > > > > > 
> > > > > > > preventing
> > > > > > > 
> > > > > > > > > the
> > > > > > > > > program from accessing anything in a directory under
> > 
> > /usr/lib/
> > 
> > > > > > > > > for
> > > > > > > > > some
> > > > > > > > > reason, as the same program runs fine in the same container
> > > > > > > > > running
> > > > > > > 
> > > > > > > under
> > > > > > > 
> > > > > > > > > docker.
> > > > > > > > > 
> > > > > > > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash
> > > > > > > > > Sridharan
> > > > > 
> > > > > wrote:
> > > > > > > > > > Is the library part of the image that you are running?
> > > > > > > > > > Also
> > > > > > > > > > you
> > > > > > > > > > might
> > > > > > > > > 
> > > > > > > > > need
> > > > > > > > > 
> > > > > > > > > > to setup LD_LIBRARY_PATH in your env while launching the
> > 
> > image
> > 
> > > > > > > > > > so
> > > > > > > 
> > > > > > > that
> > > > > > > 
> > > > > > > > > the
> > > > > > > > > 
> > > > > > > > > > container process knows where to look for the shared
> > 
> > object.
> > 
> > > > > > > > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > > > > > > > > 
> > > > > > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > > > > > 
> > > > > > > > > > > wrote:
> > > > > > > > > > > 
> > > > > > > > > > > Hi all,
> > > > > > > > > > > 
> > > > > > > > > > > I've been working with the mesos containerizer to handle
> > 
> > my
> > 
> > > > > docker
> > > > > 
> > > > > > > > > > > containers
> > > > > > > > > > > recently. I created a docker container that requires
> > > > > 
> > > > > libtiff.so.5,
> > > > > 
> > > > > > > and
> > > > > > > 
> > > > > > > > > the
> > > > > > > > > 
> > > > > > > > > > > application within it runs well. But when I try to run
> > > > > > > > > > > it
> > > > > 
> > > > > within
> > > > > 
> > > > > > > the
> > > > > > > 
> > > > > > > > > mesos
> > > > > > > > > 
> > > > > > > > > > > containerizer I get an error saying libtiff.so.5 doesn't
> > > > > > > > > > > exist.
> > > > > > > > > > > 
> > > > > > > > > > > The application is being launched via java's process
> > > > > > > > > > > mechanism
> > > > > > > > > > > from
> > > > > > > > > 
> > > > > > > > > inside
> > > > > > > > > 
> > > > > > > > > > > a
> > > > > > > > > > > java thread in a custom java executor if that makes a
> > > > > 
> > > > > difference.
> > > > > 
> > > > > > > > > > > Any idea what could be causing this change in behavior?
> > 
> > As
> > 
> > > > > > > > > > > you
> > > > > 
> > > > > can
> > > > > 
> > > > > > > see
> > > > > > > 
> > > > > > > > > in
> > > > > > > > > 
> > > > > > > > > > > the
> > > > > > > > > > > attached log file, I check /usr/lib, and a symbolic link
> > 
> > to
> > 
> > > > > > > > > > > /usr/lib/x86..../
> > > > > > > > > > > libtiff.so.5 exists in /usr/lib so the program should be
> > > > > > > > > > > able
> > > > > 
> > > > > to
> > > > > 
> > > > > > > find
> > > > > > > 
> > > > > > > > > and
> > > > > > > > > 
> > > > > > > > > > > load
> > > > > > > > > > > that....
> > > > > > > > > > > ----
> > > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > > > > > 
> > > > > > > > > --
> > > > > > > > > ----
> > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > > > 
> > > > > > > --
> > > > > > > ----
> > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > 
> > > > > --
> > > > > ----
> > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > 
> > --
> > ----
> > Mark Edgar Hammons II | +33 06 03 69 56 56


-- 
----
Mark Edgar Hammons II | +33 06 03 69 56 56

Re: Problems with shared libraries with mesos containerization....

Posted by Avinash Sridharan <av...@mesosphere.io>.

can't seem to open the attached logs, is it gzip?

On Mon, Oct 17, 2016 at 8:14 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> wrote:

> Adding LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu doesn't fix the error.
>
> On Monday, October 17, 2016 5:06:34 PM CEST Mark Hammons wrote:
> > As I've shown in the original logs, the symbolic link libtiff.so.5 is in
> > /usr/ lib. I used LD_PRELOAD just to try to force /usr/lib/libtiff.so.5
> to
> > be loaded since it wasn't for some reason.
> >
> > On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan wrote:
> > > Can you prepend your command with `ls -al /usr/lib` and check in
> stdout if
> > > you are seeing the shared object? By the way why are you using
> LD_PRELOAD
> > > instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to load
> your
> > > library before any other libraries. It's usually used to override
> system
> > > libraries so found a bit odd that you are using it here?
> > >
> > > On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons
> > > <ma...@inaf.cnrs-gif.fr>
> > > > wrote:
> > > >
> > > > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for the
> process
> > > > environment it says it can't load /usr/lib/libtiff.so.5.
> > > >
> > > > On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan wrote:
> > > > > Are you setting the env in the dockerfile?
> > > > >
> > > > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
> > > >
> > > > mark.hammons@inaf.cnrs-gif.fr
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > Here's the code I define my executor to mesos with:
> > > > > >         val iuwtURI = CommandInfo.URI.newBuilder().
> setValue("http://
> > > >
> > > > ***/
> > > >
> > > > > > IUWT.tar.gz").setExtract(true).setCache(false).build()
> > > > > >
> > > > > >         val iuwtjURI =
> > > > > >         CommandInfo.URI.newBuilder().setValue("http://
> > > >
> > > > ***/
> > > >
> > > > > > iuwtExecutor-assembly-0.1-
> > > > > > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
> > > > > >
> > > > > >         val iuwtExec = "java -jar
> > > > > >         iuwtExecutor-assembly-0.1-SNAPSHOT.jar
> > > >
> > > > -
> > > >
> > > > > > Xmx1024M -Xmx128M"
> > > > > >
> > > > > >         val iuwtCommand =
> > > > > >
> > > > > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(
> List(iuwtjURI,
> > > > > > iuwtURI).asJava).setShell(true).build()
> > > > > >
> > > > > >         val iuwtImageInfo =
> > > > > >
> > > > > > Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> > > > > > Image.Docker.newBuilder.setName("ubuntu-
> > > > > > mesos:0.11-17102016-IUWT").build()).build()
> > > > > >
> > > > > >         val iuwtContInfo =
> > > > > >
> > > > > > ContainerInfo.MesosInfo.newBuilder().setImage(
> iuwtImageInfo).build()
> > > > > >
> > > > > >         val iuwtContainer = ContainerInfo.newBuilder()
> > > > > >
> > > > > >                 .setMesos(iuwtContInfo)
> > > > > >
> > > > > >           .setType(ContainerInfo.Type.MESOS)
> > > > > >           .build()
> > > > > >
> > > > > >         val iuwtExecutor = ExecutorInfo.newBuilder()
> > > > > >
> > > > > >                 .setCommand(iuwtCommand)
> > > > > >                 .setContainer(iuwtContainer)
> > > > > >                 .setExecutorId(ExecutorID.
> newBuilder().setValue("iuw
> > > > > >                 t-
> > > > > >
> > > > > > executor"))
> > > > > >
> > > > > >                 .setName("iuwt-executor").build()
> > > > > >
> > > > > > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted docker
> > > > > > image.
> > > > > > IUWT is
> > > > > > the program I'm trying to run, and it runs perfectly fine in the
> > > > > > aforementioned docker container when running on docker. The
> > > >
> > > > libtiff.so.5
> > > >
> > > > > > problem only manifests when I'm using mesos' containerizer.
> > > > > >
> > > > > > On Monday, October 17, 2016 6:52:17 AM CEST Avinash Sridharan
> wrote:
> > > > > > > You are running a container with its own image right? So is
> > > > > >
> > > > > > /usr/lib/x86_64
> > > > > >
> > > > > > > in the container's file system or the host file system?
> > > > > > >
> > > > > > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> > > > > >
> > > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > Yes, it's installed under /usr/lib/x86_64 or whatever the
> > > > > > > > multilib
> > > > > >
> > > > > > path is
> > > > > >
> > > > > > > > in
> > > > > > > > debian. It seems files under this path are not accessible.
> > > > > > > >
> > > > > > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the
> > > >
> > > > symlink
> > > >
> > > > > > to
> > > > > >
> > > > > > > > load
> > > > > > > > but it refused to load it. I think the mesos containerizer is
> > > > > >
> > > > > > preventing
> > > > > >
> > > > > > > > the
> > > > > > > > program from accessing anything in a directory under
> /usr/lib/
> > > > > > > > for
> > > > > > > > some
> > > > > > > > reason, as the same program runs fine in the same container
> > > > > > > > running
> > > > > >
> > > > > > under
> > > > > >
> > > > > > > > docker.
> > > > > > > >
> > > > > > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash Sridharan
> > > >
> > > > wrote:
> > > > > > > > > Is the library part of the image that you are running? Also
> > > > > > > > > you
> > > > > > > > > might
> > > > > > > >
> > > > > > > > need
> > > > > > > >
> > > > > > > > > to setup LD_LIBRARY_PATH in your env while launching the
> image
> > > > > > > > > so
> > > > > >
> > > > > > that
> > > > > >
> > > > > > > > the
> > > > > > > >
> > > > > > > > > container process knows where to look for the shared
> object.
> > > > > > > > >
> > > > > > > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > > > > > > >
> > > > > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > > > I've been working with the mesos containerizer to handle
> my
> > > >
> > > > docker
> > > >
> > > > > > > > > > containers
> > > > > > > > > > recently. I created a docker container that requires
> > > >
> > > > libtiff.so.5,
> > > >
> > > > > > and
> > > > > >
> > > > > > > > the
> > > > > > > >
> > > > > > > > > > application within it runs well. But when I try to run it
> > > >
> > > > within
> > > >
> > > > > > the
> > > > > >
> > > > > > > > mesos
> > > > > > > >
> > > > > > > > > > containerizer I get an error saying libtiff.so.5 doesn't
> > > > > > > > > > exist.
> > > > > > > > > >
> > > > > > > > > > The application is being launched via java's process
> > > > > > > > > > mechanism
> > > > > > > > > > from
> > > > > > > >
> > > > > > > > inside
> > > > > > > >
> > > > > > > > > > a
> > > > > > > > > > java thread in a custom java executor if that makes a
> > > >
> > > > difference.
> > > >
> > > > > > > > > > Any idea what could be causing this change in behavior?
> As
> > > > > > > > > > you
> > > >
> > > > can
> > > >
> > > > > > see
> > > > > >
> > > > > > > > in
> > > > > > > >
> > > > > > > > > > the
> > > > > > > > > > attached log file, I check /usr/lib, and a symbolic link
> to
> > > > > > > > > > /usr/lib/x86..../
> > > > > > > > > > libtiff.so.5 exists in /usr/lib so the program should be
> > > > > > > > > > able
> > > >
> > > > to
> > > >
> > > > > > find
> > > > > >
> > > > > > > > and
> > > > > > > >
> > > > > > > > > > load
> > > > > > > > > > that....
> > > > > > > > > > ----
> > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > > > >
> > > > > > > > --
> > > > > > > > ----
> > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > >
> > > > > > --
> > > > > > ----
> > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > >
> > > > --
> > > > ----
> > > > Mark Edgar Hammons II | +33 06 03 69 56 56
>
>
> --
> ----
> Mark Edgar Hammons II | +33 06 03 69 56 56
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245

Re: Problems with shared libraries with mesos containerization....

Posted by Mark Hammons <ma...@inaf.cnrs-gif.fr>.

Adding LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu doesn't fix the error.

On Monday, October 17, 2016 5:06:34 PM CEST Mark Hammons wrote:
> As I've shown in the original logs, the symbolic link libtiff.so.5 is in
> /usr/ lib. I used LD_PRELOAD just to try to force /usr/lib/libtiff.so.5 to
> be loaded since it wasn't for some reason.
> 
> On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan wrote:
> > Can you prepend your command with `ls -al /usr/lib` and check in stdout if
> > you are seeing the shared object? By the way why are you using LD_PRELOAD
> > instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to load your
> > library before any other libraries. It's usually used to override system
> > libraries so found a bit odd that you are using it here?
> > 
> > On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons
> > <ma...@inaf.cnrs-gif.fr> 
> > > wrote:
> > > 
> > > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for the process
> > > environment it says it can't load /usr/lib/libtiff.so.5.
> > > 
> > > On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan wrote:
> > > > Are you setting the env in the dockerfile?
> > > > 
> > > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
> > > 
> > > mark.hammons@inaf.cnrs-gif.fr
> > > 
> > > > > wrote:
> > > > > 
> > > > > Here's the code I define my executor to mesos with:
> > > > >         val iuwtURI = CommandInfo.URI.newBuilder().setValue("http://
> > > 
> > > ***/
> > > 
> > > > > IUWT.tar.gz").setExtract(true).setCache(false).build()
> > > > > 
> > > > >         val iuwtjURI =
> > > > >         CommandInfo.URI.newBuilder().setValue("http://
> > > 
> > > ***/
> > > 
> > > > > iuwtExecutor-assembly-0.1-
> > > > > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
> > > > > 
> > > > >         val iuwtExec = "java -jar
> > > > >         iuwtExecutor-assembly-0.1-SNAPSHOT.jar
> > > 
> > > -
> > > 
> > > > > Xmx1024M -Xmx128M"
> > > > > 
> > > > >         val iuwtCommand =
> > > > > 
> > > > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(List(iuwtjURI,
> > > > > iuwtURI).asJava).setShell(true).build()
> > > > > 
> > > > >         val iuwtImageInfo =
> > > > > 
> > > > > Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> > > > > Image.Docker.newBuilder.setName("ubuntu-
> > > > > mesos:0.11-17102016-IUWT").build()).build()
> > > > > 
> > > > >         val iuwtContInfo =
> > > > > 
> > > > > ContainerInfo.MesosInfo.newBuilder().setImage(iuwtImageInfo).build()
> > > > > 
> > > > >         val iuwtContainer = ContainerInfo.newBuilder()
> > > > >         
> > > > >                 .setMesos(iuwtContInfo)
> > > > >           
> > > > >           .setType(ContainerInfo.Type.MESOS)
> > > > >           .build()
> > > > >         
> > > > >         val iuwtExecutor = ExecutorInfo.newBuilder()
> > > > >         
> > > > >                 .setCommand(iuwtCommand)
> > > > >                 .setContainer(iuwtContainer)
> > > > >                 .setExecutorId(ExecutorID.newBuilder().setValue("iuw
> > > > >                 t-
> > > > > 
> > > > > executor"))
> > > > > 
> > > > >                 .setName("iuwt-executor").build()
> > > > > 
> > > > > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted docker
> > > > > image.
> > > > > IUWT is
> > > > > the program I'm trying to run, and it runs perfectly fine in the
> > > > > aforementioned docker container when running on docker. The
> > > 
> > > libtiff.so.5
> > > 
> > > > > problem only manifests when I'm using mesos' containerizer.
> > > > > 
> > > > > On Monday, October 17, 2016 6:52:17 AM CEST Avinash Sridharan wrote:
> > > > > > You are running a container with its own image right? So is
> > > > > 
> > > > > /usr/lib/x86_64
> > > > > 
> > > > > > in the container's file system or the host file system?
> > > > > > 
> > > > > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> > > > > 
> > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > 
> > > > > > > wrote:
> > > > > > > 
> > > > > > > Yes, it's installed under /usr/lib/x86_64 or whatever the
> > > > > > > multilib
> > > > > 
> > > > > path is
> > > > > 
> > > > > > > in
> > > > > > > debian. It seems files under this path are not accessible.
> > > > > > > 
> > > > > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the
> > > 
> > > symlink
> > > 
> > > > > to
> > > > > 
> > > > > > > load
> > > > > > > but it refused to load it. I think the mesos containerizer is
> > > > > 
> > > > > preventing
> > > > > 
> > > > > > > the
> > > > > > > program from accessing anything in a directory under /usr/lib/
> > > > > > > for
> > > > > > > some
> > > > > > > reason, as the same program runs fine in the same container
> > > > > > > running
> > > > > 
> > > > > under
> > > > > 
> > > > > > > docker.
> > > > > > > 
> > > > > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash Sridharan
> > > 
> > > wrote:
> > > > > > > > Is the library part of the image that you are running? Also
> > > > > > > > you
> > > > > > > > might
> > > > > > > 
> > > > > > > need
> > > > > > > 
> > > > > > > > to setup LD_LIBRARY_PATH in your env while launching the image
> > > > > > > > so
> > > > > 
> > > > > that
> > > > > 
> > > > > > > the
> > > > > > > 
> > > > > > > > container process knows where to look for the shared object.
> > > > > > > > 
> > > > > > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > > > > > > 
> > > > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > > > 
> > > > > > > > > wrote:
> > > > > > > > > 
> > > > > > > > > Hi all,
> > > > > > > > > 
> > > > > > > > > I've been working with the mesos containerizer to handle my
> > > 
> > > docker
> > > 
> > > > > > > > > containers
> > > > > > > > > recently. I created a docker container that requires
> > > 
> > > libtiff.so.5,
> > > 
> > > > > and
> > > > > 
> > > > > > > the
> > > > > > > 
> > > > > > > > > application within it runs well. But when I try to run it
> > > 
> > > within
> > > 
> > > > > the
> > > > > 
> > > > > > > mesos
> > > > > > > 
> > > > > > > > > containerizer I get an error saying libtiff.so.5 doesn't
> > > > > > > > > exist.
> > > > > > > > > 
> > > > > > > > > The application is being launched via java's process
> > > > > > > > > mechanism
> > > > > > > > > from
> > > > > > > 
> > > > > > > inside
> > > > > > > 
> > > > > > > > > a
> > > > > > > > > java thread in a custom java executor if that makes a
> > > 
> > > difference.
> > > 
> > > > > > > > > Any idea what could be causing this change in behavior? As
> > > > > > > > > you
> > > 
> > > can
> > > 
> > > > > see
> > > > > 
> > > > > > > in
> > > > > > > 
> > > > > > > > > the
> > > > > > > > > attached log file, I check /usr/lib, and a symbolic link to
> > > > > > > > > /usr/lib/x86..../
> > > > > > > > > libtiff.so.5 exists in /usr/lib so the program should be
> > > > > > > > > able
> > > 
> > > to
> > > 
> > > > > find
> > > > > 
> > > > > > > and
> > > > > > > 
> > > > > > > > > load
> > > > > > > > > that....
> > > > > > > > > ----
> > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > > > 
> > > > > > > --
> > > > > > > ----
> > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > 
> > > > > --
> > > > > ----
> > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > 
> > > --
> > > ----
> > > Mark Edgar Hammons II | +33 06 03 69 56 56


-- 
----
Mark Edgar Hammons II | +33 06 03 69 56 56

Re: Problems with shared libraries with mesos containerization....

Posted by Mark Hammons <ma...@inaf.cnrs-gif.fr>.

As I've shown in the original logs, the symbolic link libtiff.so.5 is in /usr/
lib. I used LD_PRELOAD just to try to force /usr/lib/libtiff.so.5 to be loaded 
since it wasn't for some reason.

On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan wrote:
> Can you prepend your command with `ls -al /usr/lib` and check in stdout if
> you are seeing the shared object? By the way why are you using LD_PRELOAD
> instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to load your
> library before any other libraries. It's usually used to override system
> libraries so found a bit odd that you are using it here?
> 
> On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> > wrote:
> > 
> > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for the process
> > environment it says it can't load /usr/lib/libtiff.so.5.
> > 
> > On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan wrote:
> > > Are you setting the env in the dockerfile?
> > > 
> > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
> > 
> > mark.hammons@inaf.cnrs-gif.fr
> > 
> > > > wrote:
> > > > 
> > > > Here's the code I define my executor to mesos with:
> > > >         val iuwtURI = CommandInfo.URI.newBuilder().setValue("http://
> > 
> > ***/
> > 
> > > > IUWT.tar.gz").setExtract(true).setCache(false).build()
> > > > 
> > > >         val iuwtjURI = CommandInfo.URI.newBuilder().setValue("http://
> > 
> > ***/
> > 
> > > > iuwtExecutor-assembly-0.1-
> > > > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
> > > > 
> > > >         val iuwtExec = "java -jar
> > > >         iuwtExecutor-assembly-0.1-SNAPSHOT.jar
> > 
> > -
> > 
> > > > Xmx1024M -Xmx128M"
> > > > 
> > > >         val iuwtCommand =
> > > > 
> > > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(List(iuwtjURI,
> > > > iuwtURI).asJava).setShell(true).build()
> > > > 
> > > >         val iuwtImageInfo =
> > > > 
> > > > Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> > > > Image.Docker.newBuilder.setName("ubuntu-
> > > > mesos:0.11-17102016-IUWT").build()).build()
> > > > 
> > > >         val iuwtContInfo =
> > > > 
> > > > ContainerInfo.MesosInfo.newBuilder().setImage(iuwtImageInfo).build()
> > > > 
> > > >         val iuwtContainer = ContainerInfo.newBuilder()
> > > >         
> > > >                 .setMesos(iuwtContInfo)
> > > >           
> > > >           .setType(ContainerInfo.Type.MESOS)
> > > >           .build()
> > > >         
> > > >         val iuwtExecutor = ExecutorInfo.newBuilder()
> > > >         
> > > >                 .setCommand(iuwtCommand)
> > > >                 .setContainer(iuwtContainer)
> > > >                 .setExecutorId(ExecutorID.newBuilder().setValue("iuwt-
> > > > 
> > > > executor"))
> > > > 
> > > >                 .setName("iuwt-executor").build()
> > > > 
> > > > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted docker image.
> > > > IUWT is
> > > > the program I'm trying to run, and it runs perfectly fine in the
> > > > aforementioned docker container when running on docker. The
> > 
> > libtiff.so.5
> > 
> > > > problem only manifests when I'm using mesos' containerizer.
> > > > 
> > > > On Monday, October 17, 2016 6:52:17 AM CEST Avinash Sridharan wrote:
> > > > > You are running a container with its own image right? So is
> > > > 
> > > > /usr/lib/x86_64
> > > > 
> > > > > in the container's file system or the host file system?
> > > > > 
> > > > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> > > > 
> > > > mark.hammons@inaf.cnrs-gif.fr
> > > > 
> > > > > > wrote:
> > > > > > 
> > > > > > Yes, it's installed under /usr/lib/x86_64 or whatever the multilib
> > > > 
> > > > path is
> > > > 
> > > > > > in
> > > > > > debian. It seems files under this path are not accessible.
> > > > > > 
> > > > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the
> > 
> > symlink
> > 
> > > > to
> > > > 
> > > > > > load
> > > > > > but it refused to load it. I think the mesos containerizer is
> > > > 
> > > > preventing
> > > > 
> > > > > > the
> > > > > > program from accessing anything in a directory under /usr/lib/ for
> > > > > > some
> > > > > > reason, as the same program runs fine in the same container
> > > > > > running
> > > > 
> > > > under
> > > > 
> > > > > > docker.
> > > > > > 
> > > > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash Sridharan
> > 
> > wrote:
> > > > > > > Is the library part of the image that you are running? Also you
> > > > > > > might
> > > > > > 
> > > > > > need
> > > > > > 
> > > > > > > to setup LD_LIBRARY_PATH in your env while launching the image
> > > > > > > so
> > > > 
> > > > that
> > > > 
> > > > > > the
> > > > > > 
> > > > > > > container process knows where to look for the shared object.
> > > > > > > 
> > > > > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > > > > > 
> > > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > > > 
> > > > > > > > wrote:
> > > > > > > > 
> > > > > > > > Hi all,
> > > > > > > > 
> > > > > > > > I've been working with the mesos containerizer to handle my
> > 
> > docker
> > 
> > > > > > > > containers
> > > > > > > > recently. I created a docker container that requires
> > 
> > libtiff.so.5,
> > 
> > > > and
> > > > 
> > > > > > the
> > > > > > 
> > > > > > > > application within it runs well. But when I try to run it
> > 
> > within
> > 
> > > > the
> > > > 
> > > > > > mesos
> > > > > > 
> > > > > > > > containerizer I get an error saying libtiff.so.5 doesn't
> > > > > > > > exist.
> > > > > > > > 
> > > > > > > > The application is being launched via java's process mechanism
> > > > > > > > from
> > > > > > 
> > > > > > inside
> > > > > > 
> > > > > > > > a
> > > > > > > > java thread in a custom java executor if that makes a
> > 
> > difference.
> > 
> > > > > > > > Any idea what could be causing this change in behavior? As you
> > 
> > can
> > 
> > > > see
> > > > 
> > > > > > in
> > > > > > 
> > > > > > > > the
> > > > > > > > attached log file, I check /usr/lib, and a symbolic link to
> > > > > > > > /usr/lib/x86..../
> > > > > > > > libtiff.so.5 exists in /usr/lib so the program should be able
> > 
> > to
> > 
> > > > find
> > > > 
> > > > > > and
> > > > > > 
> > > > > > > > load
> > > > > > > > that....
> > > > > > > > ----
> > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > > > 
> > > > > > --
> > > > > > ----
> > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > 
> > > > --
> > > > ----
> > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > 
> > --
> > ----
> > Mark Edgar Hammons II | +33 06 03 69 56 56


-- 
----
Mark Edgar Hammons II | +33 06 03 69 56 56

Re: Problems with shared libraries with mesos containerization....

Posted by Avinash Sridharan <av...@mesosphere.io>.

Can you prepend your command with `ls -al /usr/lib` and check in stdout if
you are seeing the shared object? By the way why are you using LD_PRELOAD
instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to load your
library before any other libraries. It's usually used to override system
libraries so found a bit odd that you are using it here?

On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> wrote:

> No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for the process
> environment it says it can't load /usr/lib/libtiff.so.5.
>
> On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan wrote:
> > Are you setting the env in the dockerfile?
> >
> > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <
> mark.hammons@inaf.cnrs-gif.fr
> > > wrote:
> > >
> > > Here's the code I define my executor to mesos with:
> > >         val iuwtURI = CommandInfo.URI.newBuilder().setValue("http://
> ***/
> > >
> > > IUWT.tar.gz").setExtract(true).setCache(false).build()
> > >
> > >         val iuwtjURI = CommandInfo.URI.newBuilder().setValue("http://
> ***/
> > >
> > > iuwtExecutor-assembly-0.1-
> > > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
> > >
> > >         val iuwtExec = "java -jar iuwtExecutor-assembly-0.1-SNAPSHOT.jar
> -
> > >
> > > Xmx1024M -Xmx128M"
> > >
> > >         val iuwtCommand =
> > >
> > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(List(iuwtjURI,
> > > iuwtURI).asJava).setShell(true).build()
> > >
> > >         val iuwtImageInfo =
> > >
> > > Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> > > Image.Docker.newBuilder.setName("ubuntu-
> > > mesos:0.11-17102016-IUWT").build()).build()
> > >
> > >         val iuwtContInfo =
> > >
> > > ContainerInfo.MesosInfo.newBuilder().setImage(iuwtImageInfo).build()
> > >
> > >         val iuwtContainer = ContainerInfo.newBuilder()
> > >
> > >                 .setMesos(iuwtContInfo)
> > >
> > >           .setType(ContainerInfo.Type.MESOS)
> > >           .build()
> > >
> > >         val iuwtExecutor = ExecutorInfo.newBuilder()
> > >
> > >                 .setCommand(iuwtCommand)
> > >                 .setContainer(iuwtContainer)
> > >                 .setExecutorId(ExecutorID.newBuilder().setValue("iuwt-
> > >
> > > executor"))
> > >
> > >                 .setName("iuwt-executor").build()
> > >
> > > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted docker image.
> > > IUWT is
> > > the program I'm trying to run, and it runs perfectly fine in the
> > > aforementioned docker container when running on docker. The
> libtiff.so.5
> > > problem only manifests when I'm using mesos' containerizer.
> > >
> > > On Monday, October 17, 2016 6:52:17 AM CEST Avinash Sridharan wrote:
> > > > You are running a container with its own image right? So is
> > >
> > > /usr/lib/x86_64
> > >
> > > > in the container's file system or the host file system?
> > > >
> > > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> > >
> > > mark.hammons@inaf.cnrs-gif.fr
> > >
> > > > > wrote:
> > > > >
> > > > > Yes, it's installed under /usr/lib/x86_64 or whatever the multilib
> > >
> > > path is
> > >
> > > > > in
> > > > > debian. It seems files under this path are not accessible.
> > > > >
> > > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the
> symlink
> > >
> > > to
> > >
> > > > > load
> > > > > but it refused to load it. I think the mesos containerizer is
> > >
> > > preventing
> > >
> > > > > the
> > > > > program from accessing anything in a directory under /usr/lib/ for
> > > > > some
> > > > > reason, as the same program runs fine in the same container running
> > >
> > > under
> > >
> > > > > docker.
> > > > >
> > > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash Sridharan
> wrote:
> > > > > > Is the library part of the image that you are running? Also you
> > > > > > might
> > > > >
> > > > > need
> > > > >
> > > > > > to setup LD_LIBRARY_PATH in your env while launching the image so
> > >
> > > that
> > >
> > > > > the
> > > > >
> > > > > > container process knows where to look for the shared object.
> > > > > >
> > > > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > > > >
> > > > > mark.hammons@inaf.cnrs-gif.fr
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > I've been working with the mesos containerizer to handle my
> docker
> > > > > > > containers
> > > > > > > recently. I created a docker container that requires
> libtiff.so.5,
> > >
> > > and
> > >
> > > > > the
> > > > >
> > > > > > > application within it runs well. But when I try to run it
> within
> > >
> > > the
> > >
> > > > > mesos
> > > > >
> > > > > > > containerizer I get an error saying libtiff.so.5 doesn't exist.
> > > > > > >
> > > > > > > The application is being launched via java's process mechanism
> > > > > > > from
> > > > >
> > > > > inside
> > > > >
> > > > > > > a
> > > > > > > java thread in a custom java executor if that makes a
> difference.
> > > > > > >
> > > > > > > Any idea what could be causing this change in behavior? As you
> can
> > >
> > > see
> > >
> > > > > in
> > > > >
> > > > > > > the
> > > > > > > attached log file, I check /usr/lib, and a symbolic link to
> > > > > > > /usr/lib/x86..../
> > > > > > > libtiff.so.5 exists in /usr/lib so the program should be able
> to
> > >
> > > find
> > >
> > > > > and
> > > > >
> > > > > > > load
> > > > > > > that....
> > > > > > > ----
> > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > >
> > > > > --
> > > > > ----
> > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > >
> > > --
> > > ----
> > > Mark Edgar Hammons II | +33 06 03 69 56 56
>
>
> --
> ----
> Mark Edgar Hammons II | +33 06 03 69 56 56
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245

Re: Problems with shared libraries with mesos containerization....

Posted by Mark Hammons <ma...@inaf.cnrs-gif.fr>.

No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for the process 
environment it says it can't load /usr/lib/libtiff.so.5.

On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan wrote:
> Are you setting the env in the dockerfile?
> 
> On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> > wrote:
> > 
> > Here's the code I define my executor to mesos with:
> >         val iuwtURI = CommandInfo.URI.newBuilder().setValue("http://***/
> > 
> > IUWT.tar.gz").setExtract(true).setCache(false).build()
> > 
> >         val iuwtjURI = CommandInfo.URI.newBuilder().setValue("http://***/
> > 
> > iuwtExecutor-assembly-0.1-
> > SNAPSHOT.jar").setExecutable(false).setCache(false).build()
> > 
> >         val iuwtExec = "java -jar iuwtExecutor-assembly-0.1-SNAPSHOT.jar -
> > 
> > Xmx1024M -Xmx128M"
> > 
> >         val iuwtCommand =
> > 
> > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(List(iuwtjURI,
> > iuwtURI).asJava).setShell(true).build()
> > 
> >         val iuwtImageInfo =
> > 
> > Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> > Image.Docker.newBuilder.setName("ubuntu-
> > mesos:0.11-17102016-IUWT").build()).build()
> > 
> >         val iuwtContInfo =
> > 
> > ContainerInfo.MesosInfo.newBuilder().setImage(iuwtImageInfo).build()
> > 
> >         val iuwtContainer = ContainerInfo.newBuilder()
> >         
> >                 .setMesos(iuwtContInfo)
> >           
> >           .setType(ContainerInfo.Type.MESOS)
> >           .build()
> >         
> >         val iuwtExecutor = ExecutorInfo.newBuilder()
> >         
> >                 .setCommand(iuwtCommand)
> >                 .setContainer(iuwtContainer)
> >                 .setExecutorId(ExecutorID.newBuilder().setValue("iuwt-
> > 
> > executor"))
> > 
> >                 .setName("iuwt-executor").build()
> > 
> > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted docker image.
> > IUWT is
> > the program I'm trying to run, and it runs perfectly fine in the
> > aforementioned docker container when running on docker. The libtiff.so.5
> > problem only manifests when I'm using mesos' containerizer.
> > 
> > On Monday, October 17, 2016 6:52:17 AM CEST Avinash Sridharan wrote:
> > > You are running a container with its own image right? So is
> > 
> > /usr/lib/x86_64
> > 
> > > in the container's file system or the host file system?
> > > 
> > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> > 
> > mark.hammons@inaf.cnrs-gif.fr
> > 
> > > > wrote:
> > > > 
> > > > Yes, it's installed under /usr/lib/x86_64 or whatever the multilib
> > 
> > path is
> > 
> > > > in
> > > > debian. It seems files under this path are not accessible.
> > > > 
> > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the symlink
> > 
> > to
> > 
> > > > load
> > > > but it refused to load it. I think the mesos containerizer is
> > 
> > preventing
> > 
> > > > the
> > > > program from accessing anything in a directory under /usr/lib/ for
> > > > some
> > > > reason, as the same program runs fine in the same container running
> > 
> > under
> > 
> > > > docker.
> > > > 
> > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash Sridharan wrote:
> > > > > Is the library part of the image that you are running? Also you
> > > > > might
> > > > 
> > > > need
> > > > 
> > > > > to setup LD_LIBRARY_PATH in your env while launching the image so
> > 
> > that
> > 
> > > > the
> > > > 
> > > > > container process knows where to look for the shared object.
> > > > > 
> > > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > > > 
> > > > mark.hammons@inaf.cnrs-gif.fr
> > > > 
> > > > > > wrote:
> > > > > > 
> > > > > > Hi all,
> > > > > > 
> > > > > > I've been working with the mesos containerizer to handle my docker
> > > > > > containers
> > > > > > recently. I created a docker container that requires libtiff.so.5,
> > 
> > and
> > 
> > > > the
> > > > 
> > > > > > application within it runs well. But when I try to run it within
> > 
> > the
> > 
> > > > mesos
> > > > 
> > > > > > containerizer I get an error saying libtiff.so.5 doesn't exist.
> > > > > > 
> > > > > > The application is being launched via java's process mechanism
> > > > > > from
> > > > 
> > > > inside
> > > > 
> > > > > > a
> > > > > > java thread in a custom java executor if that makes a difference.
> > > > > > 
> > > > > > Any idea what could be causing this change in behavior? As you can
> > 
> > see
> > 
> > > > in
> > > > 
> > > > > > the
> > > > > > attached log file, I check /usr/lib, and a symbolic link to
> > > > > > /usr/lib/x86..../
> > > > > > libtiff.so.5 exists in /usr/lib so the program should be able to
> > 
> > find
> > 
> > > > and
> > > > 
> > > > > > load
> > > > > > that....
> > > > > > ----
> > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > > > 
> > > > --
> > > > ----
> > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > 
> > --
> > ----
> > Mark Edgar Hammons II | +33 06 03 69 56 56


-- 
----
Mark Edgar Hammons II | +33 06 03 69 56 56

Re: Problems with shared libraries with mesos containerization....

Posted by Avinash Sridharan <av...@mesosphere.io>.

Are you setting the env in the dockerfile?

On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> wrote:

> Here's the code I define my executor to mesos with:
>
>         val iuwtURI = CommandInfo.URI.newBuilder().setValue("http://***/
> IUWT.tar.gz").setExtract(true).setCache(false).build()
>
>         val iuwtjURI = CommandInfo.URI.newBuilder().setValue("http://***/
> iuwtExecutor-assembly-0.1-
> SNAPSHOT.jar").setExecutable(false).setCache(false).build()
>
>         val iuwtExec = "java -jar iuwtExecutor-assembly-0.1-SNAPSHOT.jar -
> Xmx1024M -Xmx128M"
>
>         val iuwtCommand =
> CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(List(iuwtjURI,
> iuwtURI).asJava).setShell(true).build()
>
>         val iuwtImageInfo =
> Image.newBuilder().setType(Image.Type.DOCKER).setDocker(
> Image.Docker.newBuilder.setName("ubuntu-
> mesos:0.11-17102016-IUWT").build()).build()
>
>         val iuwtContInfo =
> ContainerInfo.MesosInfo.newBuilder().setImage(iuwtImageInfo).build()
>
>
>         val iuwtContainer = ContainerInfo.newBuilder()
>                 .setMesos(iuwtContInfo)
>           .setType(ContainerInfo.Type.MESOS)
>           .build()
>
>         val iuwtExecutor = ExecutorInfo.newBuilder()
>                 .setCommand(iuwtCommand)
>                 .setContainer(iuwtContainer)
>                 .setExecutorId(ExecutorID.newBuilder().setValue("iuwt-
> executor"))
>                 .setName("iuwt-executor").build()
>
> The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted docker image.
> IUWT is
> the program I'm trying to run, and it runs perfectly fine in the
> aforementioned docker container when running on docker. The libtiff.so.5
> problem only manifests when I'm using mesos' containerizer.
>
> On Monday, October 17, 2016 6:52:17 AM CEST Avinash Sridharan wrote:
> > You are running a container with its own image right? So is
> /usr/lib/x86_64
> > in the container's file system or the host file system?
> >
> > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <
> mark.hammons@inaf.cnrs-gif.fr
> > > wrote:
> > >
> > > Yes, it's installed under /usr/lib/x86_64 or whatever the multilib
> path is
> > > in
> > > debian. It seems files under this path are not accessible.
> > >
> > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the symlink
> to
> > > load
> > > but it refused to load it. I think the mesos containerizer is
> preventing
> > > the
> > > program from accessing anything in a directory under /usr/lib/ for some
> > > reason, as the same program runs fine in the same container running
> under
> > > docker.
> > >
> > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash Sridharan wrote:
> > > > Is the library part of the image that you are running? Also you might
> > >
> > > need
> > >
> > > > to setup LD_LIBRARY_PATH in your env while launching the image so
> that
> > >
> > > the
> > >
> > > > container process knows where to look for the shared object.
> > > >
> > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > >
> > > mark.hammons@inaf.cnrs-gif.fr
> > >
> > > > > wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > I've been working with the mesos containerizer to handle my docker
> > > > > containers
> > > > > recently. I created a docker container that requires libtiff.so.5,
> and
> > >
> > > the
> > >
> > > > > application within it runs well. But when I try to run it within
> the
> > >
> > > mesos
> > >
> > > > > containerizer I get an error saying libtiff.so.5 doesn't exist.
> > > > >
> > > > > The application is being launched via java's process mechanism from
> > >
> > > inside
> > >
> > > > > a
> > > > > java thread in a custom java executor if that makes a difference.
> > > > >
> > > > > Any idea what could be causing this change in behavior? As you can
> see
> > >
> > > in
> > >
> > > > > the
> > > > > attached log file, I check /usr/lib, and a symbolic link to
> > > > > /usr/lib/x86..../
> > > > > libtiff.so.5 exists in /usr/lib so the program should be able to
> find
> > >
> > > and
> > >
> > > > > load
> > > > > that....
> > > > > ----
> > > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > >
> > > --
> > > ----
> > > Mark Edgar Hammons II | +33 06 03 69 56 56
>
>
> --
> ----
> Mark Edgar Hammons II | +33 06 03 69 56 56
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245

Re: Problems with shared libraries with mesos containerization....

Posted by Mark Hammons <ma...@inaf.cnrs-gif.fr>.

Here's the code I define my executor to mesos with:

	val iuwtURI = CommandInfo.URI.newBuilder().setValue("http://***/
IUWT.tar.gz").setExtract(true).setCache(false).build()

	val iuwtjURI = CommandInfo.URI.newBuilder().setValue("http://***/
iuwtExecutor-assembly-0.1-
SNAPSHOT.jar").setExecutable(false).setCache(false).build()

	val iuwtExec = "java -jar iuwtExecutor-assembly-0.1-SNAPSHOT.jar -
Xmx1024M -Xmx128M"

	val iuwtCommand = 
CommandInfo.newBuilder.setValue(iuwtExec).addAllUris(List(iuwtjURI, 
iuwtURI).asJava).setShell(true).build()

	val iuwtImageInfo = 
Image.newBuilder().setType(Image.Type.DOCKER).setDocker(Image.Docker.newBuilder.setName("ubuntu-
mesos:0.11-17102016-IUWT").build()).build()

	val iuwtContInfo = 
ContainerInfo.MesosInfo.newBuilder().setImage(iuwtImageInfo).build()


	val iuwtContainer = ContainerInfo.newBuilder()
		.setMesos(iuwtContInfo)
	  .setType(ContainerInfo.Type.MESOS)
	  .build()

	val iuwtExecutor = ExecutorInfo.newBuilder()
		.setCommand(iuwtCommand)
		.setContainer(iuwtContainer)
		.setExecutorId(ExecutorID.newBuilder().setValue("iuwt-executor"))
		.setName("iuwt-executor").build()

The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted docker image. IUWT is 
the program I'm trying to run, and it runs perfectly fine in the 
aforementioned docker container when running on docker. The libtiff.so.5 
problem only manifests when I'm using mesos' containerizer.

On Monday, October 17, 2016 6:52:17 AM CEST Avinash Sridharan wrote:
> You are running a container with its own image right? So is  /usr/lib/x86_64
> in the container's file system or the host file system?
> 
> On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> > wrote:
> > 
> > Yes, it's installed under /usr/lib/x86_64 or whatever the multilib path is
> > in
> > debian. It seems files under this path are not accessible.
> > 
> > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the symlink to
> > load
> > but it refused to load it. I think the mesos containerizer is preventing
> > the
> > program from accessing anything in a directory under /usr/lib/ for some
> > reason, as the same program runs fine in the same container running under
> > docker.
> > 
> > On Monday, October 17, 2016 6:40:49 AM CEST Avinash Sridharan wrote:
> > > Is the library part of the image that you are running? Also you might
> > 
> > need
> > 
> > > to setup LD_LIBRARY_PATH in your env while launching the image so that
> > 
> > the
> > 
> > > container process knows where to look for the shared object.
> > > 
> > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > 
> > mark.hammons@inaf.cnrs-gif.fr
> > 
> > > > wrote:
> > > > 
> > > > Hi all,
> > > > 
> > > > I've been working with the mesos containerizer to handle my docker
> > > > containers
> > > > recently. I created a docker container that requires libtiff.so.5, and
> > 
> > the
> > 
> > > > application within it runs well. But when I try to run it within the
> > 
> > mesos
> > 
> > > > containerizer I get an error saying libtiff.so.5 doesn't exist.
> > > > 
> > > > The application is being launched via java's process mechanism from
> > 
> > inside
> > 
> > > > a
> > > > java thread in a custom java executor if that makes a difference.
> > > > 
> > > > Any idea what could be causing this change in behavior? As you can see
> > 
> > in
> > 
> > > > the
> > > > attached log file, I check /usr/lib, and a symbolic link to
> > > > /usr/lib/x86..../
> > > > libtiff.so.5 exists in /usr/lib so the program should be able to find
> > 
> > and
> > 
> > > > load
> > > > that....
> > > > ----
> > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > 
> > --
> > ----
> > Mark Edgar Hammons II | +33 06 03 69 56 56


-- 
----
Mark Edgar Hammons II | +33 06 03 69 56 56

Re: Problems with shared libraries with mesos containerization....

Posted by Mark Hammons <ma...@inaf.cnrs-gif.fr>.

Inside the container's file system. I was hoping that as far as my application 
is concerned my host file system doesn't exist.

On Monday, October 17, 2016 6:52:17 AM CEST Avinash Sridharan wrote:
> You are running a container with its own image right? So is  /usr/lib/x86_64
> in the container's file system or the host file system?
> 
> On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> > wrote:
> > 
> > Yes, it's installed under /usr/lib/x86_64 or whatever the multilib path is
> > in
> > debian. It seems files under this path are not accessible.
> > 
> > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the symlink to
> > load
> > but it refused to load it. I think the mesos containerizer is preventing
> > the
> > program from accessing anything in a directory under /usr/lib/ for some
> > reason, as the same program runs fine in the same container running under
> > docker.
> > 
> > On Monday, October 17, 2016 6:40:49 AM CEST Avinash Sridharan wrote:
> > > Is the library part of the image that you are running? Also you might
> > 
> > need
> > 
> > > to setup LD_LIBRARY_PATH in your env while launching the image so that
> > 
> > the
> > 
> > > container process knows where to look for the shared object.
> > > 
> > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> > 
> > mark.hammons@inaf.cnrs-gif.fr
> > 
> > > > wrote:
> > > > 
> > > > Hi all,
> > > > 
> > > > I've been working with the mesos containerizer to handle my docker
> > > > containers
> > > > recently. I created a docker container that requires libtiff.so.5, and
> > 
> > the
> > 
> > > > application within it runs well. But when I try to run it within the
> > 
> > mesos
> > 
> > > > containerizer I get an error saying libtiff.so.5 doesn't exist.
> > > > 
> > > > The application is being launched via java's process mechanism from
> > 
> > inside
> > 
> > > > a
> > > > java thread in a custom java executor if that makes a difference.
> > > > 
> > > > Any idea what could be causing this change in behavior? As you can see
> > 
> > in
> > 
> > > > the
> > > > attached log file, I check /usr/lib, and a symbolic link to
> > > > /usr/lib/x86..../
> > > > libtiff.so.5 exists in /usr/lib so the program should be able to find
> > 
> > and
> > 
> > > > load
> > > > that....
> > > > ----
> > > > Mark Edgar Hammons II | +33 06 03 69 56 56
> > 
> > --
> > ----
> > Mark Edgar Hammons II | +33 06 03 69 56 56


-- 
----
Mark Edgar Hammons II | +33 06 03 69 56 56

Re: Problems with shared libraries with mesos containerization....

Posted by Avinash Sridharan <av...@mesosphere.io>.

You are running a container with its own image right? So is  /usr/lib/x86_64
in the container's file system or the host file system?

On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> wrote:

> Yes, it's installed under /usr/lib/x86_64 or whatever the multilib path is
> in
> debian. It seems files under this path are not accessible.
>
> I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the symlink to
> load
> but it refused to load it. I think the mesos containerizer is preventing
> the
> program from accessing anything in a directory under /usr/lib/ for some
> reason, as the same program runs fine in the same container running under
> docker.
>
> On Monday, October 17, 2016 6:40:49 AM CEST Avinash Sridharan wrote:
> > Is the library part of the image that you are running? Also you might
> need
> > to setup LD_LIBRARY_PATH in your env while launching the image so that
> the
> > container process knows where to look for the shared object.
> >
> > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <
> mark.hammons@inaf.cnrs-gif.fr
> > > wrote:
> > >
> > > Hi all,
> > >
> > > I've been working with the mesos containerizer to handle my docker
> > > containers
> > > recently. I created a docker container that requires libtiff.so.5, and
> the
> > > application within it runs well. But when I try to run it within the
> mesos
> > > containerizer I get an error saying libtiff.so.5 doesn't exist.
> > >
> > > The application is being launched via java's process mechanism from
> inside
> > > a
> > > java thread in a custom java executor if that makes a difference.
> > >
> > > Any idea what could be causing this change in behavior? As you can see
> in
> > > the
> > > attached log file, I check /usr/lib, and a symbolic link to
> > > /usr/lib/x86..../
> > > libtiff.so.5 exists in /usr/lib so the program should be able to find
> and
> > > load
> > > that....
> > > ----
> > > Mark Edgar Hammons II | +33 06 03 69 56 56
>
>
> --
> ----
> Mark Edgar Hammons II | +33 06 03 69 56 56
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245

Re: Problems with shared libraries with mesos containerization....

Posted by Mark Hammons <ma...@inaf.cnrs-gif.fr>.

Yes, it's installed under /usr/lib/x86_64 or whatever the multilib path is in 
debian. It seems files under this path are not accessible.

I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to force the symlink to load 
but it refused to load it. I think the mesos containerizer is preventing the 
program from accessing anything in a directory under /usr/lib/ for some 
reason, as the same program runs fine in the same container running under 
docker.

On Monday, October 17, 2016 6:40:49 AM CEST Avinash Sridharan wrote:
> Is the library part of the image that you are running? Also you might need
> to setup LD_LIBRARY_PATH in your env while launching the image so that the
> container process knows where to look for the shared object.
> 
> On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> > wrote:
> > 
> > Hi all,
> > 
> > I've been working with the mesos containerizer to handle my docker
> > containers
> > recently. I created a docker container that requires libtiff.so.5, and the
> > application within it runs well. But when I try to run it within the mesos
> > containerizer I get an error saying libtiff.so.5 doesn't exist.
> > 
> > The application is being launched via java's process mechanism from inside
> > a
> > java thread in a custom java executor if that makes a difference.
> > 
> > Any idea what could be causing this change in behavior? As you can see in
> > the
> > attached log file, I check /usr/lib, and a symbolic link to
> > /usr/lib/x86..../
> > libtiff.so.5 exists in /usr/lib so the program should be able to find and
> > load
> > that....
> > ----
> > Mark Edgar Hammons II | +33 06 03 69 56 56


-- 
----
Mark Edgar Hammons II | +33 06 03 69 56 56

Re: Problems with shared libraries with mesos containerization....

Posted by Avinash Sridharan <av...@mesosphere.io>.

Is the library part of the image that you are running? Also you might need
to setup LD_LIBRARY_PATH in your env while launching the image so that the
container process knows where to look for the shared object.

On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> wrote:

> Hi all,
>
> I've been working with the mesos containerizer to handle my docker
> containers
> recently. I created a docker container that requires libtiff.so.5, and the
> application within it runs well. But when I try to run it within the mesos
> containerizer I get an error saying libtiff.so.5 doesn't exist.
>
> The application is being launched via java's process mechanism from inside
> a
> java thread in a custom java executor if that makes a difference.
>
> Any idea what could be causing this change in behavior? As you can see in
> the
> attached log file, I check /usr/lib, and a symbolic link to
> /usr/lib/x86..../
> libtiff.so.5 exists in /usr/lib so the program should be able to find and
> load
> that....
> ----
> Mark Edgar Hammons II | +33 06 03 69 56 56




-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245