Re: issue running toree kernel on mesos cluster

[Paco Nathan <pn...@oreilly.com>]

I'll chime in on this, too -

We run Jupyter kernels inside Docker containers, running on an Apache Mesos
cluster. This is used for online publishing
<http://www.oreilly.com/oriole/index.html>.

Spark and related projects are popular among our authors at O'Reilly, so
we've got a big need to be using Toree.  Works great when run on laptops,
though with an Apache 2.x version of Toree inside a container we encounter
errors -- as Eszti described above,

Some of our ops team at O'Reilly can rewrite the build, to work around
these runtime errors in Docker. We'd like to contribute that back to the
Toree project, rather than maintain separately.

First we wanted to check with Toree devs, to see if that fits, doesn't
conflict with your plans, etc. ?  Also, have there been any related bug
reports -- might be useful.

Thanks,
Paco

PS: we may have some history here, not 100% sure: saw the commit from Osh
-- over a year ago we had Jeremy Freeman's team (mostly Osh
<https://github.com/andrewosh>) consult to develop a tutorial based on
Spark in a container, and I have a hunch some of that code may have gone
into Toree?

Re: issue running toree kernel on mesos cluster

[Chip Senkbeil <ch...@gmail.com>]

For some reason, I'm missing the first half of this thread. I only have
Paco Nathan's email. I don't have one from Eszti and I don't see one in our
mailing archive, either. Can someone repost his part?

As for the runtime errors in Docker, we'd be happy to accept any and all
contributions back into the main project.

In terms of related reports, I don't remember any with regard to Docker,
but I'd need to look back through our JIRAs to be certain.

On Wed, Jan 11, 2017 at 6:25 PM Paco Nathan <pn...@oreilly.com> wrote:

> I'll chime in on this, too -
>
> We run Jupyter kernels inside Docker containers, running on an Apache Mesos
> cluster. This is used for online publishing
> <http://www.oreilly.com/oriole/index.html>.
>
> Spark and related projects are popular among our authors at O'Reilly, so
> we've got a big need to be using Toree.  Works great when run on laptops,
> though with an Apache 2.x version of Toree inside a container we encounter
> errors -- as Eszti described above,
>
> Some of our ops team at O'Reilly can rewrite the build, to work around
> these runtime errors in Docker. We'd like to contribute that back to the
> Toree project, rather than maintain separately.
>
> First we wanted to check with Toree devs, to see if that fits, doesn't
> conflict with your plans, etc. ?  Also, have there been any related bug
> reports -- might be useful.
>
> Thanks,
> Paco
>
> PS: we may have some history here, not 100% sure: saw the commit from Osh
> -- over a year ago we had Jeremy Freeman's team (mostly Osh
> <https://github.com/andrewosh>) consult to develop a tutorial based on
> Spark in a container, and I have a hunch some of that code may have gone
> into Toree?
>