You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@karaf.apache.org by ra...@enjekt.org on 2019/02/06 15:06:48 UTC
Microkernel Karaf fit...
Is anyone aware of a microkernel running a JVM/Karaf? I think Karaf could
really take advantage of microkernels avoiding the need for Kubernetes,
Docker and so on. Karaf is uniquely qualified for this by the fact that it
has its own hooks to repositories, a console and monitoring with things like
HawtIO. If the JVM/Karaf/Felix is running in the kernel itself and that is
running on the hypervisor, there's not a lot of overhead. That's a stark
contrast to the world of Kubernetes/Docker/VM/hypervisor.
With microkernels and rump kernels getting a lot of attention and
development these days, there seems to be a great opportunity for Karaf
running in a microkernel's
Camel->Spring Boot->JVM->Docker->Kubernetes->Linux->VM->hypervisor->server.
Camel K->JVM->Docker->Kubernetes->Linux->VM->hypervisor->server.
Server->hypervisor->microkernel->JVM->Karaf(!)
Recently I was reading a bit more about Camel K. That's Camel running
directly on Kubernettes sans container - no Spring Boot, Karaf/Felix, or
EAP. It's a move in the right direction, I think, but as I think about it
Karaf seems uniquely poised to really jump to the front of the line. James
Strachan recently commented that he was concerned about the future of the
JVM due to the enormous overhead of running it in a Docker world.
It isn't simply that Karaf/Felix can run on a JVM in the kernel space and
avoid all the other overhead. OSGi bundles and Karaf features essentially
allow one to create microservices as groups of bundles that can be deployed
to separate Karaf instances or to the same Karaf instances. That simply
isn't possible with Spring Boot, Camel K or other stacks.
If anyone is aware of a microkernel or rump kernel or exokernel running a
JVM/Karaf I'd really appreciate a pointer.
Brad
Re: Microkernel Karaf fit...
Posted by Ranx <ra...@enjekt.org>.
Serge,
I hadn't heard of Loom. I'll have to look into that a bit more. Like
François, I think the unikernels with Karaf would be dynamite. There are a
number of different unikernel and rump kernel projects out there right now.
The one that seems ready to move is OSv but it also appears to be a big
bigger than other unikernels. Not that that's necessarily a terrible thing
when you think about ending up with a Karaf/JVM/Unikernel/Hypervisor stack
and nothing else. Jettisoning the virtual machine, Linux and Docker means
you're way ahead of the game in boot up speed, size, and ultimately
performance.
Karaf is rather uniquely placed for working in that world as it has long had
the monitoring and deployment tools that obviated working in the actual
operating system. The unikernel only runs a single process so there isn't a
command line. That might hurt for other stacks like Spring Boot where you
don't have a way to get at information or debug the running application.
With Karaf, we really don't change anything about how we interact. Karaf has
always been something of a miniature OS with command line, remote log in,
provisioning, monitoring and so on.
--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Re: Microkernel Karaf fit...
Posted by Serge Huber <sh...@apache.org>.
I was just looking at what Project Loom
(https://jaxenter.com/project-loom-trend-watch-jvm-ecosystem-147448.html)
could mean for the JDK. Imagine having Java Fibers available in Karaf
you could build a highly scalable microservices platform that
maximizes hardware usage like crazy. If this is paired with an
optimized microkernel or lightweight OS it could be useful for use
cases ranging from embedded platforms all the way to complex workloads
for enterprise applications.
Regards,
Serge...
On Wed, Feb 13, 2019 at 8:50 AM Francois Papon
<fr...@openobject.fr> wrote:
>
> Hi Ranx,
>
> I realy like your point about the overhead of running a JVM in a Docker world ;)
>
> I think Karaf is a very good alternative to reduce this overhead and all the tooling provided by Karaf make it the perfect solution :)
>
> "Server->hypervisor->microkernel->JVM->Karaf(!)"
>
> Regards,
>
> François Papon
> fpapon@apache.org
>
> Le 06/02/2019 à 19:06, ranx@enjekt.org a écrit :
>
> Is anyone aware of a microkernel running a JVM/Karaf? I think Karaf could really take advantage of microkernels avoiding the need for Kubernetes, Docker and so on. Karaf is uniquely qualified for this by the fact that it has its own hooks to repositories, a console and monitoring with things like HawtIO. If the JVM/Karaf/Felix is running in the kernel itself and that is running on the hypervisor, there’s not a lot of overhead. That’s a stark contrast to the world of Kubernetes/Docker/VM/hypervisor.
>
>
>
> With microkernels and rump kernels getting a lot of attention and development these days, there seems to be a great opportunity for Karaf running in a microkernel’s
>
>
>
> Camel->Spring Boot->JVM->Docker->Kubernetes->Linux->VM->hypervisor->server.
>
> Camel K->JVM->Docker->Kubernetes->Linux->VM->hypervisor->server.
>
>
>
> Server->hypervisor->microkernel->JVM->Karaf(!)
>
>
>
> Recently I was reading a bit more about Camel K. That’s Camel running directly on Kubernettes sans container – no Spring Boot, Karaf/Felix, or EAP. It’s a move in the right direction, I think, but as I think about it Karaf seems uniquely poised to really jump to the front of the line. James Strachan recently commented that he was concerned about the future of the JVM due to the enormous overhead of running it in a Docker world.
>
>
>
> It isn’t simply that Karaf/Felix can run on a JVM in the kernel space and avoid all the other overhead. OSGi bundles and Karaf features essentially allow one to create microservices as groups of bundles that can be deployed to separate Karaf instances or to the same Karaf instances. That simply isn’t possible with Spring Boot, Camel K or other stacks.
>
>
>
> If anyone is aware of a microkernel or rump kernel or exokernel running a JVM/Karaf I’d really appreciate a pointer.
>
>
>
> Brad
>
>
>
>
Re: Microkernel Karaf fit...
Posted by Francois Papon <fr...@openobject.fr>.
Hi Ranx,
I realy like your point about the overhead of running a JVM in a Docker
world ;)
I think Karaf is a very good alternative to reduce this overhead and all
the tooling provided by Karaf make it the perfect solution :)
"Server->hypervisor->microkernel->JVM->Karaf(!)"
Regards,
François Papon
fpapon@apache.org
Le 06/02/2019 à 19:06, ranx@enjekt.org a écrit :
>
> Is anyone aware of a microkernel running a JVM/Karaf? I think Karaf
> could really take advantage of microkernels avoiding the need for
> Kubernetes, Docker and so on. Karaf is uniquely qualified for this by
> the fact that it has its own hooks to repositories, a console and
> monitoring with things like HawtIO. If the JVM/Karaf/Felix is running
> in the kernel itself and that is running on the hypervisor, there’s
> not a lot of overhead. That’s a stark contrast to the world of
> Kubernetes/Docker/VM/hypervisor.
>
>
>
> With microkernels and rump kernels getting a lot of attention and
> development these days, there seems to be a great opportunity for
> Karaf running in a microkernel’s
>
>
>
> Camel->Spring
> Boot->JVM->Docker->Kubernetes->Linux->VM->hypervisor->server.
>
> Camel K->JVM->Docker->Kubernetes->Linux->VM->hypervisor->server.
>
>
>
> Server->hypervisor->microkernel->JVM->Karaf(!)
>
>
>
> Recently I was reading a bit more about Camel K. That’s Camel running
> directly on Kubernettes sans container – no Spring Boot, Karaf/Felix,
> or EAP. It’s a move in the right direction, I think, but as I think
> about it Karaf seems uniquely poised to really jump to the front of
> the line. James Strachan recently commented that he was concerned
> about the future of the JVM due to the enormous overhead of running it
> in a Docker world.
>
>
>
> It isn’t simply that Karaf/Felix can run on a JVM in the kernel space
> and avoid all the other overhead. OSGi bundles and Karaf features
> essentially allow one to create microservices as groups of bundles
> that can be deployed to separate Karaf instances or to the same Karaf
> instances. That simply isn’t possible with Spring Boot, Camel K or
> other stacks.
>
>
>
> If anyone is aware of a microkernel or rump kernel or exokernel
> running a JVM/Karaf I’d really appreciate a pointer.
>
>
>
> Brad
>
>
>
>
>
Re: Microkernel Karaf fit...
Posted by Ranx <ra...@enjekt.org>.
JB,
Thanks for the feedback. I'll check out the blog and look at where you've
been going.
This is sort of on track for what I'm thinking about. One has to read
between the lines a bit because the example is using VirtualBox and Docker
but OSv is the microkernel being used. In reality, what I'd be after is the
microkernel - OSv or Mirage or whatever - directly running on the
hypervisor. Which is what OSv and other ukernels are really designed for.
What I wasn't sure about until I read this is one of the microkernels
running Java. Obviously once one has a JVM running then putting Karaf/Felix
on it is possible.
https://github.com/solo-io/unik/blob/master/docs/getting_started_java.md
From what I understand microkernels generally only permit a single process
to be booted. That can be a big challenge for many applications and
architectures. Karaf has long worked in a way that when it runs on a VM you
really don't have to log into the VM itself because all the tools are built
in. That makes it ideal for use in a microkernel because I don't believe
most microkernels provide command lines or log ins.
Think about a compare and contrast of Karaf running on OSv versus Spring
Boot. Karaf would run on the JVM and you could SSH in or go to Hawtio or
interact with it and its file system in many different manners. With Spring
Boot, once you've run it on the microkernel that's it. You can't log into it
or look at its file system or tweak configurations for the running instance.
Hypervisor->OSv ukernel->JVM->Karaf
Very fast start up. Low overhead. JVM running in kernel mode with privileged
acess and low latency. Karaf console/SSH, Maven deployment of artifacts,
etc.
While I understand Docker and Kubernetes, I've always thought of it as an
"elegant hack". Virtual machines are big and slow and resource hungry so how
do we get around that problem? Hack security barriers into the virtual
machine to partition it and then create management technologies. It seems to
solve the problem at the wrong level of abstraction and with Java that
creates a problem - the JVM isn't shared across Docker images.
So we end up with a way to share the resources of a virtual machine with
multiple Docker images but if they are JVM based, each JVM is going to grab
its own chunk of threads, memory and so on. Of course, if you're running
Spring Boot instance then each of those instances loads a large number of
identical classes and bytecode - even if it is just the JDK. That problem
persists even if you run Spring Boot on a microkernel.
Running Karaf on a microkernel is another matter. If I structure my features
along functional lines I can install them all in one Karaf container or if I
want to break them out later I can run them in separate containers in their
own microkernel instance. Obviously that's not unique to microkernels as you
can do that today in Docker images if you desired. But Karaf in Docker
doesn't make a lot of sense to me.
But let's get to the original problem - VMs are big and slow and take a lot
of resources to boot up an operating system that lets your run your
applications. Docker partitions the VM to make it a little more parsimonious
and faster. Kubernetes manages the resources for you. Now, let's get rid of
the VM, Docker and Kubernetes and run a microkernel directly on the
hypervisor which spins up a JVM and Karaf. Fast. Streamlined. Managing my
Java applications on that stack is no different than interacting with it
running on my desktop or on a server or on a VM or in a Docker image running
in a VM.
--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Re: Microkernel Karaf fit...
Posted by Ranx <ra...@enjekt.org>.
Serge,
Right. And all the issues you point out are compounded when you work in the
VM/Docker/JVM world. If overall memory consumption and garbage collection
are an issue in general, then carving up a VM to share a Linux distro among
multiple JVMs in their own Docker images for Spring Boot instances is pretty
lame.
With Karaf/OSGi whether I'm creating microservices or not can be largely a
deployment issue. If I have a set of features that install the bundles
associated with some functionality, I can install them in the same Karaf
container and they'll play nicely together or I can install them in separate
instances.
I don't have the quote in front of me right now but when James Strachan left
Red Hat about a year ago, he'd just completed working with them on creating
the OpenShift environment with Kubernetes/Docker. As he left, he expressed
his concern about the future of Java given these environmental concerns and
the size of the JVM and its performance. I'd had a nagging suspicion and
feeling about it and Spring Boot definitely fed into the sense of bloat and
overkill in that environment.
OSGi doesn't need fat jars for microservices like Spring Boot so doesn't
need to use the JVM as a container. If the JVM in Docker is a problem then
there really isn't much that Docker-JVM-Karaf brings to the table.
--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Re: Microkernel Karaf fit...
Posted by Serge Huber <sh...@apache.org>.
Yes I understand what you mean, but in some ways the JVM needs big
improvements. The first one that comes to mind is getting rid of the
garbage collector because it causes so many difficult problems in
terms of predictability. One of the reasons I love Swift so much is
that it manages to do away with such a construct.
Also the JVM is (still) full of stuff that doesn't make that much
sense in a microkernel environment, especially for running
micro-services. But at least with Jigsaw it becomes possible to make
it lighter.
The overall memory consumption of JVMs is also a concern.
Anyway, realistically, what can we achieve today? JVMs like the
OpenJDK one still need a lot of OS-level services to operate, and in
the short term it will be difficult to do without.
Regards,
Serge...
On Wed, Feb 13, 2019 at 2:56 PM Ranx <ra...@enjekt.org> wrote:
>
> Serge,
>
> Sure. Absolutely. What I meant is that part of the problem that Loom is
> solving is that of system thread context switching from user to kerenel mode
> and back.
>
> Running the JVM in kernel mode in an unikernel eliminates that context
> switch without the need for an extra construct.
>
> But that's really an aside for me, it's the ability to run the JVM directly
> on the hypervisor with only a slim OS between the two that appeals to me.
>
>
>
>
> --
> Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Re: Microkernel Karaf fit...
Posted by Ranx <ra...@enjekt.org>.
Serge,
Sure. Absolutely. What I meant is that part of the problem that Loom is
solving is that of system thread context switching from user to kerenel mode
and back.
Running the JVM in kernel mode in an unikernel eliminates that context
switch without the need for an extra construct.
But that's really an aside for me, it's the ability to run the JVM directly
on the hypervisor with only a slim OS between the two that appeals to me.
--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Re: Microkernel Karaf fit...
Posted by Serge Huber <sh...@apache.org>.
Actually Project Loom is all about providing user mode threads to the
JVM, so no more context switches and large memory footprint. Their
objective is to be able to support millions of lightweights threads on
a JVM that could normally only handle thousands of threads.
Regards,
Serge...
On Wed, Feb 13, 2019 at 2:41 PM Ranx <ra...@enjekt.org> wrote:
>
> One thing I forgot to mention when I brought this up is how fast threading in
> the JVM might be in a unikernel. Because the JVM/Karaf is running in a
> single process in the kernel, thread switching would not involve the
> laborious user mode to kernel mode switches that current thread switching
> does.
>
> So not only would this eliminate the overhead of virtual machine, operating
> system, and Docker, it would make the JVM's threading operate faster. So
> even if the unikernel isn't the most performant right out of the box, it
> gets some big boosts and over time one would expect it to get even better.
>
> That's why I thought I'd ask if anyone here has been working with it as I
> suspect in two or three years they will take the world by storm.
>
>
>
> --
> Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Re: Microkernel Karaf fit...
Posted by Ranx <ra...@enjekt.org>.
One thing I forgot to mention when I brought this up is how fast threading in
the JVM might be in a unikernel. Because the JVM/Karaf is running in a
single process in the kernel, thread switching would not involve the
laborious user mode to kernel mode switches that current thread switching
does.
So not only would this eliminate the overhead of virtual machine, operating
system, and Docker, it would make the JVM's threading operate faster. So
even if the unikernel isn't the most performant right out of the box, it
gets some big boosts and over time one would expect it to get even better.
That's why I thought I'd ask if anyone here has been working with it as I
suspect in two or three years they will take the world by storm.
--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Re: Microkernel Karaf fit...
Posted by Ranx <ra...@enjekt.org>.
This is a little dated but it means someone was taking a swing at this back
with Karaf 3.x. I did see where Onos has moved on to using Karaf 4.x
although I don't know if they are still using OSv. Being able to boot Karaf
into unikernel means start up would be fast, there's little attack surface,
and it won't have the overhead of VMs/Docker on top. Although I gather OSv
could run in that environment as well as could other unikernels. Has anyone
had any experience attempting to use Karaf with any unikernels good, bad or
indifferent. I'm certainly for pursuing this farther but if it's already out
there then there isn't any point in reinventing the wheel.
One complaint or problem with unikernels is there is only one allowed
process running. You can't log into it as user. That's also part of its
inherent security as well. That's why Karaf in that environment interests me
so much as I don't need a user login to the operating system if I can log
into Karaf itself.
Unfortunately my knowledge is somewhat limited here that's why I though I'd
ask if anyone had worked with it. Ideally I'd get a server with Xen running
on it so I could compile this and test it.
https://github.com/cloudius-systems/osv-apps/blob/master/onos/Capstanfile
base: cloudius/osv-openjdk
cmdline: >
/java.so
-Xms128M
-Xmx512M
-XX:+UnlockDiagnosticVMOptions
-XX:+UnsyncloadClass
-Dcom.sun.management.jmxremote
-Djava.endorsed.dirs=/usr/lib/jvm/java/jre/lib/endorsed:/usr/lib/jvm/java/lib/endorsed:/onos/apache-karaf-3.0.3/lib/endorsed
-Djava.ext.dirs=/usr/lib/jvm/java/jre/lib/ext:/usr/lib/jvm/java/lib/ext:/onos/apache-karaf-3.0.3/lib/ext
-Dkaraf.instances=/onos/apache-karaf-3.0.3/instances
-Dkaraf.home=/onos/apache-karaf-3.0.3
-Dkaraf.base=/onos/apache-karaf-3.0.3
-Dkaraf.data=/onos/apache-karaf-3.0.3/data
-Dkaraf.etc=/onos/apache-karaf-3.0.3/etc
-Djava.io.tmpdir=/onos/apache-karaf-3.0.3/data/tmp
-Djava.util.logging.config.file=/onos/apache-karaf-3.0.3/etc/java.util.logging.properties
-Dkaraf.startLocalConsole=true
-Dkaraf.startRemoteShell=true
-classpath
/onos/apache-karaf-3.0.3/lib/karaf-jaas-boot.jar:/onos/apache-karaf-3.0.3/lib/karaf.jar:/onos/apache-karaf-3.0.3/lib/karaf-jmx-boot.jar:/onos/apache-karaf-3.0.3/lib/karaf-org.osgi.core.jar
org.apache.karaf.main.Main
build: make
--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
Re: Microkernel Karaf fit...
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Brad
I know multiple Karaf use cases:
- embedded (I know in trucks, in the ESA spatial station ;))
- on prem
- on cloud
About hypervisor/cloud/container, I did a blog about
Karaf/Docker/Kubernetes:
http://blog.nanthrax.net/?p=849
So, what are you looking for ? A dedicated hypervisor for Karaf (a bit
like in Cellar and the kloud initiative I started).
By the way about the kloud initiative, the first action around this is to:
1. provide tooling for dev (easily create a custom karaf runtime
embedded business code, based on annotations for instance)
2. provide tooling for devops (easily create jar/docker image/tar.gz
turnkey to start powered by Karaf)
I would be more than happy to chat with you about the target ;)
Thanks for bringing the discussion anyway !
Regards
JB
On 06/02/2019 16:06, ranx@enjekt.org wrote:
> Is anyone aware of a microkernel running a JVM/Karaf? I think Karaf
> could really take advantage of microkernels avoiding the need for
> Kubernetes, Docker and so on. Karaf is uniquely qualified for this by
> the fact that it has its own hooks to repositories, a console and
> monitoring with things like HawtIO. If the JVM/Karaf/Felix is running in
> the kernel itself and that is running on the hypervisor, there’s not a
> lot of overhead. That’s a stark contrast to the world of
> Kubernetes/Docker/VM/hypervisor.
>
>
>
> With microkernels and rump kernels getting a lot of attention and
> development these days, there seems to be a great opportunity for Karaf
> running in a microkernel’s
>
>
>
> Camel->Spring Boot->JVM->Docker->Kubernetes->Linux->VM->hypervisor->server.
>
> Camel K->JVM->Docker->Kubernetes->Linux->VM->hypervisor->server.
>
>
>
> Server->hypervisor->microkernel->JVM->Karaf(!)
>
>
>
> Recently I was reading a bit more about Camel K. That’s Camel running
> directly on Kubernettes sans container – no Spring Boot, Karaf/Felix, or
> EAP. It’s a move in the right direction, I think, but as I think about
> it Karaf seems uniquely poised to really jump to the front of the line.
> James Strachan recently commented that he was concerned about the future
> of the JVM due to the enormous overhead of running it in a Docker world.
>
>
>
> It isn’t simply that Karaf/Felix can run on a JVM in the kernel space
> and avoid all the other overhead. OSGi bundles and Karaf features
> essentially allow one to create microservices as groups of bundles that
> can be deployed to separate Karaf instances or to the same Karaf
> instances. That simply isn’t possible with Spring Boot, Camel K or other
> stacks.
>
>
>
> If anyone is aware of a microkernel or rump kernel or exokernel running
> a JVM/Karaf I’d really appreciate a pointer.
>
>
>
> Brad
>
>
>
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com