You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@karaf.apache.org by CodeCola <pr...@rogers.com> on 2017/01/11 20:07:00 UTC
Levels of Containerization - focus on Docker and Karaf
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Vincent Zurczak <vi...@linagora.com>.
Le 12/01/2017 00:04, Brad Johnson a crit :
>
> And I suppose youll commonly run multiple Karaf instances in single
> Docker instance.<http://vzurczak.wordpress.com>
>
Although it is feasible, I think you do not need Karaf instances when
you use Karaf Docker containers.
The default instance is enough.
When to use which one?
If you have JRE on your host system, you could directly use Karaf instances.
If you want to provide Docker images as a quick install solution (e.g.
users do not have to install a JRE on their system), just deliver a
Docker image with a single Karaf instance. Then, running multiple
instances will be achieved by launching seveal containers. It is not a
technical choice per se, it is about how users will manipulate your product.
Regards,
--
Vincent Zurczak
Linagora: www.linagora.com <http://www.linagora.com/>
Twitter <https://twitter.com/VincentZurczak> Vincent Zurczak @ LinkedIn
<http://fr.linkedin.com/pub/vincent-zurczak/18/b35/6a7> My Skype ID
<callto://vincent.zurczak> My English blog <http://vzurczak.wordpress.com>
RE: Levels of Containerization - focus on Docker and Karaf
Posted by Brad Johnson <br...@redhat.com>.
As you say, the levels of concern between Docker and Karaf are different and
not incompatible. And I suppose youll commonly run multiple Karaf
instances in single Docker instance. Im down in the OSGi/Karaf level so
much I dont get up into the containerization of things like Docker,
Kubernettes, etc.
Ive heard about some of the flakiness you mention. I suspect that sort of
thing will get hammered out over time. Someone will one day realize why a
thread race or deadlock happens when the moon is full and the tide is low.
I think if I were to get more involved with Docker my chief concern would be
the direction of the framework and the contentiousness thats been there for
some time. Rocket left in 2014 and 2016 ended with a lot of different
companies and contributors talking about forking the codebase to bring more
stability. That seems like such a mundane and common development practice
that it surprises me. From the way Linux comes out with new releases and bug
fixes to the way ServiceMix and Red Hat turns ServiceMix into Fuse. When I
work in Fuse Im stepping into yesteryear. Thats the cost of getting
hardened, hammered and tested code.
Why isnt Docker following a similar path? If someone wants to use the
older, stable relase thats supported by some organization then go for it.
If you want to use the bleeding edge and contribute, thats cool too. It
seems such a common mechanism that Im sure I must be missing something.
The problem of being tangentially aware.
Brad
From: Vincent Zurczak [mailto:vincent.zurczak@linagora.com]
Sent: Wednesday, January 11, 2017 3:55 PM
To: user@karaf.apache.org
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Hi,
Le 11/01/2017 à 22:18, Brad Johnson a écrit :
So I don't know enough about Docker internals/configuration requirements but
I have a question. Is installing a JRE in each Docker container the common
practice as opposed to some mechanism for sharing it? The JRE alone is
180MB.
Just get or build a Docker image with the JRE inside.
This way, you will only use 180MB on your disk (plus the OS size).
You then instantiate this image to create one or several containers. One
definition, several instances, just like classes: one definition in jars,
but several instances in memory.
Le 11/01/2017 à 21:07, CodeCola a écrit :
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
I work on a project which is relies on Karaf and that manages cloud
infrastructures (so, VMs) but also Docker containers.
It all depends on what you want. If all your software components are Java
ones, using an OSGi container is enough (my main interest in OSGi is about
class loading isolation, even though there are some other pretty good
stuff). If you have heterogenous components (different languages, different
versions...), Docker containers can help a lot. But they cannot solve all
the problems.
First, the host system has limited resources. When you reach a given number
of containers or resources consumption, new ones will starve. So, you need
new VMs. Second, Docker containers are not totally isolated. They use the
same kernel than the host system, as well as some properties (global
namespace). And I have already seen a Docker container that could not stop
(I had to restart the host system... :(). This is why you also need VM
management. This is about architecture and addressing fail-over,
replication, load balancing and so on. Container solutions like Docker Swarm
or Kubernetes generally rely on a cluster of VMs. And thanks to placement
constraints, you create, dispatch, move Docker containers on this or that
node.
Then, I regularly run Docker containers that embed a Karaf server, and the
container itself runs in a VM that was dynamically created in a cloud
infrastructure. They work together, each layer having its own purpose. This
is a matter of design. If you deal with (Java) application development, you
can benefit from Java containers (OSGi, JEE...). If you deal with
architecturing a solution with several applications, you may consider using
VMs (old-school) or Docker containers. When you deal with containers, you
will eventually manage several VMs. And you can also mix containers and VMs.
Indeed, some applications just do not fit well within containers. Those that
manage data, such as databases, can be part of them.
So, they SHOULD not be used together.
They CAN be used together. It is then up to you to determine what you need
and why you make this or that choice. Should I say "as usual"? ;)
Regards,
--
Vincent Zurczak
Linagora: www.linagora.com <http://www.linagora.com/>
<https://twitter.com/VincentZurczak>
<http://fr.linkedin.com/pub/vincent-zurczak/18/b35/6a7>
<callto://vincent.zurczak> <http://vzurczak.wordpress.com>
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Vincent Zurczak <vi...@linagora.com>.
Hi,
Le 11/01/2017 22:18, Brad Johnson a crit :
> So I don't know enough about Docker internals/configuration requirements but
> I have a question. Is installing a JRE in each Docker container the common
> practice as opposed to some mechanism for sharing it? The JRE alone is
> 180MB.
Just get or build a Docker image with the JRE inside.
This way, you will only use 180MB on your disk (plus the OS size).
You then instantiate this image to create one or several containers. One
definition, several instances, just like classes: one definition in
jars, but several instances in memory.
Le 11/01/2017 21:07, CodeCola a crit :
> Not a question but a request for comments. With a focus on Java.
>
> Container technology has traditionally been messy with dependencies and no
> easy failsafe way until Docker came along to really pack ALL dependencies
> (including the JVM) together in one ready-to-ship image that was faster,
> more comfortable, and easier to understand than other container and code
> shipping methods out there. The spectrum from (Classical) Java EE Containers
> (e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
> (Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
> Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
> with different goals (abstraction, isolation and delivery).
>
> What are the choices, how should they play together, should they be used in
> conjunction with each other as they offer different kinds of
> Containerization?
I work on a project which is relies on Karaf and that manages cloud
infrastructures (so, VMs) but also Docker containers.
It all depends on what you want. If all your software components are
Java ones, using an OSGi container is enough (my main interest in OSGi
is about class loading isolation, even though there are some other
pretty good stuff). If you have heterogenous components (different
languages, different versions...), Docker containers can help a lot. But
they cannot solve all the problems.
First, the host system has limited resources. When you reach a given
number of containers or resources consumption, new ones will starve. So,
you need new VMs. Second, Docker containers are not totally isolated.
They use the same kernel than the host system, as well as some
properties (global namespace). And I have already seen a Docker
container that could not stop (I had to restart the host system... :().
This is why you also need VM management. This is about architecture and
addressing fail-over, replication, load balancing and so on. Container
solutions like Docker Swarm or Kubernetes generally rely on a cluster of
VMs. And thanks to placement constraints, you create, dispatch, move
Docker containers on this or that node.
Then, I regularly run Docker containers that embed a Karaf server, and
the container itself runs in a VM that was dynamically created in a
cloud infrastructure. They work together, each layer having its own
purpose. This is a matter of design. If you deal with (Java) application
development, you can benefit from Java containers (OSGi, JEE...). If you
deal with architecturing a solution with several applications, you may
consider using VMs (old-school) or Docker containers. When you deal with
containers, you will eventually manage several VMs. And you can also mix
containers and VMs. Indeed, some applications just do not fit well
within containers. Those that manage data, such as databases, can be
part of them.
So, they SHOULD not be used together.
They CAN be used together. It is then up to you to determine what you
need and why you make this or that choice. Should I say "as usual"? ;)
Regards,
--
Vincent Zurczak
Linagora: www.linagora.com <http://www.linagora.com/>
Twitter <https://twitter.com/VincentZurczak> Vincent Zurczak @ LinkedIn
<http://fr.linkedin.com/pub/vincent-zurczak/18/b35/6a7> My Skype ID
<callto://vincent.zurczak> My English blog <http://vzurczak.wordpress.com>
RE: Levels of Containerization - focus on Docker and Karaf
Posted by Brad Johnson <br...@redhat.com>.
So I don't know enough about Docker internals/configuration requirements but
I have a question. Is installing a JRE in each Docker container the common
practice as opposed to some mechanism for sharing it? The JRE alone is
180MB.
This might be a quibble but I'm not sure I'd characterize Karaf as an
application server. Depends on your audience I think.
Brad
-----Original Message-----
From: CodeCola [mailto:prasenjit@rogers.com]
Sent: Wednesday, January 11, 2017 2:07 PM
To: user@karaf.apache.org
Subject: Levels of Containerization - focus on Docker and Karaf
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.
png>
--
View this message in context:
http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker
-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
RE: Levels of Containerization - focus on Docker and Karaf
Posted by Brad Johnson <br...@redhat.com>.
Agreed about the testing. One problem with much of the Camel documentation is it shows the use of Processors/Exchanges instead of just POJOs. For the most part that sort of unit testing is simple if one isn’t using Processor/Exchanges. The CDI makes it even easier yet. But PAX and Camel Blueprint Test Support can be slow and fiddly.
Since Jetty is a simple web server it is commonly included in Karaf/application stacks. Its concerns are orthogonal to those of OSGi/Felix (or Equinox)/Karaf. OSGi based application servers like Glassfish use it for the web server/webservices. I think Jetty is about 10MB soaking wet.
I’m not sure what dependency issues you ran into. One of the great benefits to having the classloader isolation in OSGi is that you can run different versions of the same library without stepping on one another.
Obviously Docker/Kubernettes, etc. are not at odds with Karaf and exist at different levels. Spinning up multiple Docker containers for single services seems (A) heavy on processes, (B) heavu on configuration of URI/ports, and (c) potentially requires separate security set up for each.. Since Camel routing for received webservice calls is so simple it makes it a relatively easy proposition. So using the Karaf instance for multiple services isn’t just about the classloader isolation.
But all that depends on the use case and it’s hard to make any sweeping generalizations. At some point that becomes as much art as science.
Testing though is the biggest issue. With the static profiles it should be easy to run a Karaf instance, exercise it as a black box against some test stubs and verify basic operations.
I wish they’d redo that Camel documentation to move the information about Procssor/Exhange to the equivalent of an Appendix.
From: Elliot Metsger [mailto:emetsger@gmail.com]
Sent: Thursday, January 12, 2017 11:32 AM
To: user@karaf.apache.org
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Hi,
I'm involved in a project [1] that is based on Karaf and Camel. We recently produced a milestone that involved a working demonstration of the code base using Docker [2]. This was one of our first forays into this technology stack, so we not only came away with some impressions of the stack but also noted some of the same questions you ask with regard to the various containerization choices.
Fundamentally, we noted that that properly testing our code to work in an OSGI container is challenging and time consuming (we used Pax Exam [3]); we would spend more time fiddling around with the test fixtures than actually writing tests and fixing any bugs they illuminated.
Secondarily, we struggled at times to find and resolve any conflicts in our Maven dependency tree related to using an OSGI bundle instead of a dependency's non-OSGI counterpart. We also struggled with some of the nuances of running Karaf and Camel, including discovering bugs in each of these frameworks or their components.
All of that being said, the result is something we are very pleased with. In order to produce our milestone demo, we containerized all of our services, which included two different instances of Karaf, an RDF triple store (Java web application server), the Fedora 4 linked data repository [4] (Java web application server), and other services (also a Java web application server). The exercise of producing this Dockerized milestone was very illuminating; we worked out some additional architectural issues and other bugs that our integration tests by themselves didn't do (and later added additional ITs to cover these discovered issues).
Personally, my observation is that there seems to be a lot of overlap in containerization technologies, but it seems the levels of abstraction differ with different tradeoffs. For example, if you desire classloader isolation, you could use a single instance of Karaf running multiple services, or you could simply spin up a Docker container for each service, running each service in a traditional web app container. You'd have to decide if embracing Karaf and OSGI way of doing things is better than working with "heavy" Docker containers and traditional, stable, well-understood Java application servers like Jetty.
Good discussion, I've enjoyed reading this thread!
[1] https://github.com/fcrepo4-labs/fcrepo-api-x
[2] https://github.com/fcrepo4-labs/fcrepo-api-x-demo
[3] https://ops4j1.jira.com/wiki/x/PwA8Aw
[4] https://github.com/fcrepo4/fcrepo4
On Wed, Jan 11, 2017 at 3:07 PM, CodeCola <prasenjit@rogers.com <ma...@rogers.com> > wrote:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Elliot Metsger <em...@gmail.com>.
Hi,
I'm involved in a project [1] that is based on Karaf and Camel. We
recently produced a milestone that involved a working demonstration of the
code base using Docker [2]. This was one of our first forays into this
technology stack, so we not only came away with some impressions of the
stack but also noted some of the same questions you ask with regard to the
various containerization choices.
Fundamentally, we noted that that properly testing our code to work in an
OSGI container is challenging and time consuming (we used Pax Exam [3]); we
would spend more time fiddling around with the test fixtures than actually
writing tests and fixing any bugs they illuminated.
Secondarily, we struggled at times to find and resolve any conflicts in our
Maven dependency tree related to using an OSGI bundle instead of a
dependency's non-OSGI counterpart. We also struggled with some of the
nuances of running Karaf and Camel, including discovering bugs in each of
these frameworks or their components.
All of that being said, the result is something we are very pleased with.
In order to produce our milestone demo, we containerized all of our
services, which included two different instances of Karaf, an RDF triple
store (Java web application server), the Fedora 4 linked data repository
[4] (Java web application server), and other services (also a Java web
application server). The exercise of producing this Dockerized milestone
was very illuminating; we worked out some additional architectural issues
and other bugs that our integration tests by themselves didn't do (and
later added additional ITs to cover these discovered issues).
Personally, my observation is that there seems to be a lot of overlap in
containerization technologies, but it seems the levels of abstraction
differ with different tradeoffs. For example, if you desire classloader
isolation, you could use a single instance of Karaf running multiple
services, or you could simply spin up a Docker container for each service,
running each service in a traditional web app container. You'd have to
decide if embracing Karaf and OSGI way of doing things is better than
working with "heavy" Docker containers and traditional, stable,
well-understood Java application servers like Jetty.
Good discussion, I've enjoyed reading this thread!
[1] https://github.com/fcrepo4-labs/fcrepo-api-x
[2] https://github.com/fcrepo4-labs/fcrepo-api-x-demo
[3] https://ops4j1.jira.com/wiki/x/PwA8Aw
[4] https://github.com/fcrepo4/fcrepo4
On Wed, Jan 11, 2017 at 3:07 PM, CodeCola <pr...@rogers.com> wrote:
> Not a question but a request for comments. With a focus on Java.
>
> Container technology has traditionally been messy with dependencies and no
> easy failsafe way until Docker came along to really pack ALL dependencies
> (including the JVM) together in one ready-to-ship image that was faster,
> more comfortable, and easier to understand than other container and code
> shipping methods out there. The spectrum from (Classical) Java EE
> Containers
> (e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
> (Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
> Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
> with different goals (abstraction, isolation and delivery).
>
> What are the choices, how should they play together, should they be used in
> conjunction with each other as they offer different kinds of
> Containerization?
>
> <http://karaf.922171.n3.nabble.com/file/n4049162/
> Levels_of_Containerization.png>
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-
> Containerization-focus-on-Docker-and-Karaf-tp4049162.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Nick Baker <nb...@pentaho.com>.
That would be good JB. People will be looking to deploy Karaf in container solutions ever more increasingly in the future.
-Nick Baker
________________________________
From: Jean-Baptiste Onofré <jb...@nanthrax.net>
Sent: Tuesday, January 17, 2017 8:26:16 AM
To: user@karaf.apache.org
Subject: Re: Levels of Containerization - focus on Docker and Karaf
It makes sense.
The only "lacking" part is about global service registry, API gateway
and governance.
I talked a bit with Guillaume some month ago and with Krystof during
last ApacheCon in Sevilla: I think it would be great to have something
around that as part of ServiceMix.
I prepared a document describing this that I can share with people
interested.
Regards
JB
On 01/17/2017 12:19 AM, souciance wrote:
> Someone may have mentioned this before but they way we do build our services
> is building karaf inside a docker container.
>
> We also deploy each Karaf instance which contains multiple Camel projects
> inside separate containers. So we have a one to one mapping between karaf
> instance and container. This is mainly because we don't want a container
> going down affecting more than one instance.
>
> Off course if our instances increase we may later have to consolidate but so
> far it was has worked. We also create our Karaf instances using custom
> distributions and then add them using dockerfiles. The main problem has been
> to get the distribution working correctly.
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162p4049260.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Jason Pratt <jp...@gmail.com>.
i'd like to see it please
jason
On Tue, Jan 17, 2017 at 5:26 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> It makes sense.
>
> The only "lacking" part is about global service registry, API gateway and
> governance.
>
> I talked a bit with Guillaume some month ago and with Krystof during last
> ApacheCon in Sevilla: I think it would be great to have something around
> that as part of ServiceMix.
>
> I prepared a document describing this that I can share with people
> interested.
>
> Regards
> JB
>
> On 01/17/2017 12:19 AM, souciance wrote:
>
>> Someone may have mentioned this before but they way we do build our
>> services
>> is building karaf inside a docker container.
>>
>> We also deploy each Karaf instance which contains multiple Camel projects
>> inside separate containers. So we have a one to one mapping between karaf
>> instance and container. This is mainly because we don't want a container
>> going down affecting more than one instance.
>>
>> Off course if our instances increase we may later have to consolidate but
>> so
>> far it was has worked. We also create our Karaf instances using custom
>> distributions and then add them using dockerfiles. The main problem has
>> been
>> to get the distribution working correctly.
>>
>>
>>
>> --
>> View this message in context: http://karaf.922171.n3.nabble.
>> com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4
>> 049162p4049260.html
>> Sent from the Karaf - User mailing list archive at Nabble.com.
>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
It makes sense.
The only "lacking" part is about global service registry, API gateway
and governance.
I talked a bit with Guillaume some month ago and with Krystof during
last ApacheCon in Sevilla: I think it would be great to have something
around that as part of ServiceMix.
I prepared a document describing this that I can share with people
interested.
Regards
JB
On 01/17/2017 12:19 AM, souciance wrote:
> Someone may have mentioned this before but they way we do build our services
> is building karaf inside a docker container.
>
> We also deploy each Karaf instance which contains multiple Camel projects
> inside separate containers. So we have a one to one mapping between karaf
> instance and container. This is mainly because we don't want a container
> going down affecting more than one instance.
>
> Off course if our instances increase we may later have to consolidate but so
> far it was has worked. We also create our Karaf instances using custom
> distributions and then add them using dockerfiles. The main problem has been
> to get the distribution working correctly.
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162p4049260.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
--
Jean-Baptiste Onofr
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Levels of Containerization - focus on Docker and Karaf
Posted by souciance <so...@gmail.com>.
Someone may have mentioned this before but they way we do build our services
is building karaf inside a docker container.
We also deploy each Karaf instance which contains multiple Camel projects
inside separate containers. So we have a one to one mapping between karaf
instance and container. This is mainly because we don't want a container
going down affecting more than one instance.
Off course if our instances increase we may later have to consolidate but so
far it was has worked. We also create our Karaf instances using custom
distributions and then add them using dockerfiles. The main problem has been
to get the distribution working correctly.
--
View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162p4049260.html
Sent from the Karaf - User mailing list archive at Nabble.com.
RE: Levels of Containerization - focus on Docker and Karaf
Posted by Brad Johnson <br...@redhat.com>.
That's pretty much it except it doesn't use the Kar file. It pulls a distribution of Karaf that has been slimmed down, installs all the bundles, configuration files, and then zips up the entire Karaf instance with all its dependencies. Unzip it, go to the bin directory, and run the Karaf container. So it may be easy to put that in a Docker container as it is generic from the perspective of Docker and doesn't have any specific requirements - just general ones like a JVM, a mechanism to unzip the Karaf container with dependencies, and a way to run it automatically.
All the required dependencies to run your application are in the Karaf/system folder. Because the dynamic install mechanisms have been removed the container is quite small.
-----Original Message-----
From: CodeCola [mailto:prasenjit@rogers.com]
Sent: Thursday, January 12, 2017 11:01 AM
To: user@karaf.apache.org
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Thanks for a great discussion especially the insights from real world deployments. This is what I understand:
*"Static" Karaf Profiles*
Docker images should be based on a "static distribution" of Karaf generated from Profiles. Starting from Profiles, one can generate a docker image and push it to a registry. The image would contain all the needed bundles and would not have to be further "provisioned". This resembles the "static container" idea which is the basis for Docker application packaging.
In principle, you want to have the following steps:
• Create a Karaf feature from your application
• Assemble it to a kar file (a custom Karaf distribution)
• Deploy the new Karaf package to Docker
fabric8 has a maven-plugin for doing this which is well documented.
https://maven.fabric8.io/
Or
Can be done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
Karaf section about custom assembly builds https://karaf.apache.org/manual/latest/#_custom_distributions
*Naming and Packaging*
Docker will likely influence how you package and provision your Karaf distributions. For instance, you will eventually have to provide a Docker image with a pre-configured Karaf, KAR files in deployment folder, etc. so that your Kubernetes container may bootstrap everything on boot.
Cellar Kubernetes discovery service is a great complement to the Karaf docker.io feature (allowing you to easily create and manage docker.io images in and for Karaf). To install the Kubernetes discovery service, simply install cellar-kubernetes feature.
karaf@root()> feature:install cellar-kubernetes
Question -* What is Karaf Boot?*
Could only find: https://github.com/jbonofre/karaf-boot
There isn't enough documentation!
--
View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162p4049188.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Levels of Containerization - focus on Docker and Karaf
Posted by CodeCola <pr...@rogers.com>.
Thanks for a great discussion especially the insights from real world
deployments. This is what I understand:
*"Static" Karaf Profiles*
Docker images should be based on a "static distribution" of Karaf generated
from Profiles. Starting from Profiles, one can generate a docker image and
push it to a registry. The image would contain all the needed bundles and
would not have to be further "provisioned". This resembles the "static
container" idea which is the basis for Docker application packaging.
In principle, you want to have the following steps:
• Create a Karaf feature from your application
• Assemble it to a kar file (a custom Karaf distribution)
• Deploy the new Karaf package to Docker
fabric8 has a maven-plugin for doing this which is well documented.
https://maven.fabric8.io/
Or
Can be done by using the karaf maven plugin and configuring startupFeatures
and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
Karaf section about custom assembly builds
https://karaf.apache.org/manual/latest/#_custom_distributions
*Naming and Packaging*
Docker will likely influence how you package and provision your Karaf
distributions. For instance, you will eventually have to provide a Docker
image with a pre-configured Karaf, KAR files in deployment folder, etc. so
that your Kubernetes container may bootstrap everything on boot.
Cellar Kubernetes discovery service is a great complement to the Karaf
docker.io feature (allowing you to easily create and manage docker.io images
in and for Karaf). To install the Kubernetes discovery service, simply
install cellar-kubernetes feature.
karaf@root()> feature:install cellar-kubernetes
Question -* What is Karaf Boot?*
Could only find: https://github.com/jbonofre/karaf-boot
There isn't enough documentation!
--
View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162p4049188.html
Sent from the Karaf - User mailing list archive at Nabble.com.
RE: Levels of Containerization - focus on Docker and Karaf
Posted by Brad Johnson <br...@redhat.com>.
Agreed. I think it’s all there but not put together in a holistic fashion. Spring Boot has support for all the different ways of working with bundles – blueprint, DS, etc. To use Nick’s term, an “opinionated” and standard way of bring all those elements together would help with adoption. While I’m in favor of CDI and the Camel Java DSL that’s obviously debatable. But one standard should be put forward. Not for Karaf itself, but for Spring Boot.
Otherwise, unlike Spring Boot which works in well defined ways, we end up with a Balkanization of technologies in Karaf Boot for a mindshare that is arguably already smaller than Spring Boot is.
From: Guillaume Nodet [mailto:gnodet@apache.org]
Sent: Thursday, January 12, 2017 10:08 AM
To: user <us...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
I fully agree.
However, I'm not sure if there are any pieces actually missing but some documentation and advertisement ...
We could eventually add a bit of tooling, but I'm not even sure that's really needed.
Guillaume
2017-01-12 16:46 GMT+01:00 Brad Johnson <bradjohn@redhat.com <ma...@redhat.com> >:
Guillaume,
I’d mentioned that in an early post as my preferred way to do microservices and perhaps a good way of doing a Karaf Boot. I’ve worked with the Karaf 4 profiles and they are great. Also used your CDI OSGi service. If we could use the Karaf 4 profiles with the CDI implementation with OSGi services and the Camel Java DSL as a standard stack, it would permit focused development and standardized bundle configurations.
When I created a zip with Karaf, CXF and Camel the footprint was 30MB.
While having Karaf Boot support DS, Blueprint, CDI, etc. I’m not sure that’s the healthiest move to encourage adoption. We need less fragmentation in the OSGi world and not more. Obviously even if Karaf Boot adopts one as the recommended standard it doesn’t mean that the others can’t be used. When reading through Camel documentation on-line, for example, the confusion such fragmentation brings is obvious. One becomes adept at converting Java DSL to blueprint or from blueprint to the Java DSL in one’s mind.
The static profiles work great and will let us create a number of standardized appliances for a wide variety of topology concerns and not just for microservices. A “switchboard” appliance, for example, might be used for orchestrating microservices and managing the APIs. A “gateway” appliance might have standard JAAS, web service configuration and a routing mechanism for calling microservices. An “AB” appliance could be used for 80/20 testing. And so on. Take the idea of enterprise integration patterns and bring it up to enterprise integration patterns using Karaf appliances.
Many appliances might be “sealed”. An appliance for AB testing, for example, would have configuration for two addresses in the configuration file and a percentage of traffic going to each. Non need to actually program or re-program the internals any more than we’d usually re-program a Camel component. But the source would be there if one wanted to create a new component or modify how an existing one functioned.
I’d vote for using CDI along with the Camel Java DSL for a couple of reasons. The first would be the standardization and portability of both code and skills. Using CDI would mean that any Glassfish, JBoss, etc. developer would feel comfortable using the code. Using Java Camel DSL would be for the same reason. It would also give a programmer a sense that if they give Karaf Boot with the static profiles a shot, they aren’t locked in but can easily move to a different stack if necessary. In a sense this is the same reason that Google chose Java language to run on the DVM. It tapped into a large existing skillbase instead of trying to get the world to adopt and learn a new language. CDI with OSGi extensions also allows developers to use one paradigm for everything from lashing up internal dependency injections to working with OSGi services. I believe when you put that CDI extension out there you used blueprint style proxies under the cover. As a developer using the CDI OSGi extension it was transparent to me. If you later decided to rework that as a DS service, it would remain transparent and very much in the whole spirit of OSGi and its mechanisms for allowing refactoring and even rewriting without breaking the world. It also makes unit testing a snap. Any of us who have wrestled with Camel Blueprint Test Support can appreciate that.
This would also permit for standardization of documentation and of Karaf Boot appliance project structures and Maven plug-in use. A bit of convention over configuration. Projects would have a standard configuration.cfg file that gets deployed via features to the pid.cfg. A standard features file in the filtered folder. Those already exist, of course, but it isn’t as standardized as it could be.
Personally I think this sort of goal with CDI, Karaf 4 and its profiles, and Camel Java DSL should be accelerated since Spring Boot is already out there. Waiting for another couple of years to release this as a standard might be too late.
The pieces are already there so it isn’t like we’d have to start from scratch. This would also play well with larger container concerns like Docker and Kubernettes.
Brad
From: Guillaume Nodet [mailto:gnodet@apache.org <ma...@apache.org> ]
Sent: Thursday, January 12, 2017 4:55 AMJ
To: user <user@karaf.apache.org <ma...@karaf.apache.org> >
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <prasenjit@rogers.com <ma...@rogers.com> >:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: gnodet@redhat.com <ma...@redhat.com>
Web: http://fusesource.com <http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: gnodet@redhat.com <ma...@redhat.com>
Web: http://fusesource.com <http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Guillaume Nodet <gn...@apache.org>.
I fully agree.
However, I'm not sure if there are any pieces actually missing but some
documentation and advertisement ...
We could eventually add a bit of tooling, but I'm not even sure that's
really needed.
Guillaume
2017-01-12 16:46 GMT+01:00 Brad Johnson <br...@redhat.com>:
> Guillaume,
>
>
>
> I’d mentioned that in an early post as my preferred way to do
> microservices and perhaps a good way of doing a Karaf Boot. I’ve worked
> with the Karaf 4 profiles and they are great. Also used your CDI OSGi
> service. If we could use the Karaf 4 profiles with the CDI implementation
> with OSGi services and the Camel Java DSL as a standard stack, it would
> permit focused development and standardized bundle configurations.
>
>
>
> When I created a zip with Karaf, CXF and Camel the footprint was 30MB.
>
>
>
> While having Karaf Boot support DS, Blueprint, CDI, etc. I’m not sure
> that’s the healthiest move to encourage adoption. We need less
> fragmentation in the OSGi world and not more. Obviously even if Karaf Boot
> adopts one as the recommended standard it doesn’t mean that the others
> can’t be used. When reading through Camel documentation on-line, for
> example, the confusion such fragmentation brings is obvious. One becomes
> adept at converting Java DSL to blueprint or from blueprint to the Java DSL
> in one’s mind.
>
>
>
> The static profiles work great and will let us create a number of
> standardized appliances for a wide variety of topology concerns and not
> just for microservices. A “switchboard” appliance, for example, might be
> used for orchestrating microservices and managing the APIs. A “gateway”
> appliance might have standard JAAS, web service configuration and a routing
> mechanism for calling microservices. An “AB” appliance could be used for
> 80/20 testing. And so on. Take the idea of enterprise integration
> patterns and bring it up to enterprise integration patterns using Karaf
> appliances.
>
>
>
> Many appliances might be “sealed”. An appliance for AB testing, for
> example, would have configuration for two addresses in the configuration
> file and a percentage of traffic going to each. Non need to actually
> program or re-program the internals any more than we’d usually re-program a
> Camel component. But the source would be there if one wanted to create a
> new component or modify how an existing one functioned.
>
>
>
> I’d vote for using CDI along with the Camel Java DSL for a couple of
> reasons. The first would be the standardization and portability of both
> code and skills. Using CDI would mean that any Glassfish, JBoss, etc.
> developer would feel comfortable using the code. Using Java Camel DSL
> would be for the same reason. It would also give a programmer a sense that
> if they give Karaf Boot with the static profiles a shot, they aren’t locked
> in but can easily move to a different stack if necessary. In a sense this
> is the same reason that Google chose Java language to run on the DVM. It
> tapped into a large existing skillbase instead of trying to get the world
> to adopt and learn a new language. CDI with OSGi extensions also allows
> developers to use one paradigm for everything from lashing up internal
> dependency injections to working with OSGi services. I believe when you put
> that CDI extension out there you used blueprint style proxies under the
> cover. As a developer using the CDI OSGi extension it was transparent to
> me. If you later decided to rework that as a DS service, it would remain
> transparent and very much in the whole spirit of OSGi and its mechanisms
> for allowing refactoring and even rewriting without breaking the world. It
> also makes unit testing a snap. Any of us who have wrestled with Camel
> Blueprint Test Support can appreciate that.
>
>
>
> This would also permit for standardization of documentation and of Karaf
> Boot appliance project structures and Maven plug-in use. A bit of
> convention over configuration. Projects would have a standard
> configuration.cfg file that gets deployed via features to the pid.cfg. A
> standard features file in the filtered folder. Those already exist, of
> course, but it isn’t as standardized as it could be.
>
>
>
> Personally I think this sort of goal with CDI, Karaf 4 and its profiles,
> and Camel Java DSL should be accelerated since Spring Boot is already out
> there. Waiting for another couple of years to release this as a standard
> might be too late.
>
>
>
> The pieces are already there so it isn’t like we’d have to start from
> scratch. This would also play well with larger container concerns like
> Docker and Kubernettes.
>
>
>
> Brad
>
>
>
> *From:* Guillaume Nodet [mailto:gnodet@apache.org]
> *Sent:* Thursday, January 12, 2017 4:55 AMJ
> *To:* user <us...@karaf.apache.org>
> *Subject:* Re: Levels of Containerization - focus on Docker and Karaf
>
>
>
> Fwiw, starting with Karaf 4.x, you can build custom distributions which
> are mostly static, and that more closely map to micro-services / docker
> images. The "static" images are called this way because you they kinda
> remove all the OSGi dynamism, i.e. no feature service, no deploy folder,
> read-only config admin, all bundles being installed at startup time from
> etc/startup.properties.
>
> This can be easily done by using the karaf maven plugin and configuring
> startupFeatures and referencing the static kar, as shown in:
>
> https://github.com/apache/karaf/blob/master/demos/
> profiles/static/pom.xml
>
>
>
>
>
> 2017-01-11 21:07 GMT+01:00 CodeCola <pr...@rogers.com>:
>
> Not a question but a request for comments. With a focus on Java.
>
> Container technology has traditionally been messy with dependencies and no
> easy failsafe way until Docker came along to really pack ALL dependencies
> (including the JVM) together in one ready-to-ship image that was faster,
> more comfortable, and easier to understand than other container and code
> shipping methods out there. The spectrum from (Classical) Java EE
> Containers
> (e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
> (Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
> Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
> with different goals (abstraction, isolation and delivery).
>
> What are the choices, how should they play together, should they be used in
> conjunction with each other as they offer different kinds of
> Containerization?
>
> <http://karaf.922171.n3.nabble.com/file/n4049162/
> Levels_of_Containerization.png>
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-
> Containerization-focus-on-Docker-and-Karaf-tp4049162.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
>
>
>
>
> --
>
> ------------------------
> Guillaume Nodet
> ------------------------
>
> Red Hat, Open Source Integration
>
>
>
> Email: gnodet@redhat.com
> Web: http://fusesource.com
> Blog: http://gnodet.blogspot.com/
>
>
>
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: gnodet@redhat.com
Web: http://fusesource.com
Blog: http://gnodet.blogspot.com/
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Achim Nierbeck <bc...@googlemail.com>.
So in the end it looks like Karaf-Boot with profiles, as I tried some long
time ago :)
would be the best match to what I've seen so far in this thread.
The idea I had at the time would be to actually merge those static profiles
with a maven plugin generating the static Karaf assembly via the maven
plugin.
All of the idea-scratch-start stuff can be found at JBs github project.
Haven't had the time to get going on it any further, just been an idea in a
seperate branch at the time:
https://github.com/jbonofre/karaf-boot/tree/profiles/karaf-boot-starters/karaf-boot-starter-web/src/main/resources/META-INF/profiles
regards, Achim
2017-01-12 16:18 GMT+00:00 Nick Baker <nb...@pentaho.com>:
> I do agree that an "opinionated" or "prescriptive" stack would help. It
> shouldn't prohibit the usage of any Karaf feature of course.
>
>
> New users gravitate to full-stack solutions. Agnostic platforms with lots
> of options and no predefined stack, while obviously having many merits and
> longer legs (don't be Wicket), just haven't been winning out in adoption.
> This applies across the spectrum of computing.
>
> ------------------------------
> *From:* Brad Johnson <br...@redhat.com>
> *Sent:* Thursday, January 12, 2017 10:46:55 AM
> *To:* user@karaf.apache.org
> *Subject:* RE: Levels of Containerization - focus on Docker and Karaf
>
>
> Guillaume,
>
>
>
> I’d mentioned that in an early post as my preferred way to do
> microservices and perhaps a good way of doing a Karaf Boot. I’ve worked
> with the Karaf 4 profiles and they are great. Also used your CDI OSGi
> service. If we could use the Karaf 4 profiles with the CDI implementation
> with OSGi services and the Camel Java DSL as a standard stack, it would
> permit focused development and standardized bundle configurations.
>
>
>
> When I created a zip with Karaf, CXF and Camel the footprint was 30MB.
>
>
>
> While having Karaf Boot support DS, Blueprint, CDI, etc. I’m not sure
> that’s the healthiest move to encourage adoption. We need less
> fragmentation in the OSGi world and not more. Obviously even if Karaf Boot
> adopts one as the recommended standard it doesn’t mean that the others
> can’t be used. When reading through Camel documentation on-line, for
> example, the confusion such fragmentation brings is obvious. One becomes
> adept at converting Java DSL to blueprint or from blueprint to the Java DSL
> in one’s mind.
>
>
>
> The static profiles work great and will let us create a number of
> standardized appliances for a wide variety of topology concerns and not
> just for microservices. A “switchboard” appliance, for example, might be
> used for orchestrating microservices and managing the APIs. A “gateway”
> appliance might have standard JAAS, web service configuration and a routing
> mechanism for calling microservices. An “AB” appliance could be used for
> 80/20 testing. And so on. Take the idea of enterprise integration
> patterns and bring it up to enterprise integration patterns using Karaf
> appliances.
>
>
>
> Many appliances might be “sealed”. An appliance for AB testing, for
> example, would have configuration for two addresses in the configuration
> file and a percentage of traffic going to each. Non need to actually
> program or re-program the internals any more than we’d usually re-program a
> Camel component. But the source would be there if one wanted to create a
> new component or modify how an existing one functioned.
>
>
>
> I’d vote for using CDI along with the Camel Java DSL for a couple of
> reasons. The first would be the standardization and portability of both
> code and skills. Using CDI would mean that any Glassfish, JBoss, etc.
> developer would feel comfortable using the code. Using Java Camel DSL
> would be for the same reason. It would also give a programmer a sense that
> if they give Karaf Boot with the static profiles a shot, they aren’t locked
> in but can easily move to a different stack if necessary. In a sense this
> is the same reason that Google chose Java language to run on the DVM. It
> tapped into a large existing skillbase instead of trying to get the world
> to adopt and learn a new language. CDI with OSGi extensions also allows
> developers to use one paradigm for everything from lashing up internal
> dependency injections to working with OSGi services. I believe when you put
> that CDI extension out there you used blueprint style proxies under the
> cover. As a developer using the CDI OSGi extension it was transparent to
> me. If you later decided to rework that as a DS service, it would remain
> transparent and very much in the whole spirit of OSGi and its mechanisms
> for allowing refactoring and even rewriting without breaking the world. It
> also makes unit testing a snap. Any of us who have wrestled with Camel
> Blueprint Test Support can appreciate that.
>
>
>
> This would also permit for standardization of documentation and of Karaf
> Boot appliance project structures and Maven plug-in use. A bit of
> convention over configuration. Projects would have a standard
> configuration.cfg file that gets deployed via features to the pid.cfg. A
> standard features file in the filtered folder. Those already exist, of
> course, but it isn’t as standardized as it could be.
>
>
>
> Personally I think this sort of goal with CDI, Karaf 4 and its profiles,
> and Camel Java DSL should be accelerated since Spring Boot is already out
> there. Waiting for another couple of years to release this as a standard
> might be too late.
>
>
>
> The pieces are already there so it isn’t like we’d have to start from
> scratch. This would also play well with larger container concerns like
> Docker and Kubernettes.
>
>
>
> Brad
>
>
>
> *From:* Guillaume Nodet [mailto:gnodet@apache.org]
> *Sent:* Thursday, January 12, 2017 4:55 AMJ
> *To:* user <us...@karaf.apache.org>
> *Subject:* Re: Levels of Containerization - focus on Docker and Karaf
>
>
>
> Fwiw, starting with Karaf 4.x, you can build custom distributions which
> are mostly static, and that more closely map to micro-services / docker
> images. The "static" images are called this way because you they kinda
> remove all the OSGi dynamism, i.e. no feature service, no deploy folder,
> read-only config admin, all bundles being installed at startup time from
> etc/startup.properties.
>
> This can be easily done by using the karaf maven plugin and configuring
> startupFeatures and referencing the static kar, as shown in:
>
> https://github.com/apache/karaf/blob/master/demos/
> profiles/static/pom.xml
>
>
>
>
>
> 2017-01-11 21:07 GMT+01:00 CodeCola <pr...@rogers.com>:
>
> Not a question but a request for comments. With a focus on Java.
>
> Container technology has traditionally been messy with dependencies and no
> easy failsafe way until Docker came along to really pack ALL dependencies
> (including the JVM) together in one ready-to-ship image that was faster,
> more comfortable, and easier to understand than other container and code
> shipping methods out there. The spectrum from (Classical) Java EE
> Containers
> (e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
> (Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
> Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
> with different goals (abstraction, isolation and delivery).
>
> What are the choices, how should they play together, should they be used in
> conjunction with each other as they offer different kinds of
> Containerization?
>
> <http://karaf.922171.n3.nabble.com/file/n4049162/
> Levels_of_Containerization.png>
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-
> Containerization-focus-on-Docker-and-Karaf-tp4049162.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
>
>
>
>
> --
>
> ------------------------
> Guillaume Nodet
> ------------------------
>
> Red Hat, Open Source Integration
>
>
>
> Email: gnodet@redhat.com
> Web: http://fusesource.com
> Blog: http://gnodet.blogspot.com/
>
>
>
--
Apache Member
Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>
Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS>
Software Architect / Project Manager / Scrum Master
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Nick Baker <nb...@pentaho.com>.
I do agree that an "opinionated" or "prescriptive" stack would help. It shouldn't prohibit the usage of any Karaf feature of course.
New users gravitate to full-stack solutions. Agnostic platforms with lots of options and no predefined stack, while obviously having many merits and longer legs (don't be Wicket), just haven't been winning out in adoption. This applies across the spectrum of computing.
________________________________
From: Brad Johnson <br...@redhat.com>
Sent: Thursday, January 12, 2017 10:46:55 AM
To: user@karaf.apache.org
Subject: RE: Levels of Containerization - focus on Docker and Karaf
Guillaume,
I’d mentioned that in an early post as my preferred way to do microservices and perhaps a good way of doing a Karaf Boot. I’ve worked with the Karaf 4 profiles and they are great. Also used your CDI OSGi service. If we could use the Karaf 4 profiles with the CDI implementation with OSGi services and the Camel Java DSL as a standard stack, it would permit focused development and standardized bundle configurations.
When I created a zip with Karaf, CXF and Camel the footprint was 30MB.
While having Karaf Boot support DS, Blueprint, CDI, etc. I’m not sure that’s the healthiest move to encourage adoption. We need less fragmentation in the OSGi world and not more. Obviously even if Karaf Boot adopts one as the recommended standard it doesn’t mean that the others can’t be used. When reading through Camel documentation on-line, for example, the confusion such fragmentation brings is obvious. One becomes adept at converting Java DSL to blueprint or from blueprint to the Java DSL in one’s mind.
The static profiles work great and will let us create a number of standardized appliances for a wide variety of topology concerns and not just for microservices. A “switchboard” appliance, for example, might be used for orchestrating microservices and managing the APIs. A “gateway” appliance might have standard JAAS, web service configuration and a routing mechanism for calling microservices. An “AB” appliance could be used for 80/20 testing. And so on. Take the idea of enterprise integration patterns and bring it up to enterprise integration patterns using Karaf appliances.
Many appliances might be “sealed”. An appliance for AB testing, for example, would have configuration for two addresses in the configuration file and a percentage of traffic going to each. Non need to actually program or re-program the internals any more than we’d usually re-program a Camel component. But the source would be there if one wanted to create a new component or modify how an existing one functioned.
I’d vote for using CDI along with the Camel Java DSL for a couple of reasons. The first would be the standardization and portability of both code and skills. Using CDI would mean that any Glassfish, JBoss, etc. developer would feel comfortable using the code. Using Java Camel DSL would be for the same reason. It would also give a programmer a sense that if they give Karaf Boot with the static profiles a shot, they aren’t locked in but can easily move to a different stack if necessary. In a sense this is the same reason that Google chose Java language to run on the DVM. It tapped into a large existing skillbase instead of trying to get the world to adopt and learn a new language. CDI with OSGi extensions also allows developers to use one paradigm for everything from lashing up internal dependency injections to working with OSGi services. I believe when you put that CDI extension out there you used blueprint style proxies under the cover. As a developer using the CDI OSGi extension it was transparent to me. If you later decided to rework that as a DS service, it would remain transparent and very much in the whole spirit of OSGi and its mechanisms for allowing refactoring and even rewriting without breaking the world. It also makes unit testing a snap. Any of us who have wrestled with Camel Blueprint Test Support can appreciate that.
This would also permit for standardization of documentation and of Karaf Boot appliance project structures and Maven plug-in use. A bit of convention over configuration. Projects would have a standard configuration.cfg file that gets deployed via features to the pid.cfg. A standard features file in the filtered folder. Those already exist, of course, but it isn’t as standardized as it could be.
Personally I think this sort of goal with CDI, Karaf 4 and its profiles, and Camel Java DSL should be accelerated since Spring Boot is already out there. Waiting for another couple of years to release this as a standard might be too late.
The pieces are already there so it isn’t like we’d have to start from scratch. This would also play well with larger container concerns like Docker and Kubernettes.
Brad
From: Guillaume Nodet [mailto:gnodet@apache.org]
Sent: Thursday, January 12, 2017 4:55 AMJ
To: user <us...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <pr...@rogers.com>>:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: gnodet@redhat.com<ma...@redhat.com>
Web: http://fusesource.com<http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
RE: Levels of Containerization - focus on Docker and Karaf
Posted by Brad Johnson <br...@redhat.com>.
Guillaume,
I’d mentioned that in an early post as my preferred way to do microservices and perhaps a good way of doing a Karaf Boot. I’ve worked with the Karaf 4 profiles and they are great. Also used your CDI OSGi service. If we could use the Karaf 4 profiles with the CDI implementation with OSGi services and the Camel Java DSL as a standard stack, it would permit focused development and standardized bundle configurations.
When I created a zip with Karaf, CXF and Camel the footprint was 30MB.
While having Karaf Boot support DS, Blueprint, CDI, etc. I’m not sure that’s the healthiest move to encourage adoption. We need less fragmentation in the OSGi world and not more. Obviously even if Karaf Boot adopts one as the recommended standard it doesn’t mean that the others can’t be used. When reading through Camel documentation on-line, for example, the confusion such fragmentation brings is obvious. One becomes adept at converting Java DSL to blueprint or from blueprint to the Java DSL in one’s mind.
The static profiles work great and will let us create a number of standardized appliances for a wide variety of topology concerns and not just for microservices. A “switchboard” appliance, for example, might be used for orchestrating microservices and managing the APIs. A “gateway” appliance might have standard JAAS, web service configuration and a routing mechanism for calling microservices. An “AB” appliance could be used for 80/20 testing. And so on. Take the idea of enterprise integration patterns and bring it up to enterprise integration patterns using Karaf appliances.
Many appliances might be “sealed”. An appliance for AB testing, for example, would have configuration for two addresses in the configuration file and a percentage of traffic going to each. Non need to actually program or re-program the internals any more than we’d usually re-program a Camel component. But the source would be there if one wanted to create a new component or modify how an existing one functioned.
I’d vote for using CDI along with the Camel Java DSL for a couple of reasons. The first would be the standardization and portability of both code and skills. Using CDI would mean that any Glassfish, JBoss, etc. developer would feel comfortable using the code. Using Java Camel DSL would be for the same reason. It would also give a programmer a sense that if they give Karaf Boot with the static profiles a shot, they aren’t locked in but can easily move to a different stack if necessary. In a sense this is the same reason that Google chose Java language to run on the DVM. It tapped into a large existing skillbase instead of trying to get the world to adopt and learn a new language. CDI with OSGi extensions also allows developers to use one paradigm for everything from lashing up internal dependency injections to working with OSGi services. I believe when you put that CDI extension out there you used blueprint style proxies under the cover. As a developer using the CDI OSGi extension it was transparent to me. If you later decided to rework that as a DS service, it would remain transparent and very much in the whole spirit of OSGi and its mechanisms for allowing refactoring and even rewriting without breaking the world. It also makes unit testing a snap. Any of us who have wrestled with Camel Blueprint Test Support can appreciate that.
This would also permit for standardization of documentation and of Karaf Boot appliance project structures and Maven plug-in use. A bit of convention over configuration. Projects would have a standard configuration.cfg file that gets deployed via features to the pid.cfg. A standard features file in the filtered folder. Those already exist, of course, but it isn’t as standardized as it could be.
Personally I think this sort of goal with CDI, Karaf 4 and its profiles, and Camel Java DSL should be accelerated since Spring Boot is already out there. Waiting for another couple of years to release this as a standard might be too late.
The pieces are already there so it isn’t like we’d have to start from scratch. This would also play well with larger container concerns like Docker and Kubernettes.
Brad
From: Guillaume Nodet [mailto:gnodet@apache.org]
Sent: Thursday, January 12, 2017 4:55 AMJ
To: user <us...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <prasenjit@rogers.com <ma...@rogers.com> >:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: gnodet@redhat.com <ma...@redhat.com>
Web: http://fusesource.com <http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Elliot Metsger <em...@gmail.com>.
Hi Dario,
On Fri, Jan 13, 2017 at 4:53 PM, Dario Amiri <da...@hotmail.com>
wrote:
> The point is that Karaf does not read configuration from an environment
> variables; it reads configuration from files. In Docker, you can pass
> environment variables in the arguments for running a container; however,
> you can't pass configuration files in those argument.
>
I think we are doing what you suggest above. Our Karaf configuration files
(e.g. [1]) reference environment variables [2] which are provided by a
docker file. [3]
[1]
https://github.com/fcrepo4-labs/fcrepo-api-x-demo/blob/master/apix/0.1.0/cfg/
[2]
https://github.com/fcrepo4-labs/fcrepo-api-x-demo/blob/master/apix/0.1.0/cfg/org.fcrepo.apix.routing.cfg#L1
[3]
https://github.com/fcrepo4-labs/fcrepo-api-x-demo/blob/master/apix/0.1.0/Dockerfile#L19
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Dario Amiri <da...@hotmail.com>.
The point is that Karaf does not read configuration from an environment variables; it reads configuration from files. In Docker, you can pass environment variables in the arguments for running a container; however, you can't pass configuration files in those argument.
Due to this limitation, rather than using a single Docker container for all possible configurations. I have to create a separate Docker image for every possible configuration. If you're not familiar with Docker, then an analogy would be if a Java application could only read it's configuration from a classpath resource of the containing jar. Then, I would have to build a the jar every time I wanted a different configuration. This is not the best way for doing things when building cloud native applications.
There's a set of well-accepted patterns and anti-patterns for building cloud native applications. Reading configuration from the environment instead of from the file system is one of them. I highly recommend having a look at: https://12factor.net
D
On 01/13/2017 01:06 PM, David Jencks wrote:
I don’t really know anything about Docker. It sounds like you can start Docker instances so that each instance has different environment variables? Otherwise it’s not clear to me what the point of what you want is.
I would think it would be pretty easy to extend fileinstall (or whatever Karaf uses to move the configuration from files to config admin) so it would do property substitution based on environment variables.
david jencks
On Jan 13, 2017, at 12:21 PM, Dario Amiri <<m...@hotmail.com>> wrote:
Let me expand on why this is desirable. Without the ability to set configuration through environment variables, I essentially have to create a docker image for each deployment. I have a root Dockerfile which assembles the main Karaf container image and brings in dependencies such as the JRE, then I have a Dockerfile for each deployment environment which builds on top of the root image by overriding deployment specific configuration. Automation reduces this burden but it is not ideal.tar
If I could set the contents of a config file in an environment variable, I could just pass the configuration directly to my root karaf docker image without having to build on top of it.
Being able to start Karaf as "java -jar karaf.jar" is desirable because it makes it easier to use a Karaf based application with PaaS such as Heroku and Cloud Foundry.
D
On 01/13/2017 12:10 PM, Dario Amiri wrote:
Ideally, I want to be able to do:
java -jar my-karaf.jar
And I can override individual configuration files using some environment variable convention.
D
On 01/13/2017 11:56 AM, Brad Johnson wrote:
Does it have to be an executable jar file or just a standalone executable? The static profiles actually create and zip up a full Karaf/felix/dependency/application implementation that when unzipped has all the standard bin directory items.
Brad
From: Dario Amiri [mailto:dariushamiri@hotmail.com]
Sent: Friday, January 13, 2017 1:28 PM
To: <ma...@karaf.apache.org> user@karaf.apache.org<ma...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
I use Docker and Karaf. I've never had a problem creating a Docker image of my Karaf container. What I gain is freedom from having to worry about dependency related issues such as whether the right JRE is available.
That being said there are some challenges when using Karaf to build 12-factor apps. FWIW here's my two item list of what would make Karaf a more attractive platform from a 12-factor app perspective.
1. The ability to inject Karaf configuration through the environment (e.g. environment variables). Not just a single property, but an entire config admin managed configuration file if necessary. Even the existing support for reading property values from the environment is cumbersome because it requires having to setup that relationship as a Java system property as well.
2. The ability to package Karaf as a standalone runnable jar. Looks like Karaf boot is addressing this. I hope it comes with tooling that makes it easy to transition to this kind of model.
D
On 01/12/2017 04:44 AM, Nick Baker wrote:
Thanks Guillaume!
This is perfect for our microservice/containerized Karaf. I'll give this a try and see if we can get our features in startup. We've had issues in the past here.
-Nick Baker
________________________________
From: Guillaume Nodet <ma...@apache.org> <gn...@apache.org>
Sent: Thursday, January 12, 2017 5:55:24 AM
To: user
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
<https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml> https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <<m...@rogers.com>>:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html> http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com<http://Nabble.com>.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: <ma...@redhat.com> gnodet@redhat.com<ma...@redhat.com>
Web: http://fusesource.com<http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
Re: Levels of Containerization - focus on Docker and Karaf
Posted by David Jencks <da...@gmail.com>.
I don’t really know anything about Docker. It sounds like you can start Docker instances so that each instance has different environment variables? Otherwise it’s not clear to me what the point of what you want is.
I would think it would be pretty easy to extend fileinstall (or whatever Karaf uses to move the configuration from files to config admin) so it would do property substitution based on environment variables.
david jencks
> On Jan 13, 2017, at 12:21 PM, Dario Amiri <da...@hotmail.com> wrote:
>
> Let me expand on why this is desirable. Without the ability to set configuration through environment variables, I essentially have to create a docker image for each deployment. I have a root Dockerfile which assembles the main Karaf container image and brings in dependencies such as the JRE, then I have a Dockerfile for each deployment environment which builds on top of the root image by overriding deployment specific configuration. Automation reduces this burden but it is not ideal.tar
>
> If I could set the contents of a config file in an environment variable, I could just pass the configuration directly to my root karaf docker image without having to build on top of it.
>
>
>
> Being able to start Karaf as "java -jar karaf.jar" is desirable because it makes it easier to use a Karaf based application with PaaS such as Heroku and Cloud Foundry.
>
> D
>
>
> On 01/13/2017 12:10 PM, Dario Amiri wrote:
>> Ideally, I want to be able to do:
>>
>> java -jar my-karaf.jar
>>
>> And I can override individual configuration files using some environment variable convention.
>>
>> D
>>
>>
>> On 01/13/2017 11:56 AM, Brad Johnson wrote:
>>> Does it have to be an executable jar file or just a standalone executable? The static profiles actually create and zip up a full Karaf/felix/dependency/application implementation that when unzipped has all the standard bin directory items.
>>>
>>> Brad
>>>
>>> From: Dario Amiri [mailto:dariushamiri@hotmail.com <ma...@hotmail.com>]
>>> Sent: Friday, January 13, 2017 1:28 PM
>>> To: user@karaf.apache.org <ma...@karaf.apache.org>
>>> Subject: Re: Levels of Containerization - focus on Docker and Karaf
>>>
>>> I use Docker and Karaf. I've never had a problem creating a Docker image of my Karaf container. What I gain is freedom from having to worry about dependency related issues such as whether the right JRE is available.
>>>
>>>
>>>
>>> That being said there are some challenges when using Karaf to build 12-factor apps. FWIW here's my two item list of what would make Karaf a more attractive platform from a 12-factor app perspective.
>>>
>>>
>>>
>>> 1. The ability to inject Karaf configuration through the environment (e.g. environment variables). Not just a single property, but an entire config admin managed configuration file if necessary. Even the existing support for reading property values from the environment is cumbersome because it requires having to setup that relationship as a Java system property as well.
>>>
>>> 2. The ability to package Karaf as a standalone runnable jar. Looks like Karaf boot is addressing this. I hope it comes with tooling that makes it easy to transition to this kind of model.
>>>
>>>
>>>
>>> D
>>>
>>>
>>> On 01/12/2017 04:44 AM, Nick Baker wrote:
>>> Thanks Guillaume!
>>>
>>>
>>>
>>> This is perfect for our microservice/containerized Karaf. I'll give this a try and see if we can get our features in startup. We've had issues in the past here.
>>>
>>>
>>>
>>> -Nick Baker
>>>
>>> From: Guillaume Nodet <gn...@apache.org> <ma...@apache.org>
>>> Sent: Thursday, January 12, 2017 5:55:24 AM
>>> To: user
>>> Subject: Re: Levels of Containerization - focus on Docker and Karaf
>>>
>>> Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
>>> This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
>>> https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml <https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml>
>>>
>>>
>>> 2017-01-11 21:07 GMT+01:00 CodeCola <prasenjit@rogers.com <ma...@rogers.com>>:
>>> Not a question but a request for comments. With a focus on Java.
>>>
>>> Container technology has traditionally been messy with dependencies and no
>>> easy failsafe way until Docker came along to really pack ALL dependencies
>>> (including the JVM) together in one ready-to-ship image that was faster,
>>> more comfortable, and easier to understand than other container and code
>>> shipping methods out there. The spectrum from (Classical) Java EE Containers
>>> (e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
>>> (Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
>>> Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
>>> with different goals (abstraction, isolation and delivery).
>>>
>>> What are the choices, how should they play together, should they be used in
>>> conjunction with each other as they offer different kinds of
>>> Containerization?
>>>
>>> <http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png <http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>>
>>>
>>>
>>>
>>> --
>>> View this message in context: <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html>http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html>
>>> Sent from the Karaf - User mailing list archive at Nabble.com.
>>>
>>>
>>>
>>> --
>>> ------------------------
>>> Guillaume Nodet
>>> ------------------------
>>> Red Hat, Open Source Integration
>>>
>>> Email: <ma...@redhat.com>gnodet@redhat.com <ma...@redhat.com>
>>> Web: http://fusesource.com <http://fusesource.com/>
>>> Blog: http://gnodet.blogspot.com/ <http://gnodet.blogspot.com/>
>>>
>>>
>>
>
>
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Elliot Metsger <em...@gmail.com>.
On Fri, Jan 13, 2017 at 5:04 PM, Dario Amiri <da...@hotmail.com>
wrote:
> I'm aware of this workaround. IIRC, you still need to include an
> additional Java system property in your invocation of karaf. It's a viable
> workaround but not the best situation.
>
We do not-
See:
https://github.com/fcrepo4-labs/fcrepo-api-x-demo/blob/master/apix/0.1.0/Dockerfile#L72
And:
https://github.com/fcrepo4-labs/fcrepo-api-x-demo/blob/master/apix/0.1.0/entrypoint.sh
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Dario Amiri <da...@hotmail.com>.
I'm aware of this workaround. IIRC, you still need to include an additional Java system property in your invocation of karaf. It's a viable workaround but not the best situation.
On 01/13/2017 01:17 PM, Elliot Metsger wrote:
We found we were able to use a formulation of property values that allowed for sensible defaults overridden by environment variables (e.g. [1]):
fcrepo.baseURI = ${env:FCREPO_BASEURI:-<http://localhost:8080/fcrepo/rest>http://localhost:8080/fcrepo/rest}
fcrepo.proxyURI = ${env:FCREPO_PROXYURI:-<http://localhost:8080/fcrepo>http://localhost:8080/fcrepo}
apix.host = ${env:APIX_HOST:-localhost}
apix.port = ${env:APIX_PORT:-80}
apix.interceptPath = ${env:APIX_INTERCEPT_PATH:-fcrepo/rest}
apix.proxyPath = ${env:APIX_PROXY_PATH:-fcrepo}
[1] https://github.com/fcrepo4-labs/fcrepo-api-x-demo/blob/master/apix/0.1.0/cfg/org.fcrepo.apix.routing.cfg
On Fri, Jan 13, 2017 at 4:13 PM, Nick Baker <nb...@pentaho.com>> wrote:
Injecting configuration into a containerized app (docker) is considered standard practice. The friction here is the level of sophistication in OSGI Configuration.
It seems to me what you need isn't some hack to push configurations through environment variables, but a new implementation of ConfigurationAdmin, or an agent which interacts with CM mirroring configurations in from an external system.
In our usage it's common to have a "tenant", think Walmart vs Target. Setting the tenant ID as an environment variable then having the configurations loaded from Zookeeper or whatever, injected into CM seems right.
-Nick
________________________________
From: Dario Amiri <<m...@hotmail.com>>
Sent: Friday, January 13, 2017 3:21:17 PM
To: user@karaf.apache.org<ma...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Let me expand on why this is desirable. Without the ability to set configuration through environment variables, I essentially have to create a docker image for each deployment. I have a root Dockerfile which assembles the main Karaf container image and brings in dependencies such as the JRE, then I have a Dockerfile for each deployment environment which builds on top of the root image by overriding deployment specific configuration. Automation reduces this burden but it is not ideal.
If I could set the contents of a config file in an environment variable, I could just pass the configuration directly to my root karaf docker image without having to build on top of it.
Being able to start Karaf as "java -jar karaf.jar" is desirable because it makes it easier to use a Karaf based application with PaaS such as Heroku and Cloud Foundry.
D
On 01/13/2017 12:10 PM, Dario Amiri wrote:
Ideally, I want to be able to do:
java -jar my-karaf.jar
And I can override individual configuration files using some environment variable convention.
D
On 01/13/2017 11:56 AM, Brad Johnson wrote:
Does it have to be an executable jar file or just a standalone executable? The static profiles actually create and zip up a full Karaf/felix/dependency/application implementation that when unzipped has all the standard bin directory items.
Brad
From: Dario Amiri [mailto:dariushamiri@hotmail.com]
Sent: Friday, January 13, 2017 1:28 PM
To: user@karaf.apache.org<ma...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
I use Docker and Karaf. I've never had a problem creating a Docker image of my Karaf container. What I gain is freedom from having to worry about dependency related issues such as whether the right JRE is available.
That being said there are some challenges when using Karaf to build 12-factor apps. FWIW here's my two item list of what would make Karaf a more attractive platform from a 12-factor app perspective.
1. The ability to inject Karaf configuration through the environment (e.g. environment variables). Not just a single property, but an entire config admin managed configuration file if necessary. Even the existing support for reading property values from the environment is cumbersome because it requires having to setup that relationship as a Java system property as well.
2. The ability to package Karaf as a standalone runnable jar. Looks like Karaf boot is addressing this. I hope it comes with tooling that makes it easy to transition to this kind of model.
D
On 01/12/2017 04:44 AM, Nick Baker wrote:
Thanks Guillaume!
This is perfect for our microservice/containerized Karaf. I'll give this a try and see if we can get our features in startup. We've had issues in the past here.
-Nick Baker
________________________________
From: Guillaume Nodet <gn...@apache.org>
Sent: Thursday, January 12, 2017 5:55:24 AM
To: user
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
<https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml> https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <<m...@rogers.com>>:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html> http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: <ma...@redhat.com> gnodet@redhat.com<ma...@redhat.com>
Web: http://fusesource.com<http://fusesource.com/>
Blog: <http://gnodet.blogspot.com/> http://gnodet.blogspot.com/
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Elliot Metsger <em...@gmail.com>.
We found we were able to use a formulation of property values that allowed
for sensible defaults overridden by environment variables (e.g. [1]):
fcrepo.baseURI = ${env:FCREPO_BASEURI:-http://localhost:8080/fcrepo/rest}
fcrepo.proxyURI = ${env:FCREPO_PROXYURI:-http://localhost:8080/fcrepo}
apix.host = ${env:APIX_HOST:-localhost}
apix.port = ${env:APIX_PORT:-80}
apix.interceptPath = ${env:APIX_INTERCEPT_PATH:-fcrepo/rest}
apix.proxyPath = ${env:APIX_PROXY_PATH:-fcrepo}
[1]
https://github.com/fcrepo4-labs/fcrepo-api-x-demo/blob/master/apix/0.1.0/cfg/org.fcrepo.apix.routing.cfg
On Fri, Jan 13, 2017 at 4:13 PM, Nick Baker <nb...@pentaho.com> wrote:
> Injecting configuration into a containerized app (docker) is considered
> standard practice. The friction here is the level of sophistication in OSGI
> Configuration.
>
> It seems to me what you need isn't some hack to push configurations
> through environment variables, but a new implementation of
> ConfigurationAdmin, or an agent which interacts with CM mirroring
> configurations in from an external system.
>
> In our usage it's common to have a "tenant", think Walmart vs Target.
> Setting the tenant ID as an environment variable then having the
> configurations loaded from Zookeeper or whatever, injected into CM seems
> right.
>
> -Nick
> ------------------------------
> *From:* Dario Amiri <da...@hotmail.com>
> *Sent:* Friday, January 13, 2017 3:21:17 PM
>
> *To:* user@karaf.apache.org
> *Subject:* Re: Levels of Containerization - focus on Docker and Karaf
>
>
> Let me expand on why this is desirable. Without the ability to set
> configuration through environment variables, I essentially have to create a
> docker image for each deployment. I have a root Dockerfile which assembles
> the main Karaf container image and brings in dependencies such as the JRE,
> then I have a Dockerfile for each deployment environment which builds on
> top of the root image by overriding deployment specific configuration.
> Automation reduces this burden but it is not ideal.
>
> If I could set the contents of a config file in an environment variable, I
> could just pass the configuration directly to my root karaf docker image
> without having to build on top of it.
>
>
> Being able to start Karaf as "java -jar karaf.jar" is desirable because it
> makes it easier to use a Karaf based application with PaaS such as Heroku
> and Cloud Foundry.
>
> D
>
> On 01/13/2017 12:10 PM, Dario Amiri wrote:
>
> Ideally, I want to be able to do:
>
> java -jar my-karaf.jar
>
> And I can override individual configuration files using some environment
> variable convention.
>
> D
>
> On 01/13/2017 11:56 AM, Brad Johnson wrote:
>
> Does it have to be an executable jar file or just a standalone executable?
> The static profiles actually create and zip up a full
> Karaf/felix/dependency/application implementation that when unzipped has
> all the standard bin directory items.
>
>
>
> Brad
>
>
>
> *From:* Dario Amiri [mailto:dariushamiri@hotmail.com
> <da...@hotmail.com>]
> *Sent:* Friday, January 13, 2017 1:28 PM
> *To:* user@karaf.apache.org
> *Subject:* Re: Levels of Containerization - focus on Docker and Karaf
>
>
>
> I use Docker and Karaf. I've never had a problem creating a Docker image
> of my Karaf container. What I gain is freedom from having to worry about
> dependency related issues such as whether the right JRE is available.
>
>
>
> That being said there are some challenges when using Karaf to build
> 12-factor apps. FWIW here's my two item list of what would make Karaf a
> more attractive platform from a 12-factor app perspective.
>
>
>
> 1. The ability to inject Karaf configuration through the environment (e.g.
> environment variables). Not just a single property, but an entire config
> admin managed configuration file if necessary. Even the existing support
> for reading property values from the environment is cumbersome because it
> requires having to setup that relationship as a Java system property as
> well.
>
> 2. The ability to package Karaf as a standalone runnable jar. Looks like
> Karaf boot is addressing this. I hope it comes with tooling that makes it
> easy to transition to this kind of model.
>
>
>
> D
>
>
>
> On 01/12/2017 04:44 AM, Nick Baker wrote:
>
> Thanks Guillaume!
>
>
>
> This is perfect for our microservice/containerized Karaf. I'll give this a
> try and see if we can get our features in startup. We've had issues in the
> past here.
>
>
>
> -Nick Baker
> ------------------------------
>
> *From:* Guillaume Nodet <gn...@apache.org> <gn...@apache.org>
> *Sent:* Thursday, January 12, 2017 5:55:24 AM
> *To:* user
> *Subject:* Re: Levels of Containerization - focus on Docker and Karaf
>
>
>
> Fwiw, starting with Karaf 4.x, you can build custom distributions which
> are mostly static, and that more closely map to micro-services / docker
> images. The "static" images are called this way because you they kinda
> remove all the OSGi dynamism, i.e. no feature service, no deploy folder,
> read-only config admin, all bundles being installed at startup time from
> etc/startup.properties.
>
> This can be easily done by using the karaf maven plugin and configuring
> startupFeatures and referencing the static kar, as shown in:
>
> https://github.com/apache/karaf/blob/master/demos/
> profiles/static/pom.xml
>
>
>
>
>
> 2017-01-11 21:07 GMT+01:00 CodeCola <pr...@rogers.com>:
>
> Not a question but a request for comments. With a focus on Java.
>
> Container technology has traditionally been messy with dependencies and no
> easy failsafe way until Docker came along to really pack ALL dependencies
> (including the JVM) together in one ready-to-ship image that was faster,
> more comfortable, and easier to understand than other container and code
> shipping methods out there. The spectrum from (Classical) Java EE
> Containers
> (e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
> (Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
> Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
> with different goals (abstraction, isolation and delivery).
>
> What are the choices, how should they play together, should they be used in
> conjunction with each other as they offer different kinds of
> Containerization?
>
> <http://karaf.922171.n3.nabble.com/file/n4049162/
> Levels_of_Containerization.png>
>
>
>
> --
> View this message in context:
> <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html>
> http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-
> Docker-and-Karaf-tp4049162.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
>
>
>
>
> --
>
> ------------------------
> Guillaume Nodet
> ------------------------
>
> Red Hat, Open Source Integration
>
>
>
> Email: <gn...@redhat.com>gnodet@redhat.com
> Web: http://fusesource.com
> Blog: http://gnodet.blogspot.com/
>
>
>
>
>
>
>
>
Re: Levels of Containerization - focus on Docker and Karaf
Posted by David Jencks <da...@gmail.com>.
Thanks for the Docker info.
You’re looking for a management agent, not a different config admin. I suggested one, it would also be easy to write one that used suitably encoded keys for environment variables to construct configurations e.g. <pid>.<configkey>=<value>. Not knowing how much of the configuration one might want to supply in environment variables, or how many values might be reused in many places, I assumed this would be less convenient to use than substituting a few values into larger files. My point is intended to be that it’s not hard to write a management agent, especially after you’ve decided what you want it to do.
david jencks
> On Jan 13, 2017, at 1:13 PM, Nick Baker <nb...@pentaho.com> wrote:
>
> Injecting configuration into a containerized app (docker) is considered standard practice. The friction here is the level of sophistication in OSGI Configuration.
>
> It seems to me what you need isn't some hack to push configurations through environment variables, but a new implementation of ConfigurationAdmin, or an agent which interacts with CM mirroring configurations in from an external system.
>
> In our usage it's common to have a "tenant", think Walmart vs Target. Setting the tenant ID as an environment variable then having the configurations loaded from Zookeeper or whatever, injected into CM seems right.
>
> -Nick
> From: Dario Amiri <da...@hotmail.com>
> Sent: Friday, January 13, 2017 3:21:17 PM
> To: user@karaf.apache.org
> Subject: Re: Levels of Containerization - focus on Docker and Karaf
>
> Let me expand on why this is desirable. Without the ability to set configuration through environment variables, I essentially have to create a docker image for each deployment. I have a root Dockerfile which assembles the main Karaf container image and brings in dependencies such as the JRE, then I have a Dockerfile for each deployment environment which builds on top of the root image by overriding deployment specific configuration. Automation reduces this burden but it is not ideal.
> If I could set the contents of a config file in an environment variable, I could just pass the configuration directly to my root karaf docker image without having to build on top of it.
>
> Being able to start Karaf as "java -jar karaf.jar" is desirable because it makes it easier to use a Karaf based application with PaaS such as Heroku and Cloud Foundry.
> D
>
> On 01/13/2017 12:10 PM, Dario Amiri wrote:
>> Ideally, I want to be able to do:
>> java -jar my-karaf.jar
>> And I can override individual configuration files using some environment variable convention.
>> D
>>
>> On 01/13/2017 11:56 AM, Brad Johnson wrote:
>>> Does it have to be an executable jar file or just a standalone executable? The static profiles actually create and zip up a full Karaf/felix/dependency/application implementation that when unzipped has all the standard bin directory items.
>>>
>>> Brad
>>>
>>> From: Dario Amiri [mailto:dariushamiri@hotmail.com <ma...@hotmail.com>]
>>> Sent: Friday, January 13, 2017 1:28 PM
>>> To: user@karaf.apache.org <ma...@karaf.apache.org>
>>> Subject: Re: Levels of Containerization - focus on Docker and Karaf
>>>
>>> I use Docker and Karaf. I've never had a problem creating a Docker image of my Karaf container. What I gain is freedom from having to worry about dependency related issues such as whether the right JRE is available.
>>>
>>> That being said there are some challenges when using Karaf to build 12-factor apps. FWIW here's my two item list of what would make Karaf a more attractive platform from a 12-factor app perspective.
>>>
>>> 1. The ability to inject Karaf configuration through the environment (e.g. environment variables). Not just a single property, but an entire config admin managed configuration file if necessary. Even the existing support for reading property values from the environment is cumbersome because it requires having to setup that relationship as a Java system property as well.
>>> 2. The ability to package Karaf as a standalone runnable jar. Looks like Karaf boot is addressing this. I hope it comes with tooling that makes it easy to transition to this kind of model.
>>>
>>> D
>>>
>>> On 01/12/2017 04:44 AM, Nick Baker wrote:
>>> Thanks Guillaume!
>>>
>>> This is perfect for our microservice/containerized Karaf. I'll give this a try and see if we can get our features in startup. We've had issues in the past here.
>>>
>>> -Nick Baker
>>> From: Guillaume Nodet <gn...@apache.org> <ma...@apache.org>
>>> Sent: Thursday, January 12, 2017 5:55:24 AM
>>> To: user
>>> Subject: Re: Levels of Containerization - focus on Docker and Karaf
>>>
>>> Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
>>> This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
>>> https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml <https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml>
>>>
>>>
>>> 2017-01-11 21:07 GMT+01:00 CodeCola <prasenjit@rogers.com <ma...@rogers.com>>:
>>> Not a question but a request for comments. With a focus on Java.
>>>
>>> Container technology has traditionally been messy with dependencies and no
>>> easy failsafe way until Docker came along to really pack ALL dependencies
>>> (including the JVM) together in one ready-to-ship image that was faster,
>>> more comfortable, and easier to understand than other container and code
>>> shipping methods out there. The spectrum from (Classical) Java EE Containers
>>> (e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
>>> (Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
>>> Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
>>> with different goals (abstraction, isolation and delivery).
>>>
>>> What are the choices, how should they play together, should they be used in
>>> conjunction with each other as they offer different kinds of
>>> Containerization?
>>>
>>> <http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png <http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>>
>>>
>>>
>>>
>>> --
>>> View this message in context: <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html>http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html>
>>> Sent from the Karaf - User mailing list archive at Nabble.com.
>>>
>>>
>>>
>>> --
>>> ------------------------
>>> Guillaume Nodet
>>> ------------------------
>>> Red Hat, Open Source Integration
>>>
>>> Email: <ma...@redhat.com>gnodet@redhat.com <ma...@redhat.com>
>>> Web: http://fusesource.com <http://fusesource.com/>
>>> Blog: http://gnodet.blogspot.com/ <http://gnodet.blogspot.com/>
>>>
>>>
>>
>
>
Re: Levels of Containerization - focus on Docker and Karaf
Posted by David Jencks <da...@gmail.com>.
I’m not sure you understand the difference between config admin and a management agent. IMO you are asking for a different management agent than the one Karaf usually uses.
So far I’ve proposed two sorts of environment-variable aware management agents:
- property substation into something like property files. This might be appropriate if the configurations needed for OSGI have a lot of stuff that doesn’t need to be configured per-docker-instance
- interpreting environment keys into pid + configuration key. This might be appropriate if all the information needed for OSGI should be configured per-docker-instance
Obviously there are a lot of other possible approaches. Figuring out how this might interact with the configurator spec proposed for R7 might be a good idea too.
Once you’ve decided what you want the management agent to do, writing it and assembling a karaf using it is pretty easy. I’m assuming that whoever does this is going to contribute it to Karaf and that since it’s obviously a good idea it will get included in some form. If I understand Elliot Metsger’s posts correctly karaf already supports the property substitution approach. What specifically do you want different from what’s already working?
david jencks
> On Jan 13, 2017, at 2:01 PM, Dario Amiri <da...@hotmail.com> wrote:
>
> You should not have to build a new docker container image for every possible type of configuration. Just look at any standard docker images in the public Docker registry. All configuration is driven from the environment variables. There is a multitude of examples in the public docker registry.
>
> A new implementation of ConfigurationAdmin sounds like a good way to solve the problem. However, I don't have the expertise to build and maintain that on my own. Nor should I because this is an ubiquitous pattern for all Dockerized applications. Karaf should do this out of the box if it wants to play well with Docker.
>
> Zookeeper is great, but what about the coordinates to Zookeeper? Where do I configure that? We're back to the same problem. Also, Zookeeper is likely too heavy a solution for some people (me included).
>
> D
>
> On 01/13/2017 01:13 PM, Nick Baker wrote:
>> Injecting configuration into a containerized app (docker) is considered standard practice. The friction here is the level of sophistication in OSGI Configuration.
>>
>> It seems to me what you need isn't some hack to push configurations through environment variables, but a new implementation of ConfigurationAdmin, or an agent which interacts with CM mirroring configurations in from an external system.
>>
>> In our usage it's common to have a "tenant", think Walmart vs Target. Setting the tenant ID as an environment variable then having the configurations loaded from Zookeeper or whatever, injected into CM seems right.
>>
>> -Nick
>> From: Dario Amiri <da...@hotmail.com> <ma...@hotmail.com>
>> Sent: Friday, January 13, 2017 3:21:17 PM
>> To: user@karaf.apache.org <ma...@karaf.apache.org>
>> Subject: Re: Levels of Containerization - focus on Docker and Karaf
>>
>> Let me expand on why this is desirable. Without the ability to set configuration through environment variables, I essentially have to create a docker image for each deployment. I have a root Dockerfile which assembles the main Karaf container image and brings in dependencies such as the JRE, then I have a Dockerfile for each deployment environment which builds on top of the root image by overriding deployment specific configuration. Automation reduces this burden but it is not ideal.
>> If I could set the contents of a config file in an environment variable, I could just pass the configuration directly to my root karaf docker image without having to build on top of it.
>>
>> Being able to start Karaf as "java -jar karaf.jar" is desirable because it makes it easier to use a Karaf based application with PaaS such as Heroku and Cloud Foundry.
>> D
>>
>> On 01/13/2017 12:10 PM, Dario Amiri wrote:
>>> Ideally, I want to be able to do:
>>> java -jar my-karaf.jar
>>> And I can override individual configuration files using some environment variable convention.
>>> D
>>>
>>> On 01/13/2017 11:56 AM, Brad Johnson wrote:
>>>> Does it have to be an executable jar file or just a standalone executable? The static profiles actually create and zip up a full Karaf/felix/dependency/application implementation that when unzipped has all the standard bin directory items.
>>>>
>>>> Brad
>>>>
>>>> From: Dario Amiri [mailto:dariushamiri@hotmail.com <ma...@hotmail.com>]
>>>> Sent: Friday, January 13, 2017 1:28 PM
>>>> To: user@karaf.apache.org <ma...@karaf.apache.org>
>>>> Subject: Re: Levels of Containerization - focus on Docker and Karaf
>>>>
>>>> I use Docker and Karaf. I've never had a problem creating a Docker image of my Karaf container. What I gain is freedom from having to worry about dependency related issues such as whether the right JRE is available.
>>>>
>>>> That being said there are some challenges when using Karaf to build 12-factor apps. FWIW here's my two item list of what would make Karaf a more attractive platform from a 12-factor app perspective.
>>>>
>>>> 1. The ability to inject Karaf configuration through the environment (e.g. environment variables). Not just a single property, but an entire config admin managed configuration file if necessary. Even the existing support for reading property values from the environment is cumbersome because it requires having to setup that relationship as a Java system property as well.
>>>> 2. The ability to package Karaf as a standalone runnable jar. Looks like Karaf boot is addressing this. I hope it comes with tooling that makes it easy to transition to this kind of model.
>>>>
>>>> D
>>>>
>>>> On 01/12/2017 04:44 AM, Nick Baker wrote:
>>>> Thanks Guillaume!
>>>>
>>>> This is perfect for our microservice/containerized Karaf. I'll give this a try and see if we can get our features in startup. We've had issues in the past here.
>>>>
>>>> -Nick Baker
>>>> From: Guillaume Nodet <gn...@apache.org> <ma...@apache.org>
>>>> Sent: Thursday, January 12, 2017 5:55:24 AM
>>>> To: user
>>>> Subject: Re: Levels of Containerization - focus on Docker and Karaf
>>>>
>>>> Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
>>>> This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
>>>> https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml <https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml>
>>>>
>>>>
>>>> 2017-01-11 21:07 GMT+01:00 CodeCola <prasenjit@rogers.com <ma...@rogers.com>>:
>>>> Not a question but a request for comments. With a focus on Java.
>>>>
>>>> Container technology has traditionally been messy with dependencies and no
>>>> easy failsafe way until Docker came along to really pack ALL dependencies
>>>> (including the JVM) together in one ready-to-ship image that was faster,
>>>> more comfortable, and easier to understand than other container and code
>>>> shipping methods out there. The spectrum from (Classical) Java EE Containers
>>>> (e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
>>>> (Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
>>>> Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
>>>> with different goals (abstraction, isolation and delivery).
>>>>
>>>> What are the choices, how should they play together, should they be used in
>>>> conjunction with each other as they offer different kinds of
>>>> Containerization?
>>>>
>>>> <http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png <http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>>
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context: <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html>http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html>
>>>> Sent from the Karaf - User mailing list archive at Nabble.com.
>>>>
>>>>
>>>>
>>>> --
>>>> ------------------------
>>>> Guillaume Nodet
>>>> ------------------------
>>>> Red Hat, Open Source Integration
>>>>
>>>> Email: <ma...@redhat.com>gnodet@redhat.com <ma...@redhat.com>
>>>> Web: http://fusesource.com <http://fusesource.com/>
>>>> Blog: http://gnodet.blogspot.com/ <http://gnodet.blogspot.com/>
>>>>
>>>>
>>>
>>
>
>
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Dario Amiri <da...@hotmail.com>.
You should not have to build a new docker container image for every possible type of configuration. Just look at any standard docker images in the public Docker registry. All configuration is driven from the environment variables. There is a multitude of examples in the public docker registry.
A new implementation of ConfigurationAdmin sounds like a good way to solve the problem. However, I don't have the expertise to build and maintain that on my own. Nor should I because this is an ubiquitous pattern for all Dockerized applications. Karaf should do this out of the box if it wants to play well with Docker.
Zookeeper is great, but what about the coordinates to Zookeeper? Where do I configure that? We're back to the same problem. Also, Zookeeper is likely too heavy a solution for some people (me included).
D
On 01/13/2017 01:13 PM, Nick Baker wrote:
Injecting configuration into a containerized app (docker) is considered standard practice. The friction here is the level of sophistication in OSGI Configuration.
It seems to me what you need isn't some hack to push configurations through environment variables, but a new implementation of ConfigurationAdmin, or an agent which interacts with CM mirroring configurations in from an external system.
In our usage it's common to have a "tenant", think Walmart vs Target. Setting the tenant ID as an environment variable then having the configurations loaded from Zookeeper or whatever, injected into CM seems right.
-Nick
________________________________
From: Dario Amiri <da...@hotmail.com>
Sent: Friday, January 13, 2017 3:21:17 PM
To: user@karaf.apache.org<ma...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Let me expand on why this is desirable. Without the ability to set configuration through environment variables, I essentially have to create a docker image for each deployment. I have a root Dockerfile which assembles the main Karaf container image and brings in dependencies such as the JRE, then I have a Dockerfile for each deployment environment which builds on top of the root image by overriding deployment specific configuration. Automation reduces this burden but it is not ideal.
If I could set the contents of a config file in an environment variable, I could just pass the configuration directly to my root karaf docker image without having to build on top of it.
Being able to start Karaf as "java -jar karaf.jar" is desirable because it makes it easier to use a Karaf based application with PaaS such as Heroku and Cloud Foundry.
D
On 01/13/2017 12:10 PM, Dario Amiri wrote:
Ideally, I want to be able to do:
java -jar my-karaf.jar
And I can override individual configuration files using some environment variable convention.
D
On 01/13/2017 11:56 AM, Brad Johnson wrote:
Does it have to be an executable jar file or just a standalone executable? The static profiles actually create and zip up a full Karaf/felix/dependency/application implementation that when unzipped has all the standard bin directory items.
Brad
From: Dario Amiri [mailto:dariushamiri@hotmail.com]
Sent: Friday, January 13, 2017 1:28 PM
To: user@karaf.apache.org<ma...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
I use Docker and Karaf. I've never had a problem creating a Docker image of my Karaf container. What I gain is freedom from having to worry about dependency related issues such as whether the right JRE is available.
That being said there are some challenges when using Karaf to build 12-factor apps. FWIW here's my two item list of what would make Karaf a more attractive platform from a 12-factor app perspective.
1. The ability to inject Karaf configuration through the environment (e.g. environment variables). Not just a single property, but an entire config admin managed configuration file if necessary. Even the existing support for reading property values from the environment is cumbersome because it requires having to setup that relationship as a Java system property as well.
2. The ability to package Karaf as a standalone runnable jar. Looks like Karaf boot is addressing this. I hope it comes with tooling that makes it easy to transition to this kind of model.
D
On 01/12/2017 04:44 AM, Nick Baker wrote:
Thanks Guillaume!
This is perfect for our microservice/containerized Karaf. I'll give this a try and see if we can get our features in startup. We've had issues in the past here.
-Nick Baker
________________________________
From: Guillaume Nodet <gn...@apache.org>
Sent: Thursday, January 12, 2017 5:55:24 AM
To: user
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <pr...@rogers.com>>:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html> http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: <ma...@redhat.com> gnodet@redhat.com<ma...@redhat.com>
Web: http://fusesource.com<http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Nick Baker <nb...@pentaho.com>.
Injecting configuration into a containerized app (docker) is considered standard practice. The friction here is the level of sophistication in OSGI Configuration.
It seems to me what you need isn't some hack to push configurations through environment variables, but a new implementation of ConfigurationAdmin, or an agent which interacts with CM mirroring configurations in from an external system.
In our usage it's common to have a "tenant", think Walmart vs Target. Setting the tenant ID as an environment variable then having the configurations loaded from Zookeeper or whatever, injected into CM seems right.
-Nick
________________________________
From: Dario Amiri <da...@hotmail.com>
Sent: Friday, January 13, 2017 3:21:17 PM
To: user@karaf.apache.org
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Let me expand on why this is desirable. Without the ability to set configuration through environment variables, I essentially have to create a docker image for each deployment. I have a root Dockerfile which assembles the main Karaf container image and brings in dependencies such as the JRE, then I have a Dockerfile for each deployment environment which builds on top of the root image by overriding deployment specific configuration. Automation reduces this burden but it is not ideal.
If I could set the contents of a config file in an environment variable, I could just pass the configuration directly to my root karaf docker image without having to build on top of it.
Being able to start Karaf as "java -jar karaf.jar" is desirable because it makes it easier to use a Karaf based application with PaaS such as Heroku and Cloud Foundry.
D
On 01/13/2017 12:10 PM, Dario Amiri wrote:
Ideally, I want to be able to do:
java -jar my-karaf.jar
And I can override individual configuration files using some environment variable convention.
D
On 01/13/2017 11:56 AM, Brad Johnson wrote:
Does it have to be an executable jar file or just a standalone executable? The static profiles actually create and zip up a full Karaf/felix/dependency/application implementation that when unzipped has all the standard bin directory items.
Brad
From: Dario Amiri [mailto:dariushamiri@hotmail.com]
Sent: Friday, January 13, 2017 1:28 PM
To: user@karaf.apache.org<ma...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
I use Docker and Karaf. I've never had a problem creating a Docker image of my Karaf container. What I gain is freedom from having to worry about dependency related issues such as whether the right JRE is available.
That being said there are some challenges when using Karaf to build 12-factor apps. FWIW here's my two item list of what would make Karaf a more attractive platform from a 12-factor app perspective.
1. The ability to inject Karaf configuration through the environment (e.g. environment variables). Not just a single property, but an entire config admin managed configuration file if necessary. Even the existing support for reading property values from the environment is cumbersome because it requires having to setup that relationship as a Java system property as well.
2. The ability to package Karaf as a standalone runnable jar. Looks like Karaf boot is addressing this. I hope it comes with tooling that makes it easy to transition to this kind of model.
D
On 01/12/2017 04:44 AM, Nick Baker wrote:
Thanks Guillaume!
This is perfect for our microservice/containerized Karaf. I'll give this a try and see if we can get our features in startup. We've had issues in the past here.
-Nick Baker
________________________________
From: Guillaume Nodet <gn...@apache.org>
Sent: Thursday, January 12, 2017 5:55:24 AM
To: user
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <pr...@rogers.com>>:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html> http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: <ma...@redhat.com> gnodet@redhat.com<ma...@redhat.com>
Web: http://fusesource.com<http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Dario Amiri <da...@hotmail.com>.
Let me expand on why this is desirable. Without the ability to set configuration through environment variables, I essentially have to create a docker image for each deployment. I have a root Dockerfile which assembles the main Karaf container image and brings in dependencies such as the JRE, then I have a Dockerfile for each deployment environment which builds on top of the root image by overriding deployment specific configuration. Automation reduces this burden but it is not ideal.
If I could set the contents of a config file in an environment variable, I could just pass the configuration directly to my root karaf docker image without having to build on top of it.
Being able to start Karaf as "java -jar karaf.jar" is desirable because it makes it easier to use a Karaf based application with PaaS such as Heroku and Cloud Foundry.
D
On 01/13/2017 12:10 PM, Dario Amiri wrote:
Ideally, I want to be able to do:
java -jar my-karaf.jar
And I can override individual configuration files using some environment variable convention.
D
On 01/13/2017 11:56 AM, Brad Johnson wrote:
Does it have to be an executable jar file or just a standalone executable? The static profiles actually create and zip up a full Karaf/felix/dependency/application implementation that when unzipped has all the standard bin directory items.
Brad
From: Dario Amiri [mailto:dariushamiri@hotmail.com]
Sent: Friday, January 13, 2017 1:28 PM
To: user@karaf.apache.org<ma...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
I use Docker and Karaf. I've never had a problem creating a Docker image of my Karaf container. What I gain is freedom from having to worry about dependency related issues such as whether the right JRE is available.
That being said there are some challenges when using Karaf to build 12-factor apps. FWIW here's my two item list of what would make Karaf a more attractive platform from a 12-factor app perspective.
1. The ability to inject Karaf configuration through the environment (e.g. environment variables). Not just a single property, but an entire config admin managed configuration file if necessary. Even the existing support for reading property values from the environment is cumbersome because it requires having to setup that relationship as a Java system property as well.
2. The ability to package Karaf as a standalone runnable jar. Looks like Karaf boot is addressing this. I hope it comes with tooling that makes it easy to transition to this kind of model.
D
On 01/12/2017 04:44 AM, Nick Baker wrote:
Thanks Guillaume!
This is perfect for our microservice/containerized Karaf. I'll give this a try and see if we can get our features in startup. We've had issues in the past here.
-Nick Baker
________________________________
From: Guillaume Nodet <gn...@apache.org>
Sent: Thursday, January 12, 2017 5:55:24 AM
To: user
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <pr...@rogers.com>>:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html> http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: <ma...@redhat.com> gnodet@redhat.com<ma...@redhat.com>
Web: http://fusesource.com<http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
Re: Levels of Containerization - focus on Docker and Karaf
Posted by David Daniel <da...@gmail.com>.
Things are working there way there with bndtools with bndrun files. You
can see a couple karaf libraries used in Christians chat sample
https://github.com/cschneider/osgi-chat The new cxf libraries also have
bndrun samples. Not everything in karaf can be generated in bndrun at this
time but I think some libraries are coming on to it. If you are interested
in that I would checkout the bndtools maven integrations in the more recent
releases.
On Fri, Jan 13, 2017 at 3:14 PM, Brad Johnson <br...@redhat.com> wrote:
> Interesting concept. That would be interesting to see.
>
>
>
> *From:* Dario Amiri [mailto:dariushamiri@hotmail.com]
> *Sent:* Friday, January 13, 2017 2:11 PM
>
> *To:* user@karaf.apache.org
> *Subject:* Re: Levels of Containerization - focus on Docker and Karaf
>
>
>
> Ideally, I want to be able to do:
>
> java -jar my-karaf.jar
>
> And I can override individual configuration files using some environment
> variable convention.
>
> D
>
>
>
> On 01/13/2017 11:56 AM, Brad Johnson wrote:
>
> Does it have to be an executable jar file or just a standalone executable?
> The static profiles actually create and zip up a full
> Karaf/felix/dependency/application implementation that when unzipped has
> all the standard bin directory items.
>
>
>
> Brad
>
>
>
> *From:* Dario Amiri [mailto:dariushamiri@hotmail.com
> <da...@hotmail.com>]
> *Sent:* Friday, January 13, 2017 1:28 PM
> *To:* user@karaf.apache.org
> *Subject:* Re: Levels of Containerization - focus on Docker and Karaf
>
>
>
> I use Docker and Karaf. I've never had a problem creating a Docker image
> of my Karaf container. What I gain is freedom from having to worry about
> dependency related issues such as whether the right JRE is available.
>
>
>
> That being said there are some challenges when using Karaf to build
> 12-factor apps. FWIW here's my two item list of what would make Karaf a
> more attractive platform from a 12-factor app perspective.
>
>
>
> 1. The ability to inject Karaf configuration through the environment (e.g.
> environment variables). Not just a single property, but an entire config
> admin managed configuration file if necessary. Even the existing support
> for reading property values from the environment is cumbersome because it
> requires having to setup that relationship as a Java system property as
> well.
>
> 2. The ability to package Karaf as a standalone runnable jar. Looks like
> Karaf boot is addressing this. I hope it comes with tooling that makes it
> easy to transition to this kind of model.
>
>
>
> D
>
>
>
> On 01/12/2017 04:44 AM, Nick Baker wrote:
>
> Thanks Guillaume!
>
>
>
> This is perfect for our microservice/containerized Karaf. I'll give this a
> try and see if we can get our features in startup. We've had issues in the
> past here.
>
>
>
> -Nick Baker
> ------------------------------
>
> *From:* Guillaume Nodet <gn...@apache.org> <gn...@apache.org>
> *Sent:* Thursday, January 12, 2017 5:55:24 AM
> *To:* user
> *Subject:* Re: Levels of Containerization - focus on Docker and Karaf
>
>
>
> Fwiw, starting with Karaf 4.x, you can build custom distributions which
> are mostly static, and that more closely map to micro-services / docker
> images. The "static" images are called this way because you they kinda
> remove all the OSGi dynamism, i.e. no feature service, no deploy folder,
> read-only config admin, all bundles being installed at startup time from
> etc/startup.properties.
>
> This can be easily done by using the karaf maven plugin and configuring
> startupFeatures and referencing the static kar, as shown in:
>
> https://github.com/apache/karaf/blob/master/demos/
> profiles/static/pom.xml
>
>
>
>
>
> 2017-01-11 21:07 GMT+01:00 CodeCola <pr...@rogers.com>:
>
> Not a question but a request for comments. With a focus on Java.
>
> Container technology has traditionally been messy with dependencies and no
> easy failsafe way until Docker came along to really pack ALL dependencies
> (including the JVM) together in one ready-to-ship image that was faster,
> more comfortable, and easier to understand than other container and code
> shipping methods out there. The spectrum from (Classical) Java EE
> Containers
> (e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
> (Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
> Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
> with different goals (abstraction, isolation and delivery).
>
> What are the choices, how should they play together, should they be used in
> conjunction with each other as they offer different kinds of
> Containerization?
>
> <http://karaf.922171.n3.nabble.com/file/n4049162/
> Levels_of_Containerization.png>
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-
> Containerization-focus-on-Docker-and-Karaf-tp4049162.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
>
>
>
>
> --
>
> ------------------------
> Guillaume Nodet
> ------------------------
>
> Red Hat, Open Source Integration
>
>
>
> Email: gnodet@redhat.com
> Web: http://fusesource.com
> Blog: http://gnodet.blogspot.com/
>
>
>
>
>
>
>
RE: Levels of Containerization - focus on Docker and Karaf
Posted by Brad Johnson <br...@redhat.com>.
Interesting concept. That would be interesting to see.
From: Dario Amiri [mailto:dariushamiri@hotmail.com]
Sent: Friday, January 13, 2017 2:11 PM
To: user@karaf.apache.org
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Ideally, I want to be able to do:
java -jar my-karaf.jar
And I can override individual configuration files using some environment
variable convention.
D
On 01/13/2017 11:56 AM, Brad Johnson wrote:
Does it have to be an executable jar file or just a standalone executable?
The static profiles actually create and zip up a full
Karaf/felix/dependency/application implementation that when unzipped has all
the standard bin directory items.
Brad
From: Dario Amiri [mailto:dariushamiri@hotmail.com]
Sent: Friday, January 13, 2017 1:28 PM
To: user@karaf.apache.org <ma...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
I use Docker and Karaf. I've never had a problem creating a Docker image of
my Karaf container. What I gain is freedom from having to worry about
dependency related issues such as whether the right JRE is available.
That being said there are some challenges when using Karaf to build
12-factor apps. FWIW here's my two item list of what would make Karaf a more
attractive platform from a 12-factor app perspective.
1. The ability to inject Karaf configuration through the environment (e.g.
environment variables). Not just a single property, but an entire config
admin managed configuration file if necessary. Even the existing support for
reading property values from the environment is cumbersome because it
requires having to setup that relationship as a Java system property as
well.
2. The ability to package Karaf as a standalone runnable jar. Looks like
Karaf boot is addressing this. I hope it comes with tooling that makes it
easy to transition to this kind of model.
D
On 01/12/2017 04:44 AM, Nick Baker wrote:
Thanks Guillaume!
This is perfect for our microservice/containerized Karaf. I'll give this a
try and see if we can get our features in startup. We've had issues in the
past here.
-Nick Baker
_____
From: Guillaume Nodet <ma...@apache.org> <gn...@apache.org>
Sent: Thursday, January 12, 2017 5:55:24 AM
To: user
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are
mostly static, and that more closely map to micro-services / docker images.
The "static" images are called this way because you they kinda remove all
the OSGi dynamism, i.e. no feature service, no deploy folder, read-only
config admin, all bundles being installed at startup time from
etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring
startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <prasenjit@rogers.com
<ma...@rogers.com> >:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.
png>
--
View this message in context:
http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker
-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: gnodet@redhat.com <ma...@redhat.com>
Web: http://fusesource.com <http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Dario Amiri <da...@hotmail.com>.
Ideally, I want to be able to do:
java -jar my-karaf.jar
And I can override individual configuration files using some environment variable convention.
D
On 01/13/2017 11:56 AM, Brad Johnson wrote:
Does it have to be an executable jar file or just a standalone executable? The static profiles actually create and zip up a full Karaf/felix/dependency/application implementation that when unzipped has all the standard bin directory items.
Brad
From: Dario Amiri [mailto:dariushamiri@hotmail.com]
Sent: Friday, January 13, 2017 1:28 PM
To: user@karaf.apache.org<ma...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
I use Docker and Karaf. I've never had a problem creating a Docker image of my Karaf container. What I gain is freedom from having to worry about dependency related issues such as whether the right JRE is available.
That being said there are some challenges when using Karaf to build 12-factor apps. FWIW here's my two item list of what would make Karaf a more attractive platform from a 12-factor app perspective.
1. The ability to inject Karaf configuration through the environment (e.g. environment variables). Not just a single property, but an entire config admin managed configuration file if necessary. Even the existing support for reading property values from the environment is cumbersome because it requires having to setup that relationship as a Java system property as well.
2. The ability to package Karaf as a standalone runnable jar. Looks like Karaf boot is addressing this. I hope it comes with tooling that makes it easy to transition to this kind of model.
D
On 01/12/2017 04:44 AM, Nick Baker wrote:
Thanks Guillaume!
This is perfect for our microservice/containerized Karaf. I'll give this a try and see if we can get our features in startup. We've had issues in the past here.
-Nick Baker
________________________________
From: Guillaume Nodet <gn...@apache.org>
Sent: Thursday, January 12, 2017 5:55:24 AM
To: user
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <pr...@rogers.com>>:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: <http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html> http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: <ma...@redhat.com> gnodet@redhat.com<ma...@redhat.com>
Web: http://fusesource.com<http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
RE: Levels of Containerization - focus on Docker and Karaf
Posted by Brad Johnson <br...@redhat.com>.
Does it have to be an executable jar file or just a standalone executable?
The static profiles actually create and zip up a full
Karaf/felix/dependency/application implementation that when unzipped has all
the standard bin directory items.
Brad
From: Dario Amiri [mailto:dariushamiri@hotmail.com]
Sent: Friday, January 13, 2017 1:28 PM
To: user@karaf.apache.org
Subject: Re: Levels of Containerization - focus on Docker and Karaf
I use Docker and Karaf. I've never had a problem creating a Docker image of
my Karaf container. What I gain is freedom from having to worry about
dependency related issues such as whether the right JRE is available.
That being said there are some challenges when using Karaf to build
12-factor apps. FWIW here's my two item list of what would make Karaf a more
attractive platform from a 12-factor app perspective.
1. The ability to inject Karaf configuration through the environment (e.g.
environment variables). Not just a single property, but an entire config
admin managed configuration file if necessary. Even the existing support for
reading property values from the environment is cumbersome because it
requires having to setup that relationship as a Java system property as
well.
2. The ability to package Karaf as a standalone runnable jar. Looks like
Karaf boot is addressing this. I hope it comes with tooling that makes it
easy to transition to this kind of model.
D
On 01/12/2017 04:44 AM, Nick Baker wrote:
Thanks Guillaume!
This is perfect for our microservice/containerized Karaf. I'll give this a
try and see if we can get our features in startup. We've had issues in the
past here.
-Nick Baker
_____
From: Guillaume Nodet <ma...@apache.org> <gn...@apache.org>
Sent: Thursday, January 12, 2017 5:55:24 AM
To: user
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are
mostly static, and that more closely map to micro-services / docker images.
The "static" images are called this way because you they kinda remove all
the OSGi dynamism, i.e. no feature service, no deploy folder, read-only
config admin, all bundles being installed at startup time from
etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring
startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <prasenjit@rogers.com
<ma...@rogers.com> >:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.
png>
--
View this message in context:
http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker
-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: gnodet@redhat.com <ma...@redhat.com>
Web: http://fusesource.com <http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Dario Amiri <da...@hotmail.com>.
I use Docker and Karaf. I've never had a problem creating a Docker image of my Karaf container. What I gain is freedom from having to worry about dependency related issues such as whether the right JRE is available.
That being said there are some challenges when using Karaf to build 12-factor apps. FWIW here's my two item list of what would make Karaf a more attractive platform from a 12-factor app perspective.
1. The ability to inject Karaf configuration through the environment (e.g. environment variables). Not just a single property, but an entire config admin managed configuration file if necessary. Even the existing support for reading property values from the environment is cumbersome because it requires having to setup that relationship as a Java system property as well.
2. The ability to package Karaf as a standalone runnable jar. Looks like Karaf boot is addressing this. I hope it comes with tooling that makes it easy to transition to this kind of model.
D
On 01/12/2017 04:44 AM, Nick Baker wrote:
Thanks Guillaume!
This is perfect for our microservice/containerized Karaf. I'll give this a try and see if we can get our features in startup. We've had issues in the past here.
-Nick Baker
________________________________
From: Guillaume Nodet <gn...@apache.org>
Sent: Thursday, January 12, 2017 5:55:24 AM
To: user
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <pr...@rogers.com>>:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: <ma...@redhat.com> gnodet@redhat.com<ma...@redhat.com>
Web: http://fusesource.com<http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
RE: Levels of Containerization - focus on Docker and Karaf
Posted by Brad Johnson <br...@redhat.com>.
Nick,
I've used the profiles and they work well. After the Maven plugin runs you
end up with a zip file that has karaf and all its dependencies
self-contained. As Guillaume notes the Karaf is slimmed down by removing
those services. But if one is compiling and zipping the whole Karaf
container with dependencies then installs as features really aren't needed.
It also will make black box testing of the resulting artifacts that much
easier.
If you look, for example, at the zip file that's created you can see how
small they are even with dependencies.
https://github.com/Ranx0r0x/Panda-Vault/tree/master/sample%20zip
I really think this could be huge if it gets standardized and standard
integration Karaf appliances get created. I'd certainly hop on board and
help develop those.
Brad
From: Nick Baker [mailto:nbaker@pentaho.com]
Sent: Thursday, January 12, 2017 6:45 AM
To: user <us...@karaf.apache.org>
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Thanks Guillaume!
This is perfect for our microservice/containerized Karaf. I'll give this a
try and see if we can get our features in startup. We've had issues in the
past here.
-Nick Baker
_____
From: Guillaume Nodet <gnodet@apache.org <ma...@apache.org> >
Sent: Thursday, January 12, 2017 5:55:24 AM
To: user
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are
mostly static, and that more closely map to micro-services / docker images.
The "static" images are called this way because you they kinda remove all
the OSGi dynamism, i.e. no feature service, no deploy folder, read-only
config admin, all bundles being installed at startup time from
etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring
startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <prasenjit@rogers.com
<ma...@rogers.com> >:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.
png>
--
View this message in context:
http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker
-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: gnodet@redhat.com <ma...@redhat.com>
Web: http://fusesource.com <http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Nick Baker <nb...@pentaho.com>.
Thanks Guillaume!
This is perfect for our microservice/containerized Karaf. I'll give this a try and see if we can get our features in startup. We've had issues in the past here.
-Nick Baker
________________________________
From: Guillaume Nodet <gn...@apache.org>
Sent: Thursday, January 12, 2017 5:55:24 AM
To: user
Subject: Re: Levels of Containerization - focus on Docker and Karaf
Fwiw, starting with Karaf 4.x, you can build custom distributions which are mostly static, and that more closely map to micro-services / docker images. The "static" images are called this way because you they kinda remove all the OSGi dynamism, i.e. no feature service, no deploy folder, read-only config admin, all bundles being installed at startup time from etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <pr...@rogers.com>>:
Not a question but a request for comments. With a focus on Java.
Container technology has traditionally been messy with dependencies and no
easy failsafe way until Docker came along to really pack ALL dependencies
(including the JVM) together in one ready-to-ship image that was faster,
more comfortable, and easier to understand than other container and code
shipping methods out there. The spectrum from (Classical) Java EE Containers
(e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
(Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
with different goals (abstraction, isolation and delivery).
What are the choices, how should they play together, should they be used in
conjunction with each other as they offer different kinds of
Containerization?
<http://karaf.922171.n3.nabble.com/file/n4049162/Levels_of_Containerization.png>
--
View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-Containerization-focus-on-Docker-and-Karaf-tp4049162.html
Sent from the Karaf - User mailing list archive at Nabble.com.
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: gnodet@redhat.com<ma...@redhat.com>
Web: http://fusesource.com<http://fusesource.com/>
Blog: http://gnodet.blogspot.com/
Re: Levels of Containerization - focus on Docker and Karaf
Posted by Guillaume Nodet <gn...@apache.org>.
Fwiw, starting with Karaf 4.x, you can build custom distributions which are
mostly static, and that more closely map to micro-services / docker
images. The "static" images are called this way because you they kinda
remove all the OSGi dynamism, i.e. no feature service, no deploy folder,
read-only config admin, all bundles being installed at startup time from
etc/startup.properties.
This can be easily done by using the karaf maven plugin and configuring
startupFeatures and referencing the static kar, as shown in:
https://github.com/apache/karaf/blob/master/demos/profiles/static/pom.xml
2017-01-11 21:07 GMT+01:00 CodeCola <pr...@rogers.com>:
> Not a question but a request for comments. With a focus on Java.
>
> Container technology has traditionally been messy with dependencies and no
> easy failsafe way until Docker came along to really pack ALL dependencies
> (including the JVM) together in one ready-to-ship image that was faster,
> more comfortable, and easier to understand than other container and code
> shipping methods out there. The spectrum from (Classical) Java EE
> Containers
> (e.g. Tomcat, Jetty) --> Java Application Servers that are containerized
> (Karaf, Wildfly, etc), Application Delivery Containers (Docker) and
> Virtualization (VMWare, Hyper-V) etc. offers a different level of isolation
> with different goals (abstraction, isolation and delivery).
>
> What are the choices, how should they play together, should they be used in
> conjunction with each other as they offer different kinds of
> Containerization?
>
> <http://karaf.922171.n3.nabble.com/file/n4049162/
> Levels_of_Containerization.png>
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Levels-of-
> Containerization-focus-on-Docker-and-Karaf-tp4049162.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: gnodet@redhat.com
Web: http://fusesource.com
Blog: http://gnodet.blogspot.com/