Deploying an application from jenkins for test

[to...@quarendon.net]

What's the best way of deploying an application from a CI build such as Jenkins,
into Karaf, for the purposes of testing?

I was hoping to set something up so that I have a jenkins job that builds my
application and then deploys the built application to a test instance on karaf.
The karaf would be running on a separate machine, so I would most likely write a
simple piece of java, either as an ANT task or similar that uses the JMX
management bean interface to interact with karaf (seems more reliable than
trying to script the shell interface and detect errors).

My question really is what the best was of transferring things was. Naively I
assumed that I would publish the artifacts into artifactory as snapshots and
then pull them from there using mvn: URLs, but maybe I just don't know enough
about how maven repositories work, as I'm getting into difficulties with the
artifacts being cached, so that what is deployed is what has just been built. 

I could probably pull them directly from jenkins using the
"lastSuccessfulBuild/artifacts" URLs, and maybe that the best way?

Any thoughts?
Thanks.

Re: Deploying an application from jenkins for test

[to...@quarendon.net]

Achim,Thanks.

I'm trying to use mvn, as it does seem the best option. If I can get it to do
what I want though.
What I'm trying to work out is how to ensure that I get exactly the bundle
that's just been published, and nothing gets cached.

I'm publishing SNAPSHOT builds to our artifactory repository.
With the default configuration, I don't get the latest snapshot. So I:

install the feature
Update the code.
Rebuild.
Publish
uninstall, then reinstall the feature.

I get the same bundle again.
Now I don't know anything about maven works, but there seems to be an "update
policy" that by default is "daily". The implication of this is that if I ran
this cycle over two days I might get what I want.

The PAX URL code seems to have a "globalUpdatePolicy" configuration. I set that
to "always", and now I get a complaint from aether about not being able to
resolve the artifact:

Could not find artifact
simple-osgi:simle.osgi.command:jar:1.0.0-20160921.072458-1 in
artifactory-snapshot(repository URL)
	at
shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)[7:org.ops4j.pax.url.mvn:2.4.7]
	at
shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)[7:org.ops4j.pax.url.mvn:2.4.7]
	at
shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223)[7:org.ops4j.pax.url.mvn:2.4.7]
	at
shaded.org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveArtifact(DefaultRepositorySystem.java:294)[7:org.ops4j.pax.url.mvn:2.4.7]
	at
org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:650)[7:org.ops4j.pax.url.mvn:2.4.7]
	at
org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:598)[7:org.ops4j.pax.url.mvn:2.4.7]
	at
org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:576)[7:org.ops4j.pax.url.mvn:2.4.7]
	at
org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:550)[7:org.ops4j.pax.url.mvn:2.4.7]
	at
org.apache.karaf.features.internal.download.impl.MavenDownloadTask.download(MavenDownloadTask.java:34)[8:org.apache.karaf.features.core:4.0.6]



The one in the artifactory is now 073344 instead.

So I don't understand what's going on here. Setting "globalUpdatePolicy" seems
to have said "ignore the cached artefacts and go get them again from the
remote", but what it doesn't seem to have said is "check for the latest SNAPSHOT
version", so it's still trying to download the version that it worked out last
time. 


As I say, my knowledge of Maven is minimal at best.
There must be a way of doing this though isn't there? I mean, convince it
somehow to just install whatever is latest on out artefactory repository? I get
the caching thing when you're dealing with release versions, but in this
environment, snapshots ought to not be cached, and attempting to get version
"1.0.0-SNAPSHOT" ought to just go and figure out what the latest version is and
download it (you can cache once you've resolved the SNAPSHOT into its actual
unique version, but not before).

Any ideas?

Thanks.

Re: Deploying an application from jenkins for test

[Achim Nierbeck <bc...@googlemail.com>]

Hi,

usually I use maven as the "transport" it's a much better integration into
the CI/CD pipeline.
So what I've given you as sample is optimized for exactly this.

regards, Achim

[1] -
http://image.slidesharecdn.com/microservices-osgi-running-with-apache-karaf-151001101049-lva1-app6891/95/microservices-osgirunningwithapachekaraf-35-638.jpg?cb=1443694516

2016-09-20 9:22 GMT+02:00 <to...@quarendon.net>:

> > you can use Pax Exam[1] for that and start Karaf embedded, this will give
> > you a clean state for every run.
>
> Initially I want to do it for testing, but I then want to deploy a
> "snapshot"
> and "release" version of the actual application for general internal demo
> use. I
> currently already have an integration test suite, but what it doesn't test
> is
> the deployment into karaf, so I want to deploy it for testing in a way
> that is
> as close to the real thing as possible.
>
> > The other way would be to have a Karaf with Jolokia running and deploy
> via
> > JMX, I once created a sample for that [2].
>
> As I say, my question was really one of what the transport is for the
> bundles.
> One way of the other I invoke the equivalent of "feature:add-repo", then
> "feature:install", my question is what are the URLs, where does karaf
> actually
> pull stuff from. To use mvn repositories I need to find a way of turning
> off the
> caching, which so far I've failed to do. Pushing the files on to the remote
> machine using SCP and then addressing them with "file:///" URLs seems
> another
> way, but it seems over complicated to push them. Addressing the artifacts
> in
> jenkins directly is another way, but it seems like I'm going to have to
> write
> something bespoke to create a feature repository with the correct URLs in
> from
> some kind of template (substituting the correct build number in all of the
> URLs).
>
> Just hoping there might be a known best way to do this kind of thing, save
> me
> figuring it out!
>
> Thanks.
>



-- 

Apache Member
Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>
Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS>

Software Architect / Project Manager / Scrum Master

Re: Deploying an application from jenkins for test

[to...@quarendon.net]

> you can use Pax Exam[1] for that and start Karaf embedded, this will give
> you a clean state for every run.

Initially I want to do it for testing, but I then want to deploy a "snapshot"
and "release" version of the actual application for general internal demo use. I
currently already have an integration test suite, but what it doesn't test is
the deployment into karaf, so I want to deploy it for testing in a way that is
as close to the real thing as possible.

> The other way would be to have a Karaf with Jolokia running and deploy via
> JMX, I once created a sample for that [2].

As I say, my question was really one of what the transport is for the bundles.
One way of the other I invoke the equivalent of "feature:add-repo", then
"feature:install", my question is what are the URLs, where does karaf actually
pull stuff from. To use mvn repositories I need to find a way of turning off the
caching, which so far I've failed to do. Pushing the files on to the remote
machine using SCP and then addressing them with "file:///" URLs seems another
way, but it seems over complicated to push them. Addressing the artifacts in
jenkins directly is another way, but it seems like I'm going to have to write
something bespoke to create a feature repository with the correct URLs in from
some kind of template (substituting the correct build number in all of the
URLs).

Just hoping there might be a known best way to do this kind of thing, save me
figuring it out!

Thanks.

Re: Deploying an application from jenkins for test

[Achim Nierbeck <bc...@googlemail.com>]

Hi,

you can use Pax Exam[1] for that and start Karaf embedded, this will give
you a clean state for every run.
The other way would be to have a Karaf with Jolokia running and deploy via
JMX, I once created a sample for that [2].

regards, Achim

[1] - https://ops4j1.jira.com/wiki/display/PAXEXAM4/Pax+Exam
[2] -
https://github.com/ANierbeck/Karaf-Microservices-Tooling/tree/master/karaf-deployer-maven-plugin

2016-09-20 9:05 GMT+02:00 <to...@quarendon.net>:

> What's the best way of deploying an application from a CI build such as
> Jenkins,
> into Karaf, for the purposes of testing?
>
> I was hoping to set something up so that I have a jenkins job that builds
> my
> application and then deploys the built application to a test instance on
> karaf.
> The karaf would be running on a separate machine, so I would most likely
> write a
> simple piece of java, either as an ANT task or similar that uses the JMX
> management bean interface to interact with karaf (seems more reliable than
> trying to script the shell interface and detect errors).
>
> My question really is what the best was of transferring things was.
> Naively I
> assumed that I would publish the artifacts into artifactory as snapshots
> and
> then pull them from there using mvn: URLs, but maybe I just don't know
> enough
> about how maven repositories work, as I'm getting into difficulties with
> the
> artifacts being cached, so that what is deployed is what has just been
> built.
>
> I could probably pull them directly from jenkins using the
> "lastSuccessfulBuild/artifacts" URLs, and maybe that the best way?
>
> Any thoughts?
> Thanks.
>



-- 

Apache Member
Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>
Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS>

Software Architect / Project Manager / Scrum Master