You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@ace.apache.org by "Robert M. Mather" <ro...@gmail.com> on 2015/11/21 18:07:20 UTC
Flaky deployment
I'm trying to deploy a set of bundles that resolves and runs fine inside
bndtools, but leads to errors on the (Felix) client. So far, I've seen:
> 2015-11-21 11:39:50 | ERROR | ACE Agent Controller |
org.apache.ace.agent.1.0.1 - Could not delete temporary deployment package
from disk
> 2015-11-21 11:39:50 | ERROR | ACE Agent Controller |
org.apache.ace.agent.1.0.1 - Installation of deployment update failed: The
InputStream is not a jar! java.io.IOException: Unknown/unexpected status
code: 500 at
org.apache.ace.agent.impl.ContentRangeInputStream.getHttpContentRangeInfo(ContentRangeInputStream.java:341)
~[na:na] at
org.apache.ace.agent.impl.ContentRangeInputStream.getContentRangeInfo(ContentRangeInputStream.java:288)
~[na:na] at
org.apache.ace.agent.impl.ContentRangeInputStream.prepareNextChunk(ContentRangeInputStream.java:418)
~[na:na] at
org.apache.ace.agent.impl.ContentRangeInputStream.read(ContentRangeInputStream.java:188)
~[na:na] at
org.apache.felix.deploymentadmin.OutputtingInputStream.read(OutputtingInputStream.java:64)
~[na:na] at java.io.FilterInputStream.read(FilterInputStream.java:133)
~[na:1.8.0_40] at
java.io.PushbackInputStream.read(PushbackInputStream.java:186)
~[na:1.8.0_40] at
java.util.zip.ZipInputStream.readFully(ZipInputStream.java:403)
~[na:1.8.0_40] at
java.util.zip.ZipInputStream.readLOC(ZipInputStream.java:278)
~[na:1.8.0_40] at
java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:122)
~[na:1.8.0_40] at
java.util.jar.JarInputStream.<init>(JarInputStream.java:83) ~[na:1.8.0_40]
at java.util.jar.JarInputStream.<init>(JarInputStream.java:62)
~[na:1.8.0_40] at
org.apache.felix.deploymentadmin.DeploymentAdminImpl.installDeploymentPackage(DeploymentAdminImpl.java:174)
~[na:na] at
org.apache.ace.agent.impl.DeploymentHandlerImpl.install(DeploymentHandlerImpl.java:237)
~[na:na] at
org.apache.ace.agent.impl.DefaultController$StreamingUpdateInstaller.doInstallUpdate(DefaultController.java:162)
~[na:na] at
org.apache.ace.agent.impl.DefaultController$UpdateInstaller.installUpdate(DefaultController.java:253)
~[na:na] at
org.apache.ace.agent.impl.DefaultController.runDeploymentUpdate(DefaultController.java:610)
~[na:na] at
org.apache.ace.agent.impl.DefaultController.run(DefaultController.java:460)
~[na:na] at
org.apache.ace.agent.impl.AgentContextImpl$1.run(AgentContextImpl.java:252)
~[na:na]
and also
> 2015-11-21 11:39:50 | ERROR | ACE Agent Controller |
org.apache.ace.agent.1.0.1 - Stream does not contain a valid Jar
java.io.IOException: Unknown/unexpected status code: 500 at
org.apache.ace.agent.impl.ContentRangeInputStream.getHttpContentRangeInfo(ContentRangeInputStream.java:341)
~[na:na] at
(rest of trace same as above)
and
INFO|1428/0||15-11-21 05:22:51|[ERROR] 05:22:51 (controller) Installation
of deployment update failed: Expected more bundles in the stream:
[com.google.guava-18.0.0.jar,
ring.order.inject.micros-1.0.6.201511210151_v120-84-g2a514b1-dirty.jar,
ring.api.menu.upload-1.0.1.201511210011_v120-84-g2a514b1-dirty.jar,
org.apache.httpcomponents.httpcore-4.3.3.jar,
ring.ping-1.0.4.201511210011_v120-84-g2a514b1-dirty.jar,
ring.menu.upload-1.0.2.201511210022_v120-84-g2a514b1-dirty.jar,
ring.creditCard.decrypt-1.0.8.201511210022_v120-84-g2a514b1-dirty.jar,
log4j.over.slf4j-1.7.12.jar,
org.apache.servicemix.bundles.json-20140107.0.0.1.jar,
ch.qos.logback.core-1.1.3.jar,
ring.conf.micros-1.0.4.201511210022_v120-84-g2a514b1-dirty.jar,
org.apache.httpcomponents.httpclient-4.3.6.jar,
osgi.cmpn-5.0.0.201305092017.jar, bcprov-1.53.jar,
com.fasterxml.jackson.core.jackson-databind-2.5.1.jar,
com.fasterxml.jackson.core.jackson-core-2.5.1.jar,
ring.menu.process-1.0.6.201511210022_v120-84-g2a514b1-dirty.jar,
org.apache.commons.io-2.4.0.jar, ch.qos.logback.classic-1.1.3.jar,
ring.conf.cache-1.0.6.201511210022_v120-84-g2a514b1-dirty.jar,
patronpath.wrap.pubnub-1.0.4.201511210011_v120-84-g2a514b1-dirty.jar,
ring.menu.extract.micros-2.0.2.201511210011_v120-84-g2a514b1-dirty.jar,
ring.util.order-1.0.5.201511210011_v120-84-g2a514b1-dirty.jar,
ring.log.fetch-1.0.5.201511210022_v120-84-g2a514b1-dirty.jar,
org.slf4j.osgi-over-slf4j-1.7.12.jar,
ring.order.process-1.1.6.201511210011_v120-84-g2a514b1-dirty.jar,
ring.util.json-1.1.4.201511210011_v120-84-g2a514b1-dirty.jar,
ring.conf.prod-3.0.0.201511210048_v120-84-g2a514b1-dirty.jar,
org.functionaljava-4.3.jar,
com.fasterxml.jackson.core.jackson-annotations-2.5.1.jar,
org.apache.felix.scr-2.0.0.jar, jcl.over.slf4j-1.7.12.jar,
ring.channel.pubnub-1.0.8.201511210022_v120-84-g2a514b1-dirty.jar,
ring.conf.boot-1.0.5.201511210022_v120-84-g2a514b1-dirty.jar] (463)!
I can't pin down whether the cause is the bundles themselves, or something
in the ace server/agent. The only things that have changed recently (as far
as I know) are upgrading to bndtools 3.0.0 and removing a set of
dependencies we no longer needed in our application.
Is it possible that bndtools is generating jars that are invalid based on
the agent's verification? Has anyone else seen errors like these?
Thanks,
Robert
Re: Flaky deployment
Posted by "Robert M. Mather" <ro...@gmail.com>.
Thanks for your response Marcel. Is there good documentation somewhere for
the gogo shell and/or the ace api the script using? I'm finding it
difficult to expand much on that initial script without lots of trial and
error. I'm assuming the gogo api is the best one to use for automated
deployment?
On Nov 24, 2015 12:52 PM, "Marcel Offermans" <ma...@luminis.nl>
wrote:
>
> Hello Robert,
>
> On 24 November 2015 at 13:31:02, Robert M. Mather (
robert.mather.rmm@gmail.com) wrote:
>>
>> I’ve narrowed the problem down to the Gogo script I’ve been using for
>> uploading bundles to the obr and linking them to a feature. I don’t know
>> what’s wrong with the script and/or that method, but when I switched
back
>> to doing it manually in the web UI, things started working normally
again.
>> I created the script starting from an example given in a presentation on
>> continuous deployment (https://www.youtube.com/watch?v=4S_zvgG_MLw).
>
> Conceptually, that script creates artifacts and links them all to a
feature. When manually linking artifacts you need to make sure that you end
up with only a single version of a bundle for each target. If you have more
than one version linked, that won’t deploy as the Deployment Admin
specfiication prohibits having more than one version in a deployment
package.
I've run into that before when doing manual linking to features, but it
gave a different error on the client, and I don't we're doing that, but I
can double check.
>
> It’s hard for me to see if that is indeed what happens when you run your
script, but it’s something worth looking out for.
>>
>> Here is the script, maybe there’s some mistake I’m missing…
>>
>> ```
>>
>> sourceIndex = (repo:index /my/local/runbundle/directory)
>>
>> sourceRepo = (repo:repo OBR $sourceIndex)
>>
>> targetRepo = (repo:repo OBR "http://ace.server.url/obr/repository.xml")
>>
>>
>> deployed = repo:cd $sourceRepo $sourceRepo $targetRepo
>>
>>
>> workspace = (ace:cw)
>>
>>
>> featureName = "myFeature|$(System:currentTimeMillis)"
>>
>>
>> $workspace cf $featureName
>
> Since you create a new feature every time you run the script, you might
end up with the situation described above if you link more than one feature
to a target.
Right now I'm basically just using features as full, immutable deployments
that I can roll back to easily. We don't really use features to compose our
application (in part because I've run into errors trying to do that before,
complaining about duplicate bundles in the deployment, but that's another
conversation).
Either way, I'm sure only one feature was deployed to the target in this
case, which also means only one version of each bundle in this case.
>
> If you’re running this in continuous deployment, I would consider adding
some code to the script that first cleans up at least the feature and all
associations (and possibly even the artifact definitions) before creating a
new one.
You mean if I were updating a feature in place? As I mentioned above, for
now we're treating features an immutable values. I'm just using the script
to automate our release process, not actual continuous integration or
anything. Based on the log output I see from the script, it already checks
for duplicate artifacts and I'm assuming it handles that properly...
>>
>> each $deployed {
>>
>> identity = $it getIdentity
>>
>> version = $it getVersion
>>
>> name = "$identity - $version"
>>
>> url = $it getUrl
>>
>> mimetype = $it getMimetype
>>
>>
>> if { (coll:first ($workspace la
>> "(&(Bundle-SymbolicName=$identity)(Bundle-Version=$version))")) } {
>>
>> echo "artifact $name already exists"
>>
>> } {
>>
>> $workspace ca [
>>
>> artifactName="$name"
>>
>> url="$url"
>>
>> mimetype="$mimetype"
>>
>> Bundle-SymbolicName="$identity"
>>
>> Bundle-Version="$version"
>>
>> ]
>>
>> }
>>
>>
>> $workspace ca2f
>> "(&(Bundle-SymbolicName=$identity)(Bundle-Version=$version))"
>> "(name=$featureName)"
>>
>> }
>>
>> $workspace commit
>>
>> exit 0
>
>
>
>
>>
>>
>> ```
>>
>>
>> On Sat, Nov 21, 2015 at 3:07 PM, Robert M. Mather <
>> robert.mather.rmm@gmail.com> wrote:
>>
>> > I'm trying to deploy a set of bundles that resolves and runs fine
inside
>> > bndtools, but leads to errors on the (Felix) client. So far, I've
seen:
>> >
>> > > 2015-11-21 11:39:50 | ERROR | ACE Agent Controller |
>> > org.apache.ace.agent.1.0.1 - Could not delete temporary deployment
package
>> > from disk
>> > > 2015-11-21 11:39:50 | ERROR | ACE Agent Controller |
>> > org.apache.ace.agent.1.0.1 - Installation of deployment update failed:
The
>> > InputStream is not a jar! java.io.IOException: Unknown/unexpected
status
>> > code: 500 at
>> >
org.apache.ace.agent.impl.ContentRangeInputStream.getHttpContentRangeInfo(ContentRangeInputStream.java:341)
>> > ~[na:na] at
>> >
org.apache.ace.agent.impl.ContentRangeInputStream.getContentRangeInfo(ContentRangeInputStream.java:288)
>> > ~[na:na] at
>> >
org.apache.ace.agent.impl.ContentRangeInputStream.prepareNextChunk(ContentRangeInputStream.java:418)
>> > ~[na:na] at
>> >
org.apache.ace.agent.impl.ContentRangeInputStream.read(ContentRangeInputStream.java:188)
>> > ~[na:na] at
>> >
org.apache.felix.deploymentadmin.OutputtingInputStream.read(OutputtingInputStream.java:64)
>> > ~[na:na] at java.io.FilterInputStream.read(FilterInputStream.java:133)
>> > ~[na:1.8.0_40] at
>> > java.io.PushbackInputStream.read(PushbackInputStream.java:186)
>> > ~[na:1.8.0_40] at
>> > java.util.zip.ZipInputStream.readFully(ZipInputStream.java:403)
>> > ~[na:1.8.0_40] at
>> > java.util.zip.ZipInputStream.readLOC(ZipInputStream.java:278)
>> > ~[na:1.8.0_40] at
>> > java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:122)
>> > ~[na:1.8.0_40] at
>> > java.util.jar.JarInputStream.<init>(JarInputStream.java:83)
~[na:1.8.0_40]
>> > at java.util.jar.JarInputStream.<init>(JarInputStream.java:62)
>> > ~[na:1.8.0_40] at
>> >
org.apache.felix.deploymentadmin.DeploymentAdminImpl.installDeploymentPackage(DeploymentAdminImpl.java:174)
>> > ~[na:na] at
>> >
org.apache.ace.agent.impl.DeploymentHandlerImpl.install(DeploymentHandlerImpl.java:237)
>> > ~[na:na] at
>> >
org.apache.ace.agent.impl.DefaultController$StreamingUpdateInstaller.doInstallUpdate(DefaultController.java:162)
>> > ~[na:na] at
>> >
org.apache.ace.agent.impl.DefaultController$UpdateInstaller.installUpdate(DefaultController.java:253)
>> > ~[na:na] at
>> >
org.apache.ace.agent.impl.DefaultController.runDeploymentUpdate(DefaultController.java:610)
>> > ~[na:na] at
>> >
org.apache.ace.agent.impl.DefaultController.run(DefaultController.java:460)
>> > ~[na:na] at
>> >
org.apache.ace.agent.impl.AgentContextImpl$1.run(AgentContextImpl.java:252)
>> > ~[na:na]
>> >
>> > and also
>> >
>> > > 2015-11-21 11:39:50 | ERROR | ACE Agent Controller |
>> > org.apache.ace.agent.1.0.1 - Stream does not contain a valid Jar
>> > java.io.IOException: Unknown/unexpected status code: 500 at
>> >
org.apache.ace.agent.impl.ContentRangeInputStream.getHttpContentRangeInfo(ContentRangeInputStream.java:341)
>> > ~[na:na] at
>> > (rest of trace same as above)
>> >
>> > and
>> >
>> > INFO|1428/0||15-11-21 05:22:51|[ERROR] 05:22:51 (controller)
Installation
>> > of deployment update failed: Expected more bundles in the stream:
>> > [com.google.guava-18.0.0.jar,
>> >
ring.order.inject.micros-1.0.6.201511210151_v120-84-g2a514b1-dirty.jar,
>> > ring.api.menu.upload-1.0.1.201511210011_v120-84-g2a514b1-dirty.jar,
>> > org.apache.httpcomponents.httpcore-4.3.3.jar,
>> > ring.ping-1.0.4.201511210011_v120-84-g2a514b1-dirty.jar,
>> > ring.menu.upload-1.0.2.201511210022_v120-84-g2a514b1-dirty.jar,
>> > ring.creditCard.decrypt-1.0.8.201511210022_v120-84-g2a514b1-dirty.jar,
>> > log4j.over.slf4j-1.7.12.jar,
>> > org.apache.servicemix.bundles.json-20140107.0.0.1.jar,
>> > ch.qos.logback.core-1.1.3.jar,
>> > ring.conf.micros-1.0.4.201511210022_v120-84-g2a514b1-dirty.jar,
>> > org.apache.httpcomponents.httpclient-4.3.6.jar,
>> > osgi.cmpn-5.0.0.201305092017.jar, bcprov-1.53.jar,
>> > com.fasterxml.jackson.core.jackson-databind-2.5.1.jar,
>> > com.fasterxml.jackson.core.jackson-core-2.5.1.jar,
>> > ring.menu.process-1.0.6.201511210022_v120-84-g2a514b1-dirty.jar,
>> > org.apache.commons.io-2.4.0.jar, ch.qos.logback.classic-1.1.3.jar,
>> > ring.conf.cache-1.0.6.201511210022_v120-84-g2a514b1-dirty.jar,
>> > patronpath.wrap.pubnub-1.0.4.201511210011_v120-84-g2a514b1-dirty.jar,
>> >
ring.menu.extract.micros-2.0.2.201511210011_v120-84-g2a514b1-dirty.jar,
>> > ring.util.order-1.0.5.201511210011_v120-84-g2a514b1-dirty.jar,
>> > ring.log.fetch-1.0.5.201511210022_v120-84-g2a514b1-dirty.jar,
>> > org.slf4j.osgi-over-slf4j-1.7.12.jar,
>> > ring.order.process-1.1.6.201511210011_v120-84-g2a514b1-dirty.jar,
>> > ring.util.json-1.1.4.201511210011_v120-84-g2a514b1-dirty.jar,
>> > ring.conf.prod-3.0.0.201511210048_v120-84-g2a514b1-dirty.jar,
>> > org.functionaljava-4.3.jar,
>> > com.fasterxml.jackson.core.jackson-annotations-2.5.1.jar,
>> > org.apache.felix.scr-2.0.0.jar, jcl.over.slf4j-1.7.12.jar,
>> > ring.channel.pubnub-1.0.8.201511210022_v120-84-g2a514b1-dirty.jar,
>> > ring.conf.boot-1.0.5.201511210022_v120-84-g2a514b1-dirty.jar] (463)!
>
> To be honest, so far I don’t see any “duplicate” bundles in this list.
Then again, the fact that some bundle in the stream appears to be “missing”
does mean something is wrong. Deployment packages get created on the fly in
ACE so when something goes wrong during that creation, that is usually only
noticed when a target asks for such a deployment package.
>>
>> > I can't pin down whether the cause is the bundles themselves, or
something
>> > in the ace server/agent. The only things that have changed recently
(as far
>> > as I know) are upgrading to bndtools 3.0.0 and removing a set of
>> > dependencies we no longer needed in our application.
>
> In theory you should also check to make sure if the MANIFEST.MF file is
indeed the first entry in the bundle. According to the spec, it should, and
most tools do respect that, but I’ve seen “bundles” in the past that were
either hand-edited, or were processed with some more generic jar/zip tools
that did not respect that.
>>
>> > Is it possible that bndtools is generating jars that are invalid based
on
>> > the agent's verification? Has anyone else seen errors like these?
>
> So far we have never seen bnd(tools) generate “invalid” bundles, but I
would definitely also check my dependencies.
>
> Ending up with more than one version of a bundle for a target would be my
next guess as explained above.
>
> If neither of those are the cause, the next step is probably to somehow
create a scenario that I can reproduce, or to step through the code with a
debugger to get a better feel for what goes wrong exactly.
>
> In any case, let us know if you need more help!
>
> Greetings, Marcel
>
>
Re: Flaky deployment
Posted by Marcel Offermans <ma...@luminis.nl>.
Hello Robert,
On 24 November 2015 at 13:31:02, Robert M. Mather (robert.mather.rmm@gmail.com) wrote:
I’ve narrowed the problem down to the Gogo script I’ve been using for
uploading bundles to the obr and linking them to a feature. I don’t know
what’s wrong with the script and/or that method, but when I switched back
to doing it manually in the web UI, things started working normally again.
I created the script starting from an example given in a presentation on
continuous deployment (https://www.youtube.com/watch?v=4S_zvgG_MLw).
Conceptually, that script creates artifacts and links them all to a feature. When manually linking artifacts you need to make sure that you end up with only a single version of a bundle for each target. If you have more than one version linked, that won’t deploy as the Deployment Admin specfiication prohibits having more than one version in a deployment package.
It’s hard for me to see if that is indeed what happens when you run your script, but it’s something worth looking out for.
Here is the script, maybe there’s some mistake I’m missing…
```
sourceIndex = (repo:index /my/local/runbundle/directory)
sourceRepo = (repo:repo OBR $sourceIndex)
targetRepo = (repo:repo OBR "http://ace.server.url/obr/repository.xml")
deployed = repo:cd $sourceRepo $sourceRepo $targetRepo
workspace = (ace:cw)
featureName = "myFeature|$(System:currentTimeMillis)"
$workspace cf $featureName
Since you create a new feature every time you run the script, you might end up with the situation described above if you link more than one feature to a target.
If you’re running this in continuous deployment, I would consider adding some code to the script that first cleans up at least the feature and all associations (and possibly even the artifact definitions) before creating a new one.
each $deployed {
identity = $it getIdentity
version = $it getVersion
name = "$identity - $version"
url = $it getUrl
mimetype = $it getMimetype
if { (coll:first ($workspace la
"(&(Bundle-SymbolicName=$identity)(Bundle-Version=$version))")) } {
echo "artifact $name already exists"
} {
$workspace ca [
artifactName="$name"
url="$url"
mimetype="$mimetype"
Bundle-SymbolicName="$identity"
Bundle-Version="$version"
]
}
$workspace ca2f
"(&(Bundle-SymbolicName=$identity)(Bundle-Version=$version))"
"(name=$featureName)"
}
$workspace commit
exit 0
```
On Sat, Nov 21, 2015 at 3:07 PM, Robert M. Mather <
robert.mather.rmm@gmail.com> wrote:
> I'm trying to deploy a set of bundles that resolves and runs fine inside
> bndtools, but leads to errors on the (Felix) client. So far, I've seen:
>
> > 2015-11-21 11:39:50 | ERROR | ACE Agent Controller |
> org.apache.ace.agent.1.0.1 - Could not delete temporary deployment package
> from disk
> > 2015-11-21 11:39:50 | ERROR | ACE Agent Controller |
> org.apache.ace.agent.1.0.1 - Installation of deployment update failed: The
> InputStream is not a jar! java.io.IOException: Unknown/unexpected status
> code: 500 at
> org.apache.ace.agent.impl.ContentRangeInputStream.getHttpContentRangeInfo(ContentRangeInputStream.java:341)
> ~[na:na] at
> org.apache.ace.agent.impl.ContentRangeInputStream.getContentRangeInfo(ContentRangeInputStream.java:288)
> ~[na:na] at
> org.apache.ace.agent.impl.ContentRangeInputStream.prepareNextChunk(ContentRangeInputStream.java:418)
> ~[na:na] at
> org.apache.ace.agent.impl.ContentRangeInputStream.read(ContentRangeInputStream.java:188)
> ~[na:na] at
> org.apache.felix.deploymentadmin.OutputtingInputStream.read(OutputtingInputStream.java:64)
> ~[na:na] at java.io.FilterInputStream.read(FilterInputStream.java:133)
> ~[na:1.8.0_40] at
> java.io.PushbackInputStream.read(PushbackInputStream.java:186)
> ~[na:1.8.0_40] at
> java.util.zip.ZipInputStream.readFully(ZipInputStream.java:403)
> ~[na:1.8.0_40] at
> java.util.zip.ZipInputStream.readLOC(ZipInputStream.java:278)
> ~[na:1.8.0_40] at
> java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:122)
> ~[na:1.8.0_40] at
> java.util.jar.JarInputStream.<init>(JarInputStream.java:83) ~[na:1.8.0_40]
> at java.util.jar.JarInputStream.<init>(JarInputStream.java:62)
> ~[na:1.8.0_40] at
> org.apache.felix.deploymentadmin.DeploymentAdminImpl.installDeploymentPackage(DeploymentAdminImpl.java:174)
> ~[na:na] at
> org.apache.ace.agent.impl.DeploymentHandlerImpl.install(DeploymentHandlerImpl.java:237)
> ~[na:na] at
> org.apache.ace.agent.impl.DefaultController$StreamingUpdateInstaller.doInstallUpdate(DefaultController.java:162)
> ~[na:na] at
> org.apache.ace.agent.impl.DefaultController$UpdateInstaller.installUpdate(DefaultController.java:253)
> ~[na:na] at
> org.apache.ace.agent.impl.DefaultController.runDeploymentUpdate(DefaultController.java:610)
> ~[na:na] at
> org.apache.ace.agent.impl.DefaultController.run(DefaultController.java:460)
> ~[na:na] at
> org.apache.ace.agent.impl.AgentContextImpl$1.run(AgentContextImpl.java:252)
> ~[na:na]
>
> and also
>
> > 2015-11-21 11:39:50 | ERROR | ACE Agent Controller |
> org.apache.ace.agent.1.0.1 - Stream does not contain a valid Jar
> java.io.IOException: Unknown/unexpected status code: 500 at
> org.apache.ace.agent.impl.ContentRangeInputStream.getHttpContentRangeInfo(ContentRangeInputStream.java:341)
> ~[na:na] at
> (rest of trace same as above)
>
> and
>
> INFO|1428/0||15-11-21 05:22:51|[ERROR] 05:22:51 (controller) Installation
> of deployment update failed: Expected more bundles in the stream:
> [com.google.guava-18.0.0.jar,
> ring.order.inject.micros-1.0.6.201511210151_v120-84-g2a514b1-dirty.jar,
> ring.api.menu.upload-1.0.1.201511210011_v120-84-g2a514b1-dirty.jar,
> org.apache.httpcomponents.httpcore-4.3.3.jar,
> ring.ping-1.0.4.201511210011_v120-84-g2a514b1-dirty.jar,
> ring.menu.upload-1.0.2.201511210022_v120-84-g2a514b1-dirty.jar,
> ring.creditCard.decrypt-1.0.8.201511210022_v120-84-g2a514b1-dirty.jar,
> log4j.over.slf4j-1.7.12.jar,
> org.apache.servicemix.bundles.json-20140107.0.0.1.jar,
> ch.qos.logback.core-1.1.3.jar,
> ring.conf.micros-1.0.4.201511210022_v120-84-g2a514b1-dirty.jar,
> org.apache.httpcomponents.httpclient-4.3.6.jar,
> osgi.cmpn-5.0.0.201305092017.jar, bcprov-1.53.jar,
> com.fasterxml.jackson.core.jackson-databind-2.5.1.jar,
> com.fasterxml.jackson.core.jackson-core-2.5.1.jar,
> ring.menu.process-1.0.6.201511210022_v120-84-g2a514b1-dirty.jar,
> org.apache.commons.io-2.4.0.jar, ch.qos.logback.classic-1.1.3.jar,
> ring.conf.cache-1.0.6.201511210022_v120-84-g2a514b1-dirty.jar,
> patronpath.wrap.pubnub-1.0.4.201511210011_v120-84-g2a514b1-dirty.jar,
> ring.menu.extract.micros-2.0.2.201511210011_v120-84-g2a514b1-dirty.jar,
> ring.util.order-1.0.5.201511210011_v120-84-g2a514b1-dirty.jar,
> ring.log.fetch-1.0.5.201511210022_v120-84-g2a514b1-dirty.jar,
> org.slf4j.osgi-over-slf4j-1.7.12.jar,
> ring.order.process-1.1.6.201511210011_v120-84-g2a514b1-dirty.jar,
> ring.util.json-1.1.4.201511210011_v120-84-g2a514b1-dirty.jar,
> ring.conf.prod-3.0.0.201511210048_v120-84-g2a514b1-dirty.jar,
> org.functionaljava-4.3.jar,
> com.fasterxml.jackson.core.jackson-annotations-2.5.1.jar,
> org.apache.felix.scr-2.0.0.jar, jcl.over.slf4j-1.7.12.jar,
> ring.channel.pubnub-1.0.8.201511210022_v120-84-g2a514b1-dirty.jar,
> ring.conf.boot-1.0.5.201511210022_v120-84-g2a514b1-dirty.jar] (463)!
To be honest, so far I don’t see any “duplicate” bundles in this list. Then again, the fact that some bundle in the stream appears to be “missing” does mean something is wrong. Deployment packages get created on the fly in ACE so when something goes wrong during that creation, that is usually only noticed when a target asks for such a deployment package.
> I can't pin down whether the cause is the bundles themselves, or something
> in the ace server/agent. The only things that have changed recently (as far
> as I know) are upgrading to bndtools 3.0.0 and removing a set of
> dependencies we no longer needed in our application.
In theory you should also check to make sure if the MANIFEST.MF file is indeed the first entry in the bundle. According to the spec, it should, and most tools do respect that, but I’ve seen “bundles” in the past that were either hand-edited, or were processed with some more generic jar/zip tools that did not respect that.
> Is it possible that bndtools is generating jars that are invalid based on
> the agent's verification? Has anyone else seen errors like these?
So far we have never seen bnd(tools) generate “invalid” bundles, but I would definitely also check my dependencies.
Ending up with more than one version of a bundle for a target would be my next guess as explained above.
If neither of those are the cause, the next step is probably to somehow create a scenario that I can reproduce, or to step through the code with a debugger to get a better feel for what goes wrong exactly.
In any case, let us know if you need more help!
Greetings, Marcel
Re: Flaky deployment
Posted by "Robert M. Mather" <ro...@gmail.com>.
I’ve narrowed the problem down to the Gogo script I’ve been using for
uploading bundles to the obr and linking them to a feature. I don’t know
what’s wrong with the script and/or that method, but when I switched back
to doing it manually in the web UI, things started working normally again.
I created the script starting from an example given in a presentation on
continuous deployment (https://www.youtube.com/watch?v=4S_zvgG_MLw).
Here is the script, maybe there’s some mistake I’m missing…
```
sourceIndex = (repo:index /my/local/runbundle/directory)
sourceRepo = (repo:repo OBR $sourceIndex)
targetRepo = (repo:repo OBR "http://ace.server.url/obr/repository.xml")
deployed = repo:cd $sourceRepo $sourceRepo $targetRepo
workspace = (ace:cw)
featureName = "myFeature|$(System:currentTimeMillis)"
$workspace cf $featureName
each $deployed {
identity = $it getIdentity
version = $it getVersion
name = "$identity - $version"
url = $it getUrl
mimetype = $it getMimetype
if { (coll:first ($workspace la
"(&(Bundle-SymbolicName=$identity)(Bundle-Version=$version))")) } {
echo "artifact $name already exists"
} {
$workspace ca [
artifactName="$name"
url="$url"
mimetype="$mimetype"
Bundle-SymbolicName="$identity"
Bundle-Version="$version"
]
}
$workspace ca2f
"(&(Bundle-SymbolicName=$identity)(Bundle-Version=$version))"
"(name=$featureName)"
}
$workspace commit
exit 0
```
On Sat, Nov 21, 2015 at 3:07 PM, Robert M. Mather <
robert.mather.rmm@gmail.com> wrote:
> I'm trying to deploy a set of bundles that resolves and runs fine inside
> bndtools, but leads to errors on the (Felix) client. So far, I've seen:
>
> > 2015-11-21 11:39:50 | ERROR | ACE Agent Controller |
> org.apache.ace.agent.1.0.1 - Could not delete temporary deployment package
> from disk
> > 2015-11-21 11:39:50 | ERROR | ACE Agent Controller |
> org.apache.ace.agent.1.0.1 - Installation of deployment update failed: The
> InputStream is not a jar! java.io.IOException: Unknown/unexpected status
> code: 500 at
> org.apache.ace.agent.impl.ContentRangeInputStream.getHttpContentRangeInfo(ContentRangeInputStream.java:341)
> ~[na:na] at
> org.apache.ace.agent.impl.ContentRangeInputStream.getContentRangeInfo(ContentRangeInputStream.java:288)
> ~[na:na] at
> org.apache.ace.agent.impl.ContentRangeInputStream.prepareNextChunk(ContentRangeInputStream.java:418)
> ~[na:na] at
> org.apache.ace.agent.impl.ContentRangeInputStream.read(ContentRangeInputStream.java:188)
> ~[na:na] at
> org.apache.felix.deploymentadmin.OutputtingInputStream.read(OutputtingInputStream.java:64)
> ~[na:na] at java.io.FilterInputStream.read(FilterInputStream.java:133)
> ~[na:1.8.0_40] at
> java.io.PushbackInputStream.read(PushbackInputStream.java:186)
> ~[na:1.8.0_40] at
> java.util.zip.ZipInputStream.readFully(ZipInputStream.java:403)
> ~[na:1.8.0_40] at
> java.util.zip.ZipInputStream.readLOC(ZipInputStream.java:278)
> ~[na:1.8.0_40] at
> java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:122)
> ~[na:1.8.0_40] at
> java.util.jar.JarInputStream.<init>(JarInputStream.java:83) ~[na:1.8.0_40]
> at java.util.jar.JarInputStream.<init>(JarInputStream.java:62)
> ~[na:1.8.0_40] at
> org.apache.felix.deploymentadmin.DeploymentAdminImpl.installDeploymentPackage(DeploymentAdminImpl.java:174)
> ~[na:na] at
> org.apache.ace.agent.impl.DeploymentHandlerImpl.install(DeploymentHandlerImpl.java:237)
> ~[na:na] at
> org.apache.ace.agent.impl.DefaultController$StreamingUpdateInstaller.doInstallUpdate(DefaultController.java:162)
> ~[na:na] at
> org.apache.ace.agent.impl.DefaultController$UpdateInstaller.installUpdate(DefaultController.java:253)
> ~[na:na] at
> org.apache.ace.agent.impl.DefaultController.runDeploymentUpdate(DefaultController.java:610)
> ~[na:na] at
> org.apache.ace.agent.impl.DefaultController.run(DefaultController.java:460)
> ~[na:na] at
> org.apache.ace.agent.impl.AgentContextImpl$1.run(AgentContextImpl.java:252)
> ~[na:na]
>
> and also
>
> > 2015-11-21 11:39:50 | ERROR | ACE Agent Controller |
> org.apache.ace.agent.1.0.1 - Stream does not contain a valid Jar
> java.io.IOException: Unknown/unexpected status code: 500 at
> org.apache.ace.agent.impl.ContentRangeInputStream.getHttpContentRangeInfo(ContentRangeInputStream.java:341)
> ~[na:na] at
> (rest of trace same as above)
>
> and
>
> INFO|1428/0||15-11-21 05:22:51|[ERROR] 05:22:51 (controller) Installation
> of deployment update failed: Expected more bundles in the stream:
> [com.google.guava-18.0.0.jar,
> ring.order.inject.micros-1.0.6.201511210151_v120-84-g2a514b1-dirty.jar,
> ring.api.menu.upload-1.0.1.201511210011_v120-84-g2a514b1-dirty.jar,
> org.apache.httpcomponents.httpcore-4.3.3.jar,
> ring.ping-1.0.4.201511210011_v120-84-g2a514b1-dirty.jar,
> ring.menu.upload-1.0.2.201511210022_v120-84-g2a514b1-dirty.jar,
> ring.creditCard.decrypt-1.0.8.201511210022_v120-84-g2a514b1-dirty.jar,
> log4j.over.slf4j-1.7.12.jar,
> org.apache.servicemix.bundles.json-20140107.0.0.1.jar,
> ch.qos.logback.core-1.1.3.jar,
> ring.conf.micros-1.0.4.201511210022_v120-84-g2a514b1-dirty.jar,
> org.apache.httpcomponents.httpclient-4.3.6.jar,
> osgi.cmpn-5.0.0.201305092017.jar, bcprov-1.53.jar,
> com.fasterxml.jackson.core.jackson-databind-2.5.1.jar,
> com.fasterxml.jackson.core.jackson-core-2.5.1.jar,
> ring.menu.process-1.0.6.201511210022_v120-84-g2a514b1-dirty.jar,
> org.apache.commons.io-2.4.0.jar, ch.qos.logback.classic-1.1.3.jar,
> ring.conf.cache-1.0.6.201511210022_v120-84-g2a514b1-dirty.jar,
> patronpath.wrap.pubnub-1.0.4.201511210011_v120-84-g2a514b1-dirty.jar,
> ring.menu.extract.micros-2.0.2.201511210011_v120-84-g2a514b1-dirty.jar,
> ring.util.order-1.0.5.201511210011_v120-84-g2a514b1-dirty.jar,
> ring.log.fetch-1.0.5.201511210022_v120-84-g2a514b1-dirty.jar,
> org.slf4j.osgi-over-slf4j-1.7.12.jar,
> ring.order.process-1.1.6.201511210011_v120-84-g2a514b1-dirty.jar,
> ring.util.json-1.1.4.201511210011_v120-84-g2a514b1-dirty.jar,
> ring.conf.prod-3.0.0.201511210048_v120-84-g2a514b1-dirty.jar,
> org.functionaljava-4.3.jar,
> com.fasterxml.jackson.core.jackson-annotations-2.5.1.jar,
> org.apache.felix.scr-2.0.0.jar, jcl.over.slf4j-1.7.12.jar,
> ring.channel.pubnub-1.0.8.201511210022_v120-84-g2a514b1-dirty.jar,
> ring.conf.boot-1.0.5.201511210022_v120-84-g2a514b1-dirty.jar] (463)!
>
> I can't pin down whether the cause is the bundles themselves, or something
> in the ace server/agent. The only things that have changed recently (as far
> as I know) are upgrading to bndtools 3.0.0 and removing a set of
> dependencies we no longer needed in our application.
>
> Is it possible that bndtools is generating jars that are invalid based on
> the agent's verification? Has anyone else seen errors like these?
>
> Thanks,
>
> Robert
>
>