You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by Marshall Schor <ms...@schor.com> on 2016/01/25 21:33:22 UTC
signing Jars
I only got to the point of setting up a way to log into the symantec site, where
I can apparently add other users, and do some other things.
To integrating this into our Apache UIMA build, I think the following needs to
be figured out:
1) what Jars to sign. The only Jars we need signed in this way I think are the
Eclipse Plugins (and maybe the Feature Jars - but I would be surprised...).
2) The build process for the plugins is complicated by the fact that both
"packed" and "regular" Jars are built. The signing has to happen after all of
that has finished, I think. I remember a long time ago reading something on the
Eclipse websites about signing Eclipse things - that might be a place to start.
3) There was an ant task done - by some other project - but I haven't looked at
it. The mail archives has some pointers (I think I may have copied info about
this into a previous email). We probably should start a UIMA website page to
collect all this info...
I'll do some digging to see if I can add Peter as a signer and send him the
credentials. The protocol used by Apache Infra was to send this info encrypted,
so I'll do that.
-Marshall
Re: signing Jars
Posted by Marshall Schor <ms...@schor.com>.
I think code signing should at this point be used just for Eclipse plugin Jars.
UIMA-AS has one Eclipse plugin jar which might use this.
-Marshall
On 1/26/2016 5:09 AM, Peter Klügl wrote:
> I'll search for some answers, too
>
> I put the rc3 on hold without a vote mail for now. Do you consider code
> signing also for uima-as?
>
> Best,
>
> Peter
>
> Am 25.01.2016 um 21:33 schrieb Marshall Schor:
>> I only got to the point of setting up a way to log into the symantec site, where
>> I can apparently add other users, and do some other things.
>>
>> To integrating this into our Apache UIMA build, I think the following needs to
>> be figured out:
>>
>> 1) what Jars to sign. The only Jars we need signed in this way I think are the
>> Eclipse Plugins (and maybe the Feature Jars - but I would be surprised...).
>>
>> 2) The build process for the plugins is complicated by the fact that both
>> "packed" and "regular" Jars are built. The signing has to happen after all of
>> that has finished, I think. I remember a long time ago reading something on the
>> Eclipse websites about signing Eclipse things - that might be a place to start.
>>
>> 3) There was an ant task done - by some other project - but I haven't looked at
>> it. The mail archives has some pointers (I think I may have copied info about
>> this into a previous email). We probably should start a UIMA website page to
>> collect all this info...
>>
>> I'll do some digging to see if I can add Peter as a signer and send him the
>> credentials. The protocol used by Apache Infra was to send this info encrypted,
>> so I'll do that.
>>
>> -Marshall
>
Re: signing Jars
Posted by Peter Klügl <pe...@averbis.com>.
I'll search for some answers, too
I put the rc3 on hold without a vote mail for now. Do you consider code
signing also for uima-as?
Best,
Peter
Am 25.01.2016 um 21:33 schrieb Marshall Schor:
> I only got to the point of setting up a way to log into the symantec site, where
> I can apparently add other users, and do some other things.
>
> To integrating this into our Apache UIMA build, I think the following needs to
> be figured out:
>
> 1) what Jars to sign. The only Jars we need signed in this way I think are the
> Eclipse Plugins (and maybe the Feature Jars - but I would be surprised...).
>
> 2) The build process for the plugins is complicated by the fact that both
> "packed" and "regular" Jars are built. The signing has to happen after all of
> that has finished, I think. I remember a long time ago reading something on the
> Eclipse websites about signing Eclipse things - that might be a place to start.
>
> 3) There was an ant task done - by some other project - but I haven't looked at
> it. The mail archives has some pointers (I think I may have copied info about
> this into a previous email). We probably should start a UIMA website page to
> collect all this info...
>
> I'll do some digging to see if I can add Peter as a signer and send him the
> credentials. The protocol used by Apache Infra was to send this info encrypted,
> so I'll do that.
>
> -Marshall
Re: signing Jars
Posted by Peter Klügl <pe...@averbis.com>.
Yes.
.. and thanks for writing to Mark. I will try to find the test option.
Best,
Peter
Am 27.01.2016 um 21:07 schrieb Marshall Schor:
> not in my browser - it forwards the https req to a related site but then says
> Secure connection failed.
>
> Is this what happened with you?
>
> -M
>
> On 1/27/2016 4:24 AM, Peter Klügl wrote:
>> Marshall, are you able to access the test system?
>>
>> https://test-securesigning.websecurity.symantec.com/csportal/
>>
>>
>> Am 27.01.2016 um 10:17 schrieb Peter Klügl:
>>> Am 26.01.2016 um 22:09 schrieb Marshall Schor:
>>>> On 1/26/2016 5:20 AM, Peter Klügl wrote:
>>>>> ...
>>>>> ect all this info...
>>>>> Should we just copy the implementation to some of our (build) projects?
>>>> It would be good to automate this of course. I think that would take using the
>>>> SOAP interface with the Ant script done by the Apache TomCat project.
>>>> However, some basic info on our web site for posterity once we forget how this
>>>> all works would be good - pointers to other pages that describe how it all
>>>> works, how to use the test stuff, etc.
>>>> -Marshall
>>>>
>>> Yes, I will create/extend a page once it is up and running.
>>>
>>> Best,
>>>
>>> Peter
>>>
Re: signing Jars
Posted by Peter Klügl <pe...@averbis.com>.
Nope, I did not find the test service on the production URL.
The help says:
"Depending on your account setup, you may be able to choose between Test
and Production signing.
- Test signing. This signing is useful for validation and testing.
Not every account allows test signing. Test signing is useful for the
accounts that use signing credits (units) because test signing does not
deduct units.
- Production signing. After production signing you can publish or
distribute the signed software.
If your account is set up to use units, each production signing
deducts one unit."
I did not find an option to select/change my account setup. I also see
no option when I choose to add new users.
Best,
Peter
Am 28.01.2016 um 18:00 schrieb Marshall Schor:
> The test signing service, using a different URL, needs a separate certificate,
> which the main ASF contact person has to issue. He is currently unable to do
> that due to other issues, but suggests as a workaround, to use the production
> URL, but there selecting the "test" mode.
>
> Are you able to access the test signing service that way?
>
> -Marshall
>
> On 1/28/2016 4:42 AM, Peter Klügl wrote:
>> Ok, I found out that I am not able to use the test signing service but
>> only the prduction signing service.
>>
>> Marshall, do you have an idea how I can extend my role in order to
>> access the test signing service? Did you see an option when you added me?
>>
>> Best,
>>
>> Peter
>>
>> Am 27.01.2016 um 21:07 schrieb Marshall Schor:
>>> not in my browser - it forwards the https req to a related site but then says
>>> Secure connection failed.
>>>
>>> Is this what happened with you?
>>>
>>> -M
>>>
>>> On 1/27/2016 4:24 AM, Peter Klügl wrote:
>>>> Marshall, are you able to access the test system?
>>>>
>>>> https://test-securesigning.websecurity.symantec.com/csportal/
>>>>
>>>>
>>>> Am 27.01.2016 um 10:17 schrieb Peter Klügl:
>>>>> Am 26.01.2016 um 22:09 schrieb Marshall Schor:
>>>>>> On 1/26/2016 5:20 AM, Peter Klügl wrote:
>>>>>>> ...
>>>>>>> ect all this info...
>>>>>>> Should we just copy the implementation to some of our (build) projects?
>>>>>> It would be good to automate this of course. I think that would take using the
>>>>>> SOAP interface with the Ant script done by the Apache TomCat project.
>>>>>> However, some basic info on our web site for posterity once we forget how this
>>>>>> all works would be good - pointers to other pages that describe how it all
>>>>>> works, how to use the test stuff, etc.
>>>>>> -Marshall
>>>>>>
>>>>> Yes, I will create/extend a page once it is up and running.
>>>>>
>>>>> Best,
>>>>>
>>>>> Peter
>>>>>
Re: signing Jars
Posted by Marshall Schor <ms...@schor.com>.
The test signing service, using a different URL, needs a separate certificate,
which the main ASF contact person has to issue. He is currently unable to do
that due to other issues, but suggests as a workaround, to use the production
URL, but there selecting the "test" mode.
Are you able to access the test signing service that way?
-Marshall
On 1/28/2016 4:42 AM, Peter Klügl wrote:
> Ok, I found out that I am not able to use the test signing service but
> only the prduction signing service.
>
> Marshall, do you have an idea how I can extend my role in order to
> access the test signing service? Did you see an option when you added me?
>
> Best,
>
> Peter
>
> Am 27.01.2016 um 21:07 schrieb Marshall Schor:
>> not in my browser - it forwards the https req to a related site but then says
>> Secure connection failed.
>>
>> Is this what happened with you?
>>
>> -M
>>
>> On 1/27/2016 4:24 AM, Peter Klügl wrote:
>>> Marshall, are you able to access the test system?
>>>
>>> https://test-securesigning.websecurity.symantec.com/csportal/
>>>
>>>
>>> Am 27.01.2016 um 10:17 schrieb Peter Klügl:
>>>> Am 26.01.2016 um 22:09 schrieb Marshall Schor:
>>>>> On 1/26/2016 5:20 AM, Peter Klügl wrote:
>>>>>> ...
>>>>>> ect all this info...
>>>>>> Should we just copy the implementation to some of our (build) projects?
>>>>> It would be good to automate this of course. I think that would take using the
>>>>> SOAP interface with the Ant script done by the Apache TomCat project.
>>>>> However, some basic info on our web site for posterity once we forget how this
>>>>> all works would be good - pointers to other pages that describe how it all
>>>>> works, how to use the test stuff, etc.
>>>>> -Marshall
>>>>>
>>>> Yes, I will create/extend a page once it is up and running.
>>>>
>>>> Best,
>>>>
>>>> Peter
>>>>
>
Re: signing Jars
Posted by Peter Klügl <pe...@averbis.com>.
Ok, I found out that I am not able to use the test signing service but
only the prduction signing service.
Marshall, do you have an idea how I can extend my role in order to
access the test signing service? Did you see an option when you added me?
Best,
Peter
Am 27.01.2016 um 21:07 schrieb Marshall Schor:
> not in my browser - it forwards the https req to a related site but then says
> Secure connection failed.
>
> Is this what happened with you?
>
> -M
>
> On 1/27/2016 4:24 AM, Peter Klügl wrote:
>> Marshall, are you able to access the test system?
>>
>> https://test-securesigning.websecurity.symantec.com/csportal/
>>
>>
>> Am 27.01.2016 um 10:17 schrieb Peter Klügl:
>>> Am 26.01.2016 um 22:09 schrieb Marshall Schor:
>>>> On 1/26/2016 5:20 AM, Peter Klügl wrote:
>>>>> ...
>>>>> ect all this info...
>>>>> Should we just copy the implementation to some of our (build) projects?
>>>> It would be good to automate this of course. I think that would take using the
>>>> SOAP interface with the Ant script done by the Apache TomCat project.
>>>> However, some basic info on our web site for posterity once we forget how this
>>>> all works would be good - pointers to other pages that describe how it all
>>>> works, how to use the test stuff, etc.
>>>> -Marshall
>>>>
>>> Yes, I will create/extend a page once it is up and running.
>>>
>>> Best,
>>>
>>> Peter
>>>
Re: signing Jars
Posted by Marshall Schor <ms...@schor.com>.
not in my browser - it forwards the https req to a related site but then says
Secure connection failed.
Is this what happened with you?
-M
On 1/27/2016 4:24 AM, Peter Klügl wrote:
> Marshall, are you able to access the test system?
>
> https://test-securesigning.websecurity.symantec.com/csportal/
>
>
> Am 27.01.2016 um 10:17 schrieb Peter Klügl:
>> Am 26.01.2016 um 22:09 schrieb Marshall Schor:
>>> On 1/26/2016 5:20 AM, Peter Klügl wrote:
>>>> ...
>>>> ect all this info...
>>>> Should we just copy the implementation to some of our (build) projects?
>>> It would be good to automate this of course. I think that would take using the
>>> SOAP interface with the Ant script done by the Apache TomCat project.
>>> However, some basic info on our web site for posterity once we forget how this
>>> all works would be good - pointers to other pages that describe how it all
>>> works, how to use the test stuff, etc.
>>> -Marshall
>>>
>> Yes, I will create/extend a page once it is up and running.
>>
>> Best,
>>
>> Peter
>>
>
Re: signing Jars
Posted by Peter Klügl <pe...@averbis.com>.
Am 29.01.2016 um 11:02 schrieb Peter Klügl:
> Hi,
>
> ...
>
> Re: artifacts.jar and content.jar
> I forgot about both. I do not know if the need to be signed, but I would
> assume it.
>
>
When installing the feature, Eclipse complains only about the feature
and the plugins. Could be that we do not need to sign the repo jars. I
will test it when we have the code signing up and running.
Best,
Peter
Re: signing Jars
Posted by Peter Klügl <pe...@averbis.com>.
Hi,
Re: repack earlier
Ah ok, the repack changes also the content of lined jars, right?
But this step also creates the pack.gz files. I think that the code
signing has to happen before that. I have to check this.
Re: artifacts.jar and content.jar
I forgot about both. I do not know if the need to be signed, but I would
assume it.
Re: avoid rebuilding
Sounds to me like a separate ant build, which is maybe a better choice
than to configure the maven build.
Re: artifactory
I really should stop to use this term synonym for maven central, nexus
and all. Yes, was thinking about maven central and the staging repository.
Re: same folder
In the ruta update site build, toBePacked contains only the new plugins.
The folder eus-work contains all features and plugins. Well, it does not
really matter since we would need to change something anyway and then we
need to take care that the folders and their content are corrrect.
Re: hold up rc3
I could really need the 2.4.0 release right now for different reasons.
It blocks some other stuff which I need to get done before mid of Feb.
I'll start the vote today without code signing.
Best,
Peter
Am 28.01.2016 um 18:17 schrieb Marshall Schor:
>
> On 1/28/2016 3:56 AM, Peter Klügl wrote:
>> Hi
>>
>> you are of course correct. It's even maybe a bit more problematic.
>>
>> Let's distinguish the two signatures:
>> - the gpg signature in a separate file together with the checksum (I
>> call it signature)
>> - the code signing signature in the META-INF within the jar (I call it
>> code signing)
>>
>> This means that the signature has to happen after the code signing since
>> the code signing signature changes the jar (independently of the
>> repacking). I think the features need also code signing but no repackaging.
> Sounds right to me.
>> So the rough workflow building the update site would look like:
>> - (optional: isApacheRelease) checkout the previous update site
>> - collect the new features and plugins
> I think you need to do any pack/repacking here of the jars, because that changes
> the Jars' data, which the code signing is based on.
>> - (optional: isCodeSigning) call the code signing service for the new features and plugins
> The repack has to be done earlier
>> - repack new plugins
>> - generate update site and p2 info
> This generates "artifacts.jar" and "content.jar". My guess is that these might
> need to be code-signed, too?
>> - (optional: isApacheRelease) sign (sha1, md5, asc) for new features and
>> plugin and their packed versions
>> - (optional: isApacheRelease) sign (sha1, md5, asc) the modified update
>> site info
>>
>> There is of course a lot of copying in between.
>>
>> If we want to avoid the code signing for every release candidate, then
>> we maybe need another stage in our release process:
>> - prepare rc (without code signing)
>> - test and vote as usual
>> - if successful: rebuild the update site with code signing with the same
>> artifacts
> Probably better to avoid "rebuilding", and just take the unsigned artifacts from
> the RC site and code-sign them, and regenerate the signatures, too.
>> - test and vote code signing and signatures of the update site
> If the post RC vote process is only to add the code-signing and redo the
> signatures, I wonder if there can be an automatic validation that compares these
> with the RC site (the Jars should be the same other than the added code-signing,
> the .asd/md5/sha1 signatures should check as OK). If we have that, then as long
> as the release manager runs the check, no additional voting would be necessary.
>
> If the RC artifacts were put into the dist.apache.org/repos/dist/dev/uima spot,
> it would be verifyable in the future, if need arises.
>> - if successful: release
>>
>> (Another question is: do we need code signing for the jars in the
>> artifactory? I would say no.)
> Not sure what the "artifactory" is? If you mean maven central, I would agree-
> not needed, but if both code-signed and code-unsigned versions are available, it
> would be nice to promote the signed versions, I think.
>> Some further comments are inline below:
>>
>> Am 27.01.2016 um 21:28 schrieb Marshall Schor:
>>> Re: when to do the signing:
>>>
>>> My understanding which might be incorrect is that the Jar "signature" is
>>> compared against on computed when the jar is downloaded.
>>>
>>> I suspect this compare will fail if you compute this **before** "Compress plugin
>>> Jars....", because the signature will be computed on the un-packed Jar, but will
>>> be verified using the packed Jar.
>>>
>>> Of course, this is something to test :-). Since both packed and unpacked
>>> versions are part of the Eclipse update site (the packed ones are tried first,
>>> unless there's some older version of Eclipse (before packing) trying to use the
>>> update site), we might need to sign both versions.
>>> ------------------------------------------
>>> Re: signing features - these are tiny jars, and therefore not packed; Eclipse
>>> might required them to be unpacked; so it's probably not a good idea to copy
>>> these to ${toBePacked}.
>> Yes, we would copy them to a different folder or we would filter the jar
>> to be repacked.
>>
>>> ------------------------------------
>>> Re: calling on all jars in the folder(s): There are a lot of jars, because the
>>> build process puts all the versions of them into the folder; I think some part
>>> of the process needs all of them to build some kind of metadata.
>> In that folder there are only the new ones right now, the older ones are
>> in a different folder I think. If that is not the situation right now,
>> we could easily change it.
> I think they're in the same folder because part of the process of creating the
> update-site "artifacts.jar" and "content.jar" needs this layout, I believe.
>>> The older Jars will already have been signed (of course, once we get this whole
>>> process figured out :-) ), so they should not be re-signed, I think.
>>> ------------------------------------
>>> Re: delaying signing an RC until the vote passes: I too feel it might be
>>> wasteful to do this for multiple release candidates; on the other hand, it does
>>> "test" the signing. I wonder if the RC's could do the build using the "test"
>>> signing service, and then have some (automated, foolproof (?)) post-vote process
>>> to switch over to the official signing?
>> I even would skip the test code signing. I sketched your idea of the
>> post-vote process above.
>>
>>> ----------------------
>>> Re: updating the top-level parent pom for all of this, cancelling pending
>>> releases - I think we can do this another way :-)
>>> The idea would be to put the changes only into the Ruta parent pom for now.
>>> Once it's worked out, then we can promote those to the overall build POM.
>> Would that not mean that I have to prepare the ruta-2.4.0-rc3 anew?
>> There is no release vote, but all other stuff is done already (staged
>> stuff and so on).
> I would say that you don't need to hold up rc3 for all this - it may take some
> time to work out the kinks.
> Of course, on the other hand, this provides the incentive/motivation to work on
> this :-)
>
> -Marshall
>> Best,
>>
>> Peter
>>
>>
>>
>>
Re: signing Jars
Posted by Richard Eckart de Castilho <re...@apache.org>.
On 29.01.2016, at 11:18, Peter Klügl <pe...@averbis.com> wrote:
>
> @Richard:
> There are plans to host the artifacts here at Averbis :-)
> I really appreciate that your department does it, but I'd say that my
> company should (at least) do it (too).
> ... and the artifacts also need some updates in order to get rid of some
> annoying error problems in Eclipse.
Sure, go ahead. It would be best if they could be uploaded to Maven Central.
It might be worth pinging upstream people at Eclipse to check if they have
any thoughts on that.
I still think the best idea would be to move them to Bintray as a "neutral"
third party.
Cheers,
-- Richard
Re: signing Jars
Posted by Peter Klügl <pe...@averbis.com>.
Hi,
... an aggregate answer mail for the last three mails...
yes, I was meaning maven central.
If I have to guess, then I would say that the artifacts are normally
code-signing but here is no .asc ect. - I have to check it.
The artifacts are not bundled, but are only required for compiling the
Eclipse plugins, especially the stuff that requires DLTK, or other stuff
that is not hosted in maven central, but only in p2 repositories.
@Richard:
There are plans to host the artifacts here at Averbis :-)
I really appreciate that your department does it, but I'd say that my
company should (at least) do it (too).
... and the artifacts also need some updates in order to get rid of some
annoying error problems in Eclipse.
Best,
Peter
Am 28.01.2016 um 21:46 schrieb Richard Eckart de Castilho:
> I have no clue. Assuming that Eclipse features/JARs are usually signed, they are likely signed by their original authors.
>
> If I remember correctly, Peter had obtained them from a local Eclipse installation and we needed them in a repo so I put them in ours - btw. we could consider to upload them to Bintray (another JFrog product/service).
>
> Cheers,
>
> -- Richard
>
>> On 28.01.2016, at 21:39, Marshall Schor <ms...@schor.com> wrote:
>>
>> That reminds me - are the artifacts in UKP Artifactory signed (either
>> code-signed or .asc etc. signed)?
>>
>> When we "sign" a jar it would be good to know there's a chain of signed and
>> verified components that went into what gets built.
>>
>> I don't remember if the artifact in the UKP Artifactory are actually bundled
>> with what Ruta builds, or if they're just needed to compile?
>>
>> -Marshall
>>
>> On 1/28/2016 3:04 PM, Richard Eckart de Castilho wrote:
>>> On 28.01.2016, at 18:17, Marshall Schor <ms...@schor.com> wrote:
>>>>> (Another question is: do we need code signing for the jars in the
>>>>> artifactory? I would say no.)
>>>> Not sure what the "artifactory" is?
>>> Artifactory is a repository server product from JFrog ;) But I
>>> would also guess Peter talks about Maven Central - unless he
>>> talks about the UKP Lab Artifactory where we still host some
>>> Eclipse artifacts that I believe Ruta needs for building.
>>>
>>> -- Richard
Re: signing Jars
Posted by Richard Eckart de Castilho <re...@apache.org>.
I have no clue. Assuming that Eclipse features/JARs are usually signed, they are likely signed by their original authors.
If I remember correctly, Peter had obtained them from a local Eclipse installation and we needed them in a repo so I put them in ours - btw. we could consider to upload them to Bintray (another JFrog product/service).
Cheers,
-- Richard
> On 28.01.2016, at 21:39, Marshall Schor <ms...@schor.com> wrote:
>
> That reminds me - are the artifacts in UKP Artifactory signed (either
> code-signed or .asc etc. signed)?
>
> When we "sign" a jar it would be good to know there's a chain of signed and
> verified components that went into what gets built.
>
> I don't remember if the artifact in the UKP Artifactory are actually bundled
> with what Ruta builds, or if they're just needed to compile?
>
> -Marshall
>
> On 1/28/2016 3:04 PM, Richard Eckart de Castilho wrote:
>> On 28.01.2016, at 18:17, Marshall Schor <ms...@schor.com> wrote:
>>>> (Another question is: do we need code signing for the jars in the
>>>> artifactory? I would say no.)
>>> Not sure what the "artifactory" is?
>> Artifactory is a repository server product from JFrog ;) But I
>> would also guess Peter talks about Maven Central - unless he
>> talks about the UKP Lab Artifactory where we still host some
>> Eclipse artifacts that I believe Ruta needs for building.
>>
>> -- Richard
Re: signing Jars
Posted by Marshall Schor <ms...@schor.com>.
That reminds me - are the artifacts in UKP Artifactory signed (either
code-signed or .asc etc. signed)?
When we "sign" a jar it would be good to know there's a chain of signed and
verified components that went into what gets built.
I don't remember if the artifact in the UKP Artifactory are actually bundled
with what Ruta builds, or if they're just needed to compile?
-Marshall
On 1/28/2016 3:04 PM, Richard Eckart de Castilho wrote:
> On 28.01.2016, at 18:17, Marshall Schor <ms...@schor.com> wrote:
>>> (Another question is: do we need code signing for the jars in the
>>> artifactory? I would say no.)
>> Not sure what the "artifactory" is?
> Artifactory is a repository server product from JFrog ;) But I
> would also guess Peter talks about Maven Central - unless he
> talks about the UKP Lab Artifactory where we still host some
> Eclipse artifacts that I believe Ruta needs for building.
>
> -- Richard
>
Re: signing Jars
Posted by Richard Eckart de Castilho <re...@apache.org>.
On 28.01.2016, at 18:17, Marshall Schor <ms...@schor.com> wrote:
>
>> (Another question is: do we need code signing for the jars in the
>> artifactory? I would say no.)
> Not sure what the "artifactory" is?
Artifactory is a repository server product from JFrog ;) But I
would also guess Peter talks about Maven Central - unless he
talks about the UKP Lab Artifactory where we still host some
Eclipse artifacts that I believe Ruta needs for building.
-- Richard
Re: signing Jars
Posted by Marshall Schor <ms...@schor.com>.
On 1/28/2016 3:56 AM, Peter Klügl wrote:
> Hi
>
> you are of course correct. It's even maybe a bit more problematic.
>
> Let's distinguish the two signatures:
> - the gpg signature in a separate file together with the checksum (I
> call it signature)
> - the code signing signature in the META-INF within the jar (I call it
> code signing)
>
> This means that the signature has to happen after the code signing since
> the code signing signature changes the jar (independently of the
> repacking). I think the features need also code signing but no repackaging.
Sounds right to me.
>
> So the rough workflow building the update site would look like:
> - (optional: isApacheRelease) checkout the previous update site
> - collect the new features and plugins
I think you need to do any pack/repacking here of the jars, because that changes
the Jars' data, which the code signing is based on.
> - (optional: isCodeSigning) call the code signing service for the new features and plugins
The repack has to be done earlier
> - repack new plugins
> - generate update site and p2 info
This generates "artifacts.jar" and "content.jar". My guess is that these might
need to be code-signed, too?
> - (optional: isApacheRelease) sign (sha1, md5, asc) for new features and
> plugin and their packed versions
> - (optional: isApacheRelease) sign (sha1, md5, asc) the modified update
> site info
>
> There is of course a lot of copying in between.
>
> If we want to avoid the code signing for every release candidate, then
> we maybe need another stage in our release process:
> - prepare rc (without code signing)
> - test and vote as usual
> - if successful: rebuild the update site with code signing with the same
> artifacts
Probably better to avoid "rebuilding", and just take the unsigned artifacts from
the RC site and code-sign them, and regenerate the signatures, too.
> - test and vote code signing and signatures of the update site
If the post RC vote process is only to add the code-signing and redo the
signatures, I wonder if there can be an automatic validation that compares these
with the RC site (the Jars should be the same other than the added code-signing,
the .asd/md5/sha1 signatures should check as OK). If we have that, then as long
as the release manager runs the check, no additional voting would be necessary.
If the RC artifacts were put into the dist.apache.org/repos/dist/dev/uima spot,
it would be verifyable in the future, if need arises.
> - if successful: release
>
> (Another question is: do we need code signing for the jars in the
> artifactory? I would say no.)
Not sure what the "artifactory" is? If you mean maven central, I would agree-
not needed, but if both code-signed and code-unsigned versions are available, it
would be nice to promote the signed versions, I think.
>
> Some further comments are inline below:
>
> Am 27.01.2016 um 21:28 schrieb Marshall Schor:
>> Re: when to do the signing:
>>
>> My understanding which might be incorrect is that the Jar "signature" is
>> compared against on computed when the jar is downloaded.
>>
>> I suspect this compare will fail if you compute this **before** "Compress plugin
>> Jars....", because the signature will be computed on the un-packed Jar, but will
>> be verified using the packed Jar.
>>
>> Of course, this is something to test :-). Since both packed and unpacked
>> versions are part of the Eclipse update site (the packed ones are tried first,
>> unless there's some older version of Eclipse (before packing) trying to use the
>> update site), we might need to sign both versions.
>> ------------------------------------------
>> Re: signing features - these are tiny jars, and therefore not packed; Eclipse
>> might required them to be unpacked; so it's probably not a good idea to copy
>> these to ${toBePacked}.
> Yes, we would copy them to a different folder or we would filter the jar
> to be repacked.
>
>> ------------------------------------
>> Re: calling on all jars in the folder(s): There are a lot of jars, because the
>> build process puts all the versions of them into the folder; I think some part
>> of the process needs all of them to build some kind of metadata.
> In that folder there are only the new ones right now, the older ones are
> in a different folder I think. If that is not the situation right now,
> we could easily change it.
I think they're in the same folder because part of the process of creating the
update-site "artifacts.jar" and "content.jar" needs this layout, I believe.
>
>> The older Jars will already have been signed (of course, once we get this whole
>> process figured out :-) ), so they should not be re-signed, I think.
>> ------------------------------------
>> Re: delaying signing an RC until the vote passes: I too feel it might be
>> wasteful to do this for multiple release candidates; on the other hand, it does
>> "test" the signing. I wonder if the RC's could do the build using the "test"
>> signing service, and then have some (automated, foolproof (?)) post-vote process
>> to switch over to the official signing?
> I even would skip the test code signing. I sketched your idea of the
> post-vote process above.
>
>> ----------------------
>> Re: updating the top-level parent pom for all of this, cancelling pending
>> releases - I think we can do this another way :-)
>> The idea would be to put the changes only into the Ruta parent pom for now.
>> Once it's worked out, then we can promote those to the overall build POM.
> Would that not mean that I have to prepare the ruta-2.4.0-rc3 anew?
> There is no release vote, but all other stuff is done already (staged
> stuff and so on).
I would say that you don't need to hold up rc3 for all this - it may take some
time to work out the kinks.
Of course, on the other hand, this provides the incentive/motivation to work on
this :-)
-Marshall
>
> Best,
>
> Peter
>
>
>
>
Re: signing Jars
Posted by Peter Klügl <pe...@averbis.com>.
Hi
you are of course correct. It's even maybe a bit more problematic.
Let's distinguish the two signatures:
- the gpg signature in a separate file together with the checksum (I
call it signature)
- the code signing signature in the META-INF within the jar (I call it
code signing)
This means that the signature has to happen after the code signing since
the code signing signature changes the jar (independently of the
repacking). I think the features need also code signing but no repackaging.
So the rough workflow building the update site would look like:
- (optional: isApacheRelease) checkout the previous update site
- collect the new features and plugins
- (optional: isCodeSigning) call the code signing service for the new
features and plugins
- repack new plugins
- generate update site and p2 info
- (optional: isApacheRelease) sign (sha1, md5, asc) for new features and
plugin and their packed versions
- (optional: isApacheRelease) sign (sha1, md5, asc) the modified update
site info
There is of course a lot of copying in between.
If we want to avoid the code signing for every release candidate, then
we maybe need another stage in our release process:
- prepare rc (without code signing)
- test and vote as usual
- if successful: rebuild the update site with code signing with the same
artifacts
- test and vote code signing and signatures of the update site
- if successful: release
(Another question is: do we need code signing for the jars in the
artifactory? I would say no.)
Some further comments are inline below:
Am 27.01.2016 um 21:28 schrieb Marshall Schor:
> Re: when to do the signing:
>
> My understanding which might be incorrect is that the Jar "signature" is
> compared against on computed when the jar is downloaded.
>
> I suspect this compare will fail if you compute this **before** "Compress plugin
> Jars....", because the signature will be computed on the un-packed Jar, but will
> be verified using the packed Jar.
>
> Of course, this is something to test :-). Since both packed and unpacked
> versions are part of the Eclipse update site (the packed ones are tried first,
> unless there's some older version of Eclipse (before packing) trying to use the
> update site), we might need to sign both versions.
> ------------------------------------------
> Re: signing features - these are tiny jars, and therefore not packed; Eclipse
> might required them to be unpacked; so it's probably not a good idea to copy
> these to ${toBePacked}.
Yes, we would copy them to a different folder or we would filter the jar
to be repacked.
> ------------------------------------
> Re: calling on all jars in the folder(s): There are a lot of jars, because the
> build process puts all the versions of them into the folder; I think some part
> of the process needs all of them to build some kind of metadata.
In that folder there are only the new ones right now, the older ones are
in a different folder I think. If that is not the situation right now,
we could easily change it.
> The older Jars will already have been signed (of course, once we get this whole
> process figured out :-) ), so they should not be re-signed, I think.
> ------------------------------------
> Re: delaying signing an RC until the vote passes: I too feel it might be
> wasteful to do this for multiple release candidates; on the other hand, it does
> "test" the signing. I wonder if the RC's could do the build using the "test"
> signing service, and then have some (automated, foolproof (?)) post-vote process
> to switch over to the official signing?
I even would skip the test code signing. I sketched your idea of the
post-vote process above.
> ----------------------
> Re: updating the top-level parent pom for all of this, cancelling pending
> releases - I think we can do this another way :-)
> The idea would be to put the changes only into the Ruta parent pom for now.
> Once it's worked out, then we can promote those to the overall build POM.
Would that not mean that I have to prepare the ruta-2.4.0-rc3 anew?
There is no release vote, but all other stuff is done already (staged
stuff and so on).
Best,
Peter
Re: signing Jars
Posted by Marshall Schor <ms...@schor.com>.
Re: when to do the signing:
My understanding which might be incorrect is that the Jar "signature" is
compared against on computed when the jar is downloaded.
I suspect this compare will fail if you compute this **before** "Compress plugin
Jars....", because the signature will be computed on the un-packed Jar, but will
be verified using the packed Jar.
Of course, this is something to test :-). Since both packed and unpacked
versions are part of the Eclipse update site (the packed ones are tried first,
unless there's some older version of Eclipse (before packing) trying to use the
update site), we might need to sign both versions.
------------------------------------------
Re: signing features - these are tiny jars, and therefore not packed; Eclipse
might required them to be unpacked; so it's probably not a good idea to copy
these to ${toBePacked}.
------------------------------------
Re: calling on all jars in the folder(s): There are a lot of jars, because the
build process puts all the versions of them into the folder; I think some part
of the process needs all of them to build some kind of metadata.
The older Jars will already have been signed (of course, once we get this whole
process figured out :-) ), so they should not be re-signed, I think.
------------------------------------
Re: delaying signing an RC until the vote passes: I too feel it might be
wasteful to do this for multiple release candidates; on the other hand, it does
"test" the signing. I wonder if the RC's could do the build using the "test"
signing service, and then have some (automated, foolproof (?)) post-vote process
to switch over to the official signing?
----------------------
Re: updating the top-level parent pom for all of this, cancelling pending
releases - I think we can do this another way :-)
The idea would be to put the changes only into the Ruta parent pom for now.
Once it's worked out, then we can promote those to the overall build POM.
-Marshall
On 1/27/2016 5:34 AM, Peter Klügl wrote:
> Here are some first observations:
>
> - the ant task for the soap service should be called right before
> "Compress plugin Jars using pack200" (parent-pom: maven-antrun-plugin ->
> BuildUpdateSite-pack-svnget-buildMetadata-commit-to-dev)
> - in case the features are need to be signed, they have also to be
> copied to ${toBePacked} or something similar
> - the ant task should be called on all jars in that folder (or also an
> additional one for the features)
>
> Some consequences:
> - we would call the fee based signing service each time we build the
> update site with the apache-release profile. I actually do this
> sometimes to build snapshot update sites (a different solution is surely
> possible). However, we do this also for *every* release candidate. (For
> ruta, that would have been three times just because of some bugs). I do
> not have a good feeling about that. Maybe it's better to sign the
> bundles manually only for accepted release candidates (with additional
> repack)?
> - in order to integrate the ant task, we need to change the parent pom.
> Meaning we need to do a parent pom release. Meaning we need to update
> the parent pom version in our update site. Meaning we have to cancel
> both release candidates.
>
> Any opinions?
>
> Best,
>
> Peter
>
>
>
>
> Am 27.01.2016 um 10:24 schrieb Peter Klügl:
>> Marshall, are you able to access the test system?
>>
>> https://test-securesigning.websecurity.symantec.com/csportal/
>>
>>
>> Am 27.01.2016 um 10:17 schrieb Peter Klügl:
>>> Am 26.01.2016 um 22:09 schrieb Marshall Schor:
>>>> On 1/26/2016 5:20 AM, Peter Klügl wrote:
>>>>> ...
>>>>> ect all this info...
>>>>> Should we just copy the implementation to some of our (build) projects?
>>>> It would be good to automate this of course. I think that would take using the
>>>> SOAP interface with the Ant script done by the Apache TomCat project.
>>>> However, some basic info on our web site for posterity once we forget how this
>>>> all works would be good - pointers to other pages that describe how it all
>>>> works, how to use the test stuff, etc.
>>>> -Marshall
>>>>
>>> Yes, I will create/extend a page once it is up and running.
>>>
>>> Best,
>>>
>>> Peter
>>>
>
Re: signing Jars
Posted by Peter Klügl <pe...@averbis.com>.
Here are some first observations:
- the ant task for the soap service should be called right before
"Compress plugin Jars using pack200" (parent-pom: maven-antrun-plugin ->
BuildUpdateSite-pack-svnget-buildMetadata-commit-to-dev)
- in case the features are need to be signed, they have also to be
copied to ${toBePacked} or something similar
- the ant task should be called on all jars in that folder (or also an
additional one for the features)
Some consequences:
- we would call the fee based signing service each time we build the
update site with the apache-release profile. I actually do this
sometimes to build snapshot update sites (a different solution is surely
possible). However, we do this also for *every* release candidate. (For
ruta, that would have been three times just because of some bugs). I do
not have a good feeling about that. Maybe it's better to sign the
bundles manually only for accepted release candidates (with additional
repack)?
- in order to integrate the ant task, we need to change the parent pom.
Meaning we need to do a parent pom release. Meaning we need to update
the parent pom version in our update site. Meaning we have to cancel
both release candidates.
Any opinions?
Best,
Peter
Am 27.01.2016 um 10:24 schrieb Peter Klügl:
> Marshall, are you able to access the test system?
>
> https://test-securesigning.websecurity.symantec.com/csportal/
>
>
> Am 27.01.2016 um 10:17 schrieb Peter Klügl:
>> Am 26.01.2016 um 22:09 schrieb Marshall Schor:
>>> On 1/26/2016 5:20 AM, Peter Klügl wrote:
>>>> ...
>>>> ect all this info...
>>>> Should we just copy the implementation to some of our (build) projects?
>>> It would be good to automate this of course. I think that would take using the
>>> SOAP interface with the Ant script done by the Apache TomCat project.
>>> However, some basic info on our web site for posterity once we forget how this
>>> all works would be good - pointers to other pages that describe how it all
>>> works, how to use the test stuff, etc.
>>> -Marshall
>>>
>> Yes, I will create/extend a page once it is up and running.
>>
>> Best,
>>
>> Peter
>>
Re: signing Jars
Posted by Peter Klügl <pe...@averbis.com>.
Marshall, are you able to access the test system?
https://test-securesigning.websecurity.symantec.com/csportal/
Am 27.01.2016 um 10:17 schrieb Peter Klügl:
>
> Am 26.01.2016 um 22:09 schrieb Marshall Schor:
>> On 1/26/2016 5:20 AM, Peter Klügl wrote:
>>> ...
>>> ect all this info...
>>> Should we just copy the implementation to some of our (build) projects?
>> It would be good to automate this of course. I think that would take using the
>> SOAP interface with the Ant script done by the Apache TomCat project.
>> However, some basic info on our web site for posterity once we forget how this
>> all works would be good - pointers to other pages that describe how it all
>> works, how to use the test stuff, etc.
>> -Marshall
>>
> Yes, I will create/extend a page once it is up and running.
>
> Best,
>
> Peter
>
Re: signing Jars
Posted by Peter Klügl <pe...@averbis.com>.
Am 26.01.2016 um 22:09 schrieb Marshall Schor:
>
> On 1/26/2016 5:20 AM, Peter Klügl wrote:
>> ...
>> ect all this info...
>> Should we just copy the implementation to some of our (build) projects?
> It would be good to automate this of course. I think that would take using the
> SOAP interface with the Ant script done by the Apache TomCat project.
> However, some basic info on our web site for posterity once we forget how this
> all works would be good - pointers to other pages that describe how it all
> works, how to use the test stuff, etc.
> -Marshall
>
Yes, I will create/extend a page once it is up and running.
Best,
Peter
Re: signing Jars
Posted by Marshall Schor <ms...@schor.com>.
On 1/26/2016 5:20 AM, Peter Klügl wrote:
> Hi,
>
> Am 25.01.2016 um 21:33 schrieb Marshall Schor:
>> I only got to the point of setting up a way to log into the symantec site, where
>> I can apparently add other users, and do some other things.
>>
>> To integrating this into our Apache UIMA build, I think the following needs to
>> be figured out:
>>
>> 1) what Jars to sign. The only Jars we need signed in this way I think are the
>> Eclipse Plugins (and maybe the Feature Jars - but I would be surprised...).
> I would assume that feature jar also need to be signed.
> See the sample code here:
> http://www.ibm.com/developerworks/library/os-eclipse-plugin-sigs/
>
>> 2) The build process for the plugins is complicated by the fact that both
>> "packed" and "regular" Jars are built. The signing has to happen after all of
>> that has finished, I think. I remember a long time ago reading something on the
>> Eclipse websites about signing Eclipse things - that might be a place to start.
> This is probably http://wiki.eclipse.org/JAR_Signing
>
>> 3) There was an ant task done - by some other project - but I haven't looked at
>> it. The mail archives has some pointers (I think I may have copied info about
>> this into a previous email). We probably should start a UIMA website page to
>> collect all this info...
> Should we just copy the implementation to some of our (build) projects?
It would be good to automate this of course. I think that would take using the
SOAP interface with the Ant script done by the Apache TomCat project.
However, some basic info on our web site for posterity once we forget how this
all works would be good - pointers to other pages that describe how it all
works, how to use the test stuff, etc.
-Marshall
>
> Best,
>
> Peter
>
>> I'll do some digging to see if I can add Peter as a signer and send him the
>> credentials. The protocol used by Apache Infra was to send this info encrypted,
>> so I'll do that.
>>
>> -Marshall
>
Re: signing Jars
Posted by Peter Klügl <pe...@averbis.com>.
Hi,
Am 25.01.2016 um 21:33 schrieb Marshall Schor:
> I only got to the point of setting up a way to log into the symantec site, where
> I can apparently add other users, and do some other things.
>
> To integrating this into our Apache UIMA build, I think the following needs to
> be figured out:
>
> 1) what Jars to sign. The only Jars we need signed in this way I think are the
> Eclipse Plugins (and maybe the Feature Jars - but I would be surprised...).
I would assume that feature jar also need to be signed.
See the sample code here:
http://www.ibm.com/developerworks/library/os-eclipse-plugin-sigs/
>
> 2) The build process for the plugins is complicated by the fact that both
> "packed" and "regular" Jars are built. The signing has to happen after all of
> that has finished, I think. I remember a long time ago reading something on the
> Eclipse websites about signing Eclipse things - that might be a place to start.
This is probably http://wiki.eclipse.org/JAR_Signing
>
> 3) There was an ant task done - by some other project - but I haven't looked at
> it. The mail archives has some pointers (I think I may have copied info about
> this into a previous email). We probably should start a UIMA website page to
> collect all this info...
Should we just copy the implementation to some of our (build) projects?
Best,
Peter
> I'll do some digging to see if I can add Peter as a signer and send him the
> credentials. The protocol used by Apache Infra was to send this info encrypted,
> so I'll do that.
>
> -Marshall