You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flex.apache.org by OmPrakash Muppirala <bi...@gmail.com> on 2014/08/14 20:08:01 UTC

Automating md5 checks

Alex,

I see that there is a job
http://apacheflexbuild.cloudapp.net:8080/job/MD5Checker/
Are you using that to automatically check the MD5?

If so, you can spit out the correct MD5 into a text file and make it
available as part of the lastSuccessfulBuild/artifacts/out directory.  You
can then access that file publicly with this url:
http://apacheflexbuild.cloudapp.net:8080/job/MD5Checker/lastSuccessfulBuild/artifact/out/md5.txt

You can load this url in the Installer and verify the MD5 against the
downloaded file.

This way, you don't need to modify the installer config every time Adobe
changes the bits.

Thanks,
Om

Re: Automating md5 checks

Posted by Alex Harui <ah...@adobe.com>.

The installer does an MD5 check on the package from the mirrors, but all
other MD5 checks for 4.13 are done by ant_on_air so it works in Ant as
well.  The ant script in installer.xml for Flex 4.13 downloads
sdk-installer-config-4.0.xml from flex.a.o and gets the md5s from there.
I don't see how we could get that ant script to use an md5 off of Jenkins
via redirection.  We could change this for 4.14, but 4.13 will still be
looking at the sdk-installer-config-4.0.xml file.

It also just occurred to me that it would be risky to involve CI servers
that are committer-owned into the install process.  In theory, these
archived releases should work for as long as flex.a.o is around.

-Alex

On 8/14/14 2:59 PM, "OmPrakash Muppirala" <bi...@gmail.com> wrote:

>On Thu, Aug 14, 2014 at 2:53 PM, Alex Harui <ah...@adobe.com> wrote:
>
>>
>>
>> On 8/14/14 2:24 PM, "OmPrakash Muppirala" <bi...@gmail.com> wrote:
>>
>> >My understanding is that you are manually updating the installer config
>> >with a new md5 value each time the artifact changes, is that correct?
>>By
>> >letting the Jenkins job compute the checksum for the artifact, you dont
>> >have to manually change the config file everytime.
>> >
>> >You can still employ your technique of looking at last modified date.
>>My
>> >suggestion does not require you to change that logic.
>> >
>> >Yes, it is a one time change to the Installer itself, but after that
>> >change
>> >you dont have to wait for the MD5 check to break, hopefully someone
>> >reports
>> >it us and we having to change the installer config.
>> I still don't get it.  The installer.xml that shipped with Flex 4.13.0
>>is
>> looking up the md5 in sdk-installer-config-4.0.xml.  I don't see any way
>> to prevent it from doing so.
>>
>
>You can modify the Installer to load the jenkins based md5 url and use
>that
>to verify the downloaded file.  For example, today,  the Apache Flex SDK
>is
>downloaded from a mirror, but the MD5 is downloaded from
>dist.apache.org/path_to_flex_artifact/artifactname.MD5.  The same way, we
>can download the Flash artifact from the Adobe server and the MD5 value
>from our trusted Jenkins url.
>
>
>
>>
>> The job currently runs once a day, but we could run it hourly.  I think
>>we
>> could try to have the job update the sdk-installer-config-4.0.xml
>> automatically if we're brave enough to let it do so.  But the main
>>reason
>> someone found the issue was that the job did not run because the jenkins
>> slave broke down.  However, even if we do automatic hourly updates we'll
>> be open to someone not being able to install for 59 minutes if we have
>>bad
>> timing.
>>
>
>In that case, we can simply ask the user to retry after an hour.
>
>
>>
>> -Alex
>>
>>

Re: Automating md5 checks

Posted by OmPrakash Muppirala <bi...@gmail.com>.

On Thu, Aug 14, 2014 at 2:53 PM, Alex Harui <ah...@adobe.com> wrote:

>
>
> On 8/14/14 2:24 PM, "OmPrakash Muppirala" <bi...@gmail.com> wrote:
>
> >My understanding is that you are manually updating the installer config
> >with a new md5 value each time the artifact changes, is that correct?  By
> >letting the Jenkins job compute the checksum for the artifact, you dont
> >have to manually change the config file everytime.
> >
> >You can still employ your technique of looking at last modified date.  My
> >suggestion does not require you to change that logic.
> >
> >Yes, it is a one time change to the Installer itself, but after that
> >change
> >you dont have to wait for the MD5 check to break, hopefully someone
> >reports
> >it us and we having to change the installer config.
> I still don't get it.  The installer.xml that shipped with Flex 4.13.0 is
> looking up the md5 in sdk-installer-config-4.0.xml.  I don't see any way
> to prevent it from doing so.
>

You can modify the Installer to load the jenkins based md5 url and use that
to verify the downloaded file.  For example, today,  the Apache Flex SDK is
downloaded from a mirror, but the MD5 is downloaded from
dist.apache.org/path_to_flex_artifact/artifactname.MD5.  The same way, we
can download the Flash artifact from the Adobe server and the MD5 value
from our trusted Jenkins url.



>
> The job currently runs once a day, but we could run it hourly.  I think we
> could try to have the job update the sdk-installer-config-4.0.xml
> automatically if we're brave enough to let it do so.  But the main reason
> someone found the issue was that the job did not run because the jenkins
> slave broke down.  However, even if we do automatic hourly updates we'll
> be open to someone not being able to install for 59 minutes if we have bad
> timing.
>

In that case, we can simply ask the user to retry after an hour.


>
> -Alex
>
>

Re: Automating md5 checks

Posted by Alex Harui <ah...@adobe.com>.

On 8/14/14 2:24 PM, "OmPrakash Muppirala" <bi...@gmail.com> wrote:

>My understanding is that you are manually updating the installer config
>with a new md5 value each time the artifact changes, is that correct?  By
>letting the Jenkins job compute the checksum for the artifact, you dont
>have to manually change the config file everytime.
>
>You can still employ your technique of looking at last modified date.  My
>suggestion does not require you to change that logic.
>
>Yes, it is a one time change to the Installer itself, but after that
>change
>you dont have to wait for the MD5 check to break, hopefully someone
>reports
>it us and we having to change the installer config.
I still don't get it.  The installer.xml that shipped with Flex 4.13.0 is
looking up the md5 in sdk-installer-config-4.0.xml.  I don't see any way
to prevent it from doing so.

The job currently runs once a day, but we could run it hourly.  I think we
could try to have the job update the sdk-installer-config-4.0.xml
automatically if we're brave enough to let it do so.  But the main reason
someone found the issue was that the job did not run because the jenkins
slave broke down.  However, even if we do automatic hourly updates we'll
be open to someone not being able to install for 59 minutes if we have bad
timing.

-Alex

Re: Automating md5 checks

Posted by OmPrakash Muppirala <bi...@gmail.com>.

My understanding is that you are manually updating the installer config
with a new md5 value each time the artifact changes, is that correct?  By
letting the Jenkins job compute the checksum for the artifact, you dont
have to manually change the config file everytime.

You can still employ your technique of looking at last modified date.  My
suggestion does not require you to change that logic.

Yes, it is a one time change to the Installer itself, but after that change
you dont have to wait for the MD5 check to break, hopefully someone reports
it us and we having to change the installer config.

Thanks,
Om


On Thu, Aug 14, 2014 at 2:04 PM, Alex Harui <ah...@adobe.com> wrote:

> Still not understanding.  How would that be less work and require a new
> installer scripts?  The current SDK and FlexJS scripts are going to look
> in the config file.
>
> Also note that, to reduce bandwidth, this job stores an identifier from
> the server response headers such as the last modified date.  That way the
> job doesn't download these huge AIR SDKs over and over again unless they
> report as changed.  Even if we created separate .md5 files, we still need
> a place to track the identifiers.
>
> -Alex
>
> On 8/14/14 1:55 PM, "OmPrakash Muppirala" <bi...@gmail.com> wrote:
>
> >On Thu, Aug 14, 2014 at 1:10 PM, Alex Harui <ah...@adobe.com> wrote:
> >
> >>
> >>
> >> On 8/14/14 11:08 AM, "OmPrakash Muppirala" <bi...@gmail.com>
> wrote:
> >>
> >> >Alex,
> >> >
> >> >I see that there is a job
> >> >http://apacheflexbuild.cloudapp.net:8080/job/MD5Checker/
> >> >Are you using that to automatically check the MD5?
> >> Yes
> >> >
> >> >If so, you can spit out the correct MD5 into a text file and make it
> >> >available as part of the lastSuccessfulBuild/artifacts/out directory.
> >>You
> >> >can then access that file publicly with this url:
> >> >
> >>
> >>
> http://apacheflexbuild.cloudapp.net:8080/job/MD5Checker/lastSuccessfulBui
> >>l
> >> >d/artifact/out/md5.txt
> >> >
> >> >You can load this url in the Installer and verify the MD5 against the
> >> >downloaded file.
> >> >
> >> >This way, you don't need to modify the installer config every time
> >>Adobe
> >> >changes the bits.
> >> I'm not understanding.  I don't see any mention of md5.txt in Installer
> >> code base.
> >>
> >
> >That was an example file name.  You can write out the md5 checksum of the
> >artifact to a file and load it from the url.
> >
> >Thanks,
> >Om
> >
> >
> >>
> >> -Alex
> >>
> >>
>
>

Re: Automating md5 checks

Posted by Alex Harui <ah...@adobe.com>.

Still not understanding.  How would that be less work and require a new
installer scripts?  The current SDK and FlexJS scripts are going to look
in the config file.

Also note that, to reduce bandwidth, this job stores an identifier from
the server response headers such as the last modified date.  That way the
job doesn't download these huge AIR SDKs over and over again unless they
report as changed.  Even if we created separate .md5 files, we still need
a place to track the identifiers.

-Alex

On 8/14/14 1:55 PM, "OmPrakash Muppirala" <bi...@gmail.com> wrote:

>On Thu, Aug 14, 2014 at 1:10 PM, Alex Harui <ah...@adobe.com> wrote:
>
>>
>>
>> On 8/14/14 11:08 AM, "OmPrakash Muppirala" <bi...@gmail.com> wrote:
>>
>> >Alex,
>> >
>> >I see that there is a job
>> >http://apacheflexbuild.cloudapp.net:8080/job/MD5Checker/
>> >Are you using that to automatically check the MD5?
>> Yes
>> >
>> >If so, you can spit out the correct MD5 into a text file and make it
>> >available as part of the lastSuccessfulBuild/artifacts/out directory.
>>You
>> >can then access that file publicly with this url:
>> >
>> 
>>http://apacheflexbuild.cloudapp.net:8080/job/MD5Checker/lastSuccessfulBui
>>l
>> >d/artifact/out/md5.txt
>> >
>> >You can load this url in the Installer and verify the MD5 against the
>> >downloaded file.
>> >
>> >This way, you don't need to modify the installer config every time
>>Adobe
>> >changes the bits.
>> I'm not understanding.  I don't see any mention of md5.txt in Installer
>> code base.
>>
>
>That was an example file name.  You can write out the md5 checksum of the
>artifact to a file and load it from the url.
>
>Thanks,
>Om
>
>
>>
>> -Alex
>>
>>

Re: Automating md5 checks

Posted by OmPrakash Muppirala <bi...@gmail.com>.

On Thu, Aug 14, 2014 at 1:10 PM, Alex Harui <ah...@adobe.com> wrote:

>
>
> On 8/14/14 11:08 AM, "OmPrakash Muppirala" <bi...@gmail.com> wrote:
>
> >Alex,
> >
> >I see that there is a job
> >http://apacheflexbuild.cloudapp.net:8080/job/MD5Checker/
> >Are you using that to automatically check the MD5?
> Yes
> >
> >If so, you can spit out the correct MD5 into a text file and make it
> >available as part of the lastSuccessfulBuild/artifacts/out directory.  You
> >can then access that file publicly with this url:
> >
> http://apacheflexbuild.cloudapp.net:8080/job/MD5Checker/lastSuccessfulBuil
> >d/artifact/out/md5.txt
> >
> >You can load this url in the Installer and verify the MD5 against the
> >downloaded file.
> >
> >This way, you don't need to modify the installer config every time Adobe
> >changes the bits.
> I'm not understanding.  I don't see any mention of md5.txt in Installer
> code base.
>

That was an example file name.  You can write out the md5 checksum of the
artifact to a file and load it from the url.

Thanks,
Om


>
> -Alex
>
>

Re: Automating md5 checks

Posted by Alex Harui <ah...@adobe.com>.


On 8/14/14 11:08 AM, "OmPrakash Muppirala" <bi...@gmail.com> wrote:

>Alex,
>
>I see that there is a job
>http://apacheflexbuild.cloudapp.net:8080/job/MD5Checker/
>Are you using that to automatically check the MD5?
Yes
>
>If so, you can spit out the correct MD5 into a text file and make it
>available as part of the lastSuccessfulBuild/artifacts/out directory.  You
>can then access that file publicly with this url:
>http://apacheflexbuild.cloudapp.net:8080/job/MD5Checker/lastSuccessfulBuil
>d/artifact/out/md5.txt
>
>You can load this url in the Installer and verify the MD5 against the
>downloaded file.
>
>This way, you don't need to modify the installer config every time Adobe
>changes the bits.
I'm not understanding.  I don't see any mention of md5.txt in Installer
code base.

-Alex