You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jmeter.apache.org by Oliver Lloyd <ol...@hotmail.com> on 2016/04/03 21:09:55 UTC
Best location to download jmeter from using an unattended script?
I have a shell script that spins up some AWS instances to run tests on. As part of the init process on each instance it downloads and installs jmeter. I've noticed lately that the location I'm using is often very slow and I was wondering if there was a better one to use?
This is where I am having the issue: https://github.com/oliverlloyd/jmeter-ec2/blob/master/install.sh#L47
Note. I could self host the files, I've taken that approach with certain plugins, but because I'm letting people choose which version they want to use it's substantially better if they don't have to wait for me to copy over distributions into my own S3 bucket when new releases are made.
Thanks
Oliver
Re: Best location to download jmeter from using an unattended script?
Posted by sebb <se...@gmail.com>.
Remember to quote the URL; it contains some shell meta-characters.
On 4 April 2016 at 13:29, Oliver Lloyd <ol...@hotmail.com> wrote:
> That url is in theory exactly what I need and it works well in a browser but I'm not able to make it work from the CLI using something like wget or curl. Does anyone know how to make that cgi script serve me the file and not just the html for the download page?
>
>
>> On 3 Apr 2016, at 22:48, sebb <se...@gmail.com> wrote:
>>
>> On 3 April 2016 at 20:46, Oliver Lloyd <ol...@hotmail.com> wrote:
>>> Bintray sounds ideal. Speeds between dl.bintray.com and AWS are very fast.
>>>
>>> My issue is I can't really use a mirror because they change but linking to the host servers is bad because they're not designed to serve these files. A distribution as a service provider like bintray would definitely solve this problem.
>>
>> Why not?
>>
>> It's possible to use the automatic mirror chooser with a parameter
>> that automatically downloads:
>>
>> http://www.apache.org/dyn/closer.cgi?filename=jmeter/binaries/apache-jmeter-2.13.zip&action=download
>>
>>> In the meantime I'm experimenting with using 'www.apache.org/dist/ <http://www.apache.org/dist/>...' vs. 'archive.apache.org/dist/ <http://archive.apache.org/dist/>...' and this seems better.
>>
>> Neither of those should be used for normal downloads.
>> They are for last resort backup purposes / archived releases only.
>>
>>>
>>>> On 3 Apr 2016, at 20:31, Vladimir Sitnikov <si...@gmail.com> wrote:
>>>>
>>>> What if we deployed binary artifacts to maven, then they would get
>>>> mirrored to bintray?
>>>>
>>>> Groovy is using bintray.com for distribution of their binaries:
>>>> http://groovy-lang.org/download.html links to
>>>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>>>
>>>> Oliver,
>>>> Can I kindly ask you to download something like
>>>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>>> to check if that will be good enough?
>>>>
>>>> Vladimir
>>>>
>>>
>>
>
Re: Best location to download jmeter from using an unattended script?
Posted by Vladimir Sitnikov <si...@gmail.com>.
sebb> Sorry, but these tasks are out of scope for the JMeter project.
I'm afraid I fail to see how distribution is "out of scope".
Is "distribution of maven artifacts" in scope?
How the scope is defined? (I mean not the full definition but related
to distribution part of it)
Vladimir
Re: Best location to download jmeter from using an unattended script?
Posted by sebb <se...@gmail.com>.
On 4 April 2016 at 21:14, Vladimir Sitnikov <si...@gmail.com> wrote:
> sebb>I just don't think we should be expected to help with this beyond
> sebb>providing the information needed to download JMeter.
> Sebb, I think JMeter should provide users an ability to stick with a
> specific version.
That is already possible.
If a user downloads a copy of JMeter, it will be available until they delete it.
If they do accidentally delete it, then they can get it again quite easily.
> It is crucial for reproducible tests. Unexpected flips of JMeter
> versions would be bad.
It is not our problem if they start using a different version of JMeter.
We have no control over that, nor any responsibility.
> There are cases when "new version" breaks here and there as there are
> just way too many
> moving samplers.
That may be true but is not relevant to how JMeter is made available.
> I would not say JMeter breaks often, however I was personally bitten
> by those issues.
Ditto.
> Being able to ship multiple versions would simplify "try 3.0" kind of scenarios.
We already provide all the published versions.
For all ASF projects. Since the ASF started.
> Bintray is a well-known distribution facility.
> It provides binary distribution for free for OSS projects, so I see absolutely
> no reason to ignore it.
I did not say ignore it, but it is out of scope for the JMeter project.
> Are there objections on not providing bintray binaries?
Yes, it is out of scope for the project.
> Apache Groovy uses bintray, so it should be doable.
Of course it is doable.
But it's not in scope for the JMeter project.
> There are three tasks:
> 0) Configure bintray repository owner somehow
> 1) Integrate bintray publish action into release process
> 2) Publish previous versions
Sorry, but these tasks are out of scope for the JMeter project.
> Vladimir
Re: Best location to download jmeter from using an unattended script?
Posted by Vladimir Sitnikov <si...@gmail.com>.
sebb>I just don't think we should be expected to help with this beyond
sebb>providing the information needed to download JMeter.
Sebb, I think JMeter should provide users an ability to stick with a
specific version.
It is crucial for reproducible tests. Unexpected flips of JMeter
versions would be bad.
There are cases when "new version" breaks here and there as there are
just way too many
moving samplers.
I would not say JMeter breaks often, however I was personally bitten
by those issues.
Being able to ship multiple versions would simplify "try 3.0" kind of scenarios.
Bintray is a well-known distribution facility.
It provides binary distribution for free for OSS projects, so I see absolutely
no reason to ignore it.
Are there objections on not providing bintray binaries?
Apache Groovy uses bintray, so it should be doable.
There are three tasks:
0) Configure bintray repository owner somehow
1) Integrate bintray publish action into release process
2) Publish previous versions
Vladimir
Re: Best location to download jmeter from using an unattended script?
Posted by sebb <se...@gmail.com>.
On 4 April 2016 at 17:56, Oliver Lloyd <ol...@hotmail.com> wrote:
> Umm, well yes, that's true, I am asking you to distribute the binaries, but you know, to be fair, it is your project, plus you already do host them using archive.apache.org <http://archive.apache.org/>, it's just slow.
> I'm a bit confused why you would expect me to host them instead of you.
We *do* host the binaries and the ASF provides the origin mirror hosts
for the 3rd party mirror system.
The mirror system is intended to help spread the load of downloads for
currently maintained software.
However there is far too much software for it all to be available from
all the mirrors regardless.
Remember there are hundreds of projects at the ASF.
We are not expecting you to host the binaries *instead* of us; we
already host them.
However you appear to be expecting *us* to provide faster downloads
for all versions of JMeter.
I'm sorry, but that's not reasonable.
> I'm also a bit confused how I managed to offend you there, I really wasn't trying to upset anyone, just asking for help.
I'm not upset.
I just don't think we should be expected to help with this beyond
providing the information needed to download JMeter.
>
>> On 4 Apr 2016, at 17:36, sebb <se...@gmail.com> wrote:
>>
>> On 4 April 2016 at 17:29, Oliver Lloyd <ol...@hotmail.com> wrote:
>>>> Why do you want to use the older versions?
>>>
>>> It's not so much that i want to use older versions. The problem is if you don't let people specify the version then they will get whatever is either: a) current or b) I give them. In both cases whenever these things change then next time the script is run (when they next test) then they will suddenly have a different version of jmeter which could, conceivably, result in different results or even problems running their scripts.
>>
>> That should not be a problem if you host the binaries.
>>
>>>> Why will it break?
>>>
>>> If I host the binary files myself then the risk is a new version of jmeter will be released that is not available in my repo (because I haven't copied it over) which will mean the script, in its current form, would fail were someone to specify the new version. Sure, I could check for a 404 response and return a useful message but this really starts to over complicate things and is quite a step backwards. Plus I'd essentially be managing my own distribution server for jmeter which I'd rather not do.
>>
>> Sorry to be so blunt, but instead you are effectively asking someone
>> else to manage the distributions server ..
>>
>>> Bintray does seem to be made for exactly this problem. It's also free for open source; do you think it is possible to set something up?
>>
>> This is completely out of scope for the JMeter project.
>>
>>>
>>>
>>>> On 4 Apr 2016, at 17:03, sebb <se...@gmail.com> wrote:
>>>>
>>>> On 4 April 2016 at 15:28, Oliver Lloyd <ol...@hotmail.com> wrote:
>>>>> Thanks Vladimir, I can pull out the host from this using:
>>>>>
>>>>> preferred=$(curl -s 'http://www.apache.org/dyn/closer.cgi?as_json=1' | grep "preferred" | cut -d ':' -f3 | cut -d'/' -f3)
>>>>
>>>> No need, the URL I gave is equivalent.
>>>>
>>>>> But there's another problem: The mirrors only seem to host the latest version of JMeter.
>>>>
>>>> Yes, that is intentional.
>>>>
>>>>> Older versions, as far as I can tell, only live on archive.apache.org <http://archive.apache.org/>
>>>>
>>>> Yes.
>>>>
>>>> or www.apache.org <http://www.apache.org/>.
>>>>
>>>> Huh? No.
>>>>
>>>>> I know that these servers are not meant to serve binaries like this but I keep ending coming back to using them.
>>>>>
>>>>> I thought about trying to host older versions myself but it will break each time a new release is made.
>>>>
>>>> Why will it break?
>>>>
>>>> And why do you want to use the older versions?
>>>>
>>>>> Another solution I considered was only allowing the latest version to be used but that would mean an uncontrolled upgrade for everyone each time a release is made, which isn't correct.
>>>>>
>>>>>
>>>>>> On 4 Apr 2016, at 13:34, Vladimir Sitnikov <si...@gmail.com> wrote:
>>>>>>
>>>>>> curl 'http://www.apache.org/dyn/closer.cgi?as_json=1'
>>>>>> gives something like
>>>>>> {
>>>>>> "backup": [ "http://www-eu.apache.org/dist/",
>>>>>> "http://www-us.apache.org/dist/" ],
>>>>>> "cca2": "ru",
>>>>>> "http": [ "http://apache-mirror.rbc.ru/pub/apache/" ],
>>>>>> "path_info": "",
>>>>>> "preferred": "http://apache-mirror.rbc.ru/pub/apache/"
>>>>>> }
>>>>>>
>>>>>> Then you grep for preferred somehow, then construct the proper URL.
>>>>>>
>>>>>> Does that work for you?
>>>>>> Vladimir
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Best location to download jmeter from using an unattended script?
Posted by Oliver Lloyd <ol...@hotmail.com>.
Umm, well yes, that's true, I am asking you to distribute the binaries, but you know, to be fair, it is your project, plus you already do host them using archive.apache.org <http://archive.apache.org/>, it's just slow.
I'm a bit confused why you would expect me to host them instead of you. I'm also a bit confused how I managed to offend you there, I really wasn't trying to upset anyone, just asking for help.
> On 4 Apr 2016, at 17:36, sebb <se...@gmail.com> wrote:
>
> On 4 April 2016 at 17:29, Oliver Lloyd <ol...@hotmail.com> wrote:
>>> Why do you want to use the older versions?
>>
>> It's not so much that i want to use older versions. The problem is if you don't let people specify the version then they will get whatever is either: a) current or b) I give them. In both cases whenever these things change then next time the script is run (when they next test) then they will suddenly have a different version of jmeter which could, conceivably, result in different results or even problems running their scripts.
>
> That should not be a problem if you host the binaries.
>
>>> Why will it break?
>>
>> If I host the binary files myself then the risk is a new version of jmeter will be released that is not available in my repo (because I haven't copied it over) which will mean the script, in its current form, would fail were someone to specify the new version. Sure, I could check for a 404 response and return a useful message but this really starts to over complicate things and is quite a step backwards. Plus I'd essentially be managing my own distribution server for jmeter which I'd rather not do.
>
> Sorry to be so blunt, but instead you are effectively asking someone
> else to manage the distributions server ..
>
>> Bintray does seem to be made for exactly this problem. It's also free for open source; do you think it is possible to set something up?
>
> This is completely out of scope for the JMeter project.
>
>>
>>
>>> On 4 Apr 2016, at 17:03, sebb <se...@gmail.com> wrote:
>>>
>>> On 4 April 2016 at 15:28, Oliver Lloyd <ol...@hotmail.com> wrote:
>>>> Thanks Vladimir, I can pull out the host from this using:
>>>>
>>>> preferred=$(curl -s 'http://www.apache.org/dyn/closer.cgi?as_json=1' | grep "preferred" | cut -d ':' -f3 | cut -d'/' -f3)
>>>
>>> No need, the URL I gave is equivalent.
>>>
>>>> But there's another problem: The mirrors only seem to host the latest version of JMeter.
>>>
>>> Yes, that is intentional.
>>>
>>>> Older versions, as far as I can tell, only live on archive.apache.org <http://archive.apache.org/>
>>>
>>> Yes.
>>>
>>> or www.apache.org <http://www.apache.org/>.
>>>
>>> Huh? No.
>>>
>>>> I know that these servers are not meant to serve binaries like this but I keep ending coming back to using them.
>>>>
>>>> I thought about trying to host older versions myself but it will break each time a new release is made.
>>>
>>> Why will it break?
>>>
>>> And why do you want to use the older versions?
>>>
>>>> Another solution I considered was only allowing the latest version to be used but that would mean an uncontrolled upgrade for everyone each time a release is made, which isn't correct.
>>>>
>>>>
>>>>> On 4 Apr 2016, at 13:34, Vladimir Sitnikov <si...@gmail.com> wrote:
>>>>>
>>>>> curl 'http://www.apache.org/dyn/closer.cgi?as_json=1'
>>>>> gives something like
>>>>> {
>>>>> "backup": [ "http://www-eu.apache.org/dist/",
>>>>> "http://www-us.apache.org/dist/" ],
>>>>> "cca2": "ru",
>>>>> "http": [ "http://apache-mirror.rbc.ru/pub/apache/" ],
>>>>> "path_info": "",
>>>>> "preferred": "http://apache-mirror.rbc.ru/pub/apache/"
>>>>> }
>>>>>
>>>>> Then you grep for preferred somehow, then construct the proper URL.
>>>>>
>>>>> Does that work for you?
>>>>> Vladimir
>>>>>
>>>>
>>>
>>
>
Re: Best location to download jmeter from using an unattended script?
Posted by sebb <se...@gmail.com>.
On 4 April 2016 at 17:29, Oliver Lloyd <ol...@hotmail.com> wrote:
>> Why do you want to use the older versions?
>
> It's not so much that i want to use older versions. The problem is if you don't let people specify the version then they will get whatever is either: a) current or b) I give them. In both cases whenever these things change then next time the script is run (when they next test) then they will suddenly have a different version of jmeter which could, conceivably, result in different results or even problems running their scripts.
That should not be a problem if you host the binaries.
>> Why will it break?
>
> If I host the binary files myself then the risk is a new version of jmeter will be released that is not available in my repo (because I haven't copied it over) which will mean the script, in its current form, would fail were someone to specify the new version. Sure, I could check for a 404 response and return a useful message but this really starts to over complicate things and is quite a step backwards. Plus I'd essentially be managing my own distribution server for jmeter which I'd rather not do.
Sorry to be so blunt, but instead you are effectively asking someone
else to manage the distributions server ..
> Bintray does seem to be made for exactly this problem. It's also free for open source; do you think it is possible to set something up?
This is completely out of scope for the JMeter project.
>
>
>> On 4 Apr 2016, at 17:03, sebb <se...@gmail.com> wrote:
>>
>> On 4 April 2016 at 15:28, Oliver Lloyd <ol...@hotmail.com> wrote:
>>> Thanks Vladimir, I can pull out the host from this using:
>>>
>>> preferred=$(curl -s 'http://www.apache.org/dyn/closer.cgi?as_json=1' | grep "preferred" | cut -d ':' -f3 | cut -d'/' -f3)
>>
>> No need, the URL I gave is equivalent.
>>
>>> But there's another problem: The mirrors only seem to host the latest version of JMeter.
>>
>> Yes, that is intentional.
>>
>>> Older versions, as far as I can tell, only live on archive.apache.org <http://archive.apache.org/>
>>
>> Yes.
>>
>> or www.apache.org <http://www.apache.org/>.
>>
>> Huh? No.
>>
>>> I know that these servers are not meant to serve binaries like this but I keep ending coming back to using them.
>>>
>>> I thought about trying to host older versions myself but it will break each time a new release is made.
>>
>> Why will it break?
>>
>> And why do you want to use the older versions?
>>
>>> Another solution I considered was only allowing the latest version to be used but that would mean an uncontrolled upgrade for everyone each time a release is made, which isn't correct.
>>>
>>>
>>>> On 4 Apr 2016, at 13:34, Vladimir Sitnikov <si...@gmail.com> wrote:
>>>>
>>>> curl 'http://www.apache.org/dyn/closer.cgi?as_json=1'
>>>> gives something like
>>>> {
>>>> "backup": [ "http://www-eu.apache.org/dist/",
>>>> "http://www-us.apache.org/dist/" ],
>>>> "cca2": "ru",
>>>> "http": [ "http://apache-mirror.rbc.ru/pub/apache/" ],
>>>> "path_info": "",
>>>> "preferred": "http://apache-mirror.rbc.ru/pub/apache/"
>>>> }
>>>>
>>>> Then you grep for preferred somehow, then construct the proper URL.
>>>>
>>>> Does that work for you?
>>>> Vladimir
>>>>
>>>
>>
>
Re: Best location to download jmeter from using an unattended script?
Posted by Oliver Lloyd <ol...@hotmail.com>.
> Why do you want to use the older versions?
It's not so much that i want to use older versions. The problem is if you don't let people specify the version then they will get whatever is either: a) current or b) I give them. In both cases whenever these things change then next time the script is run (when they next test) then they will suddenly have a different version of jmeter which could, conceivably, result in different results or even problems running their scripts.
> Why will it break?
If I host the binary files myself then the risk is a new version of jmeter will be released that is not available in my repo (because I haven't copied it over) which will mean the script, in its current form, would fail were someone to specify the new version. Sure, I could check for a 404 response and return a useful message but this really starts to over complicate things and is quite a step backwards. Plus I'd essentially be managing my own distribution server for jmeter which I'd rather not do.
Bintray does seem to be made for exactly this problem. It's also free for open source; do you think it is possible to set something up?
> On 4 Apr 2016, at 17:03, sebb <se...@gmail.com> wrote:
>
> On 4 April 2016 at 15:28, Oliver Lloyd <ol...@hotmail.com> wrote:
>> Thanks Vladimir, I can pull out the host from this using:
>>
>> preferred=$(curl -s 'http://www.apache.org/dyn/closer.cgi?as_json=1' | grep "preferred" | cut -d ':' -f3 | cut -d'/' -f3)
>
> No need, the URL I gave is equivalent.
>
>> But there's another problem: The mirrors only seem to host the latest version of JMeter.
>
> Yes, that is intentional.
>
>> Older versions, as far as I can tell, only live on archive.apache.org <http://archive.apache.org/>
>
> Yes.
>
> or www.apache.org <http://www.apache.org/>.
>
> Huh? No.
>
>> I know that these servers are not meant to serve binaries like this but I keep ending coming back to using them.
>>
>> I thought about trying to host older versions myself but it will break each time a new release is made.
>
> Why will it break?
>
> And why do you want to use the older versions?
>
>> Another solution I considered was only allowing the latest version to be used but that would mean an uncontrolled upgrade for everyone each time a release is made, which isn't correct.
>>
>>
>>> On 4 Apr 2016, at 13:34, Vladimir Sitnikov <si...@gmail.com> wrote:
>>>
>>> curl 'http://www.apache.org/dyn/closer.cgi?as_json=1'
>>> gives something like
>>> {
>>> "backup": [ "http://www-eu.apache.org/dist/",
>>> "http://www-us.apache.org/dist/" ],
>>> "cca2": "ru",
>>> "http": [ "http://apache-mirror.rbc.ru/pub/apache/" ],
>>> "path_info": "",
>>> "preferred": "http://apache-mirror.rbc.ru/pub/apache/"
>>> }
>>>
>>> Then you grep for preferred somehow, then construct the proper URL.
>>>
>>> Does that work for you?
>>> Vladimir
>>>
>>
>
Re: Best location to download jmeter from using an unattended script?
Posted by sebb <se...@gmail.com>.
On 4 April 2016 at 15:28, Oliver Lloyd <ol...@hotmail.com> wrote:
> Thanks Vladimir, I can pull out the host from this using:
>
> preferred=$(curl -s 'http://www.apache.org/dyn/closer.cgi?as_json=1' | grep "preferred" | cut -d ':' -f3 | cut -d'/' -f3)
No need, the URL I gave is equivalent.
> But there's another problem: The mirrors only seem to host the latest version of JMeter.
Yes, that is intentional.
> Older versions, as far as I can tell, only live on archive.apache.org <http://archive.apache.org/>
Yes.
or www.apache.org <http://www.apache.org/>.
Huh? No.
> I know that these servers are not meant to serve binaries like this but I keep ending coming back to using them.
>
> I thought about trying to host older versions myself but it will break each time a new release is made.
Why will it break?
And why do you want to use the older versions?
> Another solution I considered was only allowing the latest version to be used but that would mean an uncontrolled upgrade for everyone each time a release is made, which isn't correct.
>
>
>> On 4 Apr 2016, at 13:34, Vladimir Sitnikov <si...@gmail.com> wrote:
>>
>> curl 'http://www.apache.org/dyn/closer.cgi?as_json=1'
>> gives something like
>> {
>> "backup": [ "http://www-eu.apache.org/dist/",
>> "http://www-us.apache.org/dist/" ],
>> "cca2": "ru",
>> "http": [ "http://apache-mirror.rbc.ru/pub/apache/" ],
>> "path_info": "",
>> "preferred": "http://apache-mirror.rbc.ru/pub/apache/"
>> }
>>
>> Then you grep for preferred somehow, then construct the proper URL.
>>
>> Does that work for you?
>> Vladimir
>>
>
Re: Best location to download jmeter from using an unattended script?
Posted by Oliver Lloyd <ol...@hotmail.com>.
Thanks Vladimir, I can pull out the host from this using:
preferred=$(curl -s 'http://www.apache.org/dyn/closer.cgi?as_json=1' | grep "preferred" | cut -d ':' -f3 | cut -d'/' -f3)
But there's another problem: The mirrors only seem to host the latest version of JMeter. Older versions, as far as I can tell, only live on archive.apache.org <http://archive.apache.org/> or www.apache.org <http://www.apache.org/>. I know that these servers are not meant to serve binaries like this but I keep ending coming back to using them.
I thought about trying to host older versions myself but it will break each time a new release is made.
Another solution I considered was only allowing the latest version to be used but that would mean an uncontrolled upgrade for everyone each time a release is made, which isn't correct.
> On 4 Apr 2016, at 13:34, Vladimir Sitnikov <si...@gmail.com> wrote:
>
> curl 'http://www.apache.org/dyn/closer.cgi?as_json=1'
> gives something like
> {
> "backup": [ "http://www-eu.apache.org/dist/",
> "http://www-us.apache.org/dist/" ],
> "cca2": "ru",
> "http": [ "http://apache-mirror.rbc.ru/pub/apache/" ],
> "path_info": "",
> "preferred": "http://apache-mirror.rbc.ru/pub/apache/"
> }
>
> Then you grep for preferred somehow, then construct the proper URL.
>
> Does that work for you?
> Vladimir
>
Re: Best location to download jmeter from using an unattended script?
Posted by Vladimir Sitnikov <si...@gmail.com>.
curl 'http://www.apache.org/dyn/closer.cgi?as_json=1'
gives something like
{
"backup": [ "http://www-eu.apache.org/dist/",
"http://www-us.apache.org/dist/" ],
"cca2": "ru",
"http": [ "http://apache-mirror.rbc.ru/pub/apache/" ],
"path_info": "",
"preferred": "http://apache-mirror.rbc.ru/pub/apache/"
}
Then you grep for preferred somehow, then construct the proper URL.
Does that work for you?
Vladimir
Re: Best location to download jmeter from using an unattended script?
Posted by Oliver Lloyd <ol...@hotmail.com>.
Thanks, this is better. I should have quoted it before, I just saw your message on that earlier.
> On 5 Apr 2016, at 16:49, sebb <se...@gmail.com> wrote:
>
> On 5 April 2016 at 16:33, Oliver Lloyd <oliver_lloyd@hotmail.com <ma...@hotmail.com>> wrote:
>> In the end I wrote a function. It solved the closer.cgi issue I was having by using the as_json=1 query param that Vladimir suggested.
>>
>> #!/bin/bash
>>
>> function install_jmeter() {
>> # ------------------------------------------------
>> # Decide where to download jmeter from
>> #
>> # Order of preference:
>> # 1. Mirror, if the desired version is current
>> # 2. S3, if not current and we have a copy
>> # 3. Archive, as a backup
>> # ------------------------------------------------
>>
>> # Mirrors only host the current version, get the preferred mirror for this location
>> preferred_mirror=$(curl -s 'http://www.apache.org/dyn/closer.cgi?as_json=1' | grep "preferred" | cut -d ':' -f3 | cut -d'"' -f1 | awk -F// '{print $NF}' | sed 's/.$//')
>
> There's no need to extract the preferred mirror, see below.
>
>> # Scrape the main binaries page to see what the current version is
>> current=$(curl -s 'http://www.apache.org/dist/jmeter/binaries/' <http://www.apache.org/dist/jmeter/binaries/'>)
>
> Once you have the current version file stem, you can to use the URL I
> gave you earlier to download it directly using a preferred mirror:
>
> wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz
> "http://www.apache.org/dyn/closer.cgi?filename=jmeter/binaries/$JMETER_VERSION.tgz&action=download <http://www.apache.org/dyn/closer.cgi?filename=jmeter/binaries/$JMETER_VERSION.tgz&action=download>"
>
> I just tried it, and it works.
>
>> if [ $(echo $current | grep -c "$JMETER_VERSION") -gt "0" ] ; then
>> # This is the current version of jmeter so use the preferred mirror to download
>> echo "downloading from preferred mirror: http://$preferred_mirror/jmeter/binaries/$JMETER_VERSION.tgz"
>> wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz http://$preferred_mirror/jmeter/binaries/$JMETER_VERSION.tgz
>> elif [ $(curl -sI https://s3.amazonaws.com/jmeter-ec2/$JMETER_VERSION.tgz | grep -c "403 Forbidden") -eq "0" ] ; then
>> # It wasn't the current version but we have a copy on S3 so use that
>> echo "Downloading jmeter from S3"
>> wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz https://s3.amazonaws.com/jmeter-ec2/$JMETER_VERSION.tgz
>> else
>> # Fall back to the archive server
>> echo "Downloading jmeter from Apache Archive"
>> wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz http://archive.apache.org/dist/jmeter/binaries/$JMETER_VERSION.tgz
>> fi
>>
>> # Untar downloaded file
>> tar -xf $REMOTE_HOME/$JMETER_VERSION.tgz
>> }
>> JMETER_VERSION="apache-jmeter-2.13"
>> REMOTE_HOME="/tmp"
>> install_jmeter
>>
>>
>>
>>
>>> On 5 Apr 2016, at 13:21, Felix Schumacher <fe...@internetallee.de> wrote:
>>>
>>>
>>>
>>> Am 4. April 2016 14:29:13 MESZ, schrieb Oliver Lloyd <ol...@hotmail.com>:
>>>> That url is in theory exactly what I need and it works well in a
>>>> browser but I'm not able to make it work from the CLI using something
>>>> like wget or curl. Does anyone know how to make that cgi script serve
>>>> me the file and not just the html for the download page?
>>>
>>> Have you tried "curl -L 'http://..."?
>>>
>>> Regards,
>>> Felix
>>>
>>>>
>>>>
>>>>> On 3 Apr 2016, at 22:48, sebb <se...@gmail.com> wrote:
>>>>>
>>>>> On 3 April 2016 at 20:46, Oliver Lloyd <ol...@hotmail.com>
>>>> wrote:
>>>>>> Bintray sounds ideal. Speeds between dl.bintray.com and AWS are very
>>>> fast.
>>>>>>
>>>>>> My issue is I can't really use a mirror because they change but
>>>> linking to the host servers is bad because they're not designed to
>>>> serve these files. A distribution as a service provider like bintray
>>>> would definitely solve this problem.
>>>>>
>>>>> Why not?
>>>>>
>>>>> It's possible to use the automatic mirror chooser with a parameter
>>>>> that automatically downloads:
>>>>>
>>>>>
>>>> http://www.apache.org/dyn/closer.cgi?filename=jmeter/binaries/apache-jmeter-2.13.zip&action=download
>>>>>
>>>>>> In the meantime I'm experimenting with using 'www.apache.org/dist/
>>>> <http://www.apache.org/dist/>...' vs. 'archive.apache.org/dist/
>>>> <http://archive.apache.org/dist/>...' and this seems better.
>>>>>
>>>>> Neither of those should be used for normal downloads.
>>>>> They are for last resort backup purposes / archived releases only.
>>>>>
>>>>>>
>>>>>>> On 3 Apr 2016, at 20:31, Vladimir Sitnikov
>>>> <si...@gmail.com> wrote:
>>>>>>>
>>>>>>> What if we deployed binary artifacts to maven, then they would get
>>>>>>> mirrored to bintray?
>>>>>>>
>>>>>>> Groovy is using bintray.com for distribution of their binaries:
>>>>>>> http://groovy-lang.org/download.html links to
>>>>>>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>>>>>>
>>>>>>> Oliver,
>>>>>>> Can I kindly ask you to download something like
>>>>>>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>>>>>> to check if that will be good enough?
>>>>>>>
>>>>>>> Vladimir
Re: Best location to download jmeter from using an unattended script?
Posted by sebb <se...@gmail.com>.
On 5 April 2016 at 16:33, Oliver Lloyd <ol...@hotmail.com> wrote:
> In the end I wrote a function. It solved the closer.cgi issue I was having by using the as_json=1 query param that Vladimir suggested.
>
> #!/bin/bash
>
> function install_jmeter() {
> # ------------------------------------------------
> # Decide where to download jmeter from
> #
> # Order of preference:
> # 1. Mirror, if the desired version is current
> # 2. S3, if not current and we have a copy
> # 3. Archive, as a backup
> # ------------------------------------------------
>
> # Mirrors only host the current version, get the preferred mirror for this location
> preferred_mirror=$(curl -s 'http://www.apache.org/dyn/closer.cgi?as_json=1' | grep "preferred" | cut -d ':' -f3 | cut -d'"' -f1 | awk -F// '{print $NF}' | sed 's/.$//')
There's no need to extract the preferred mirror, see below.
> # Scrape the main binaries page to see what the current version is
> current=$(curl -s 'http://www.apache.org/dist/jmeter/binaries/')
Once you have the current version file stem, you can to use the URL I
gave you earlier to download it directly using a preferred mirror:
wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz
"http://www.apache.org/dyn/closer.cgi?filename=jmeter/binaries/$JMETER_VERSION.tgz&action=download"
I just tried it, and it works.
> if [ $(echo $current | grep -c "$JMETER_VERSION") -gt "0" ] ; then
> # This is the current version of jmeter so use the preferred mirror to download
> echo "downloading from preferred mirror: http://$preferred_mirror/jmeter/binaries/$JMETER_VERSION.tgz"
> wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz http://$preferred_mirror/jmeter/binaries/$JMETER_VERSION.tgz
> elif [ $(curl -sI https://s3.amazonaws.com/jmeter-ec2/$JMETER_VERSION.tgz | grep -c "403 Forbidden") -eq "0" ] ; then
> # It wasn't the current version but we have a copy on S3 so use that
> echo "Downloading jmeter from S3"
> wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz https://s3.amazonaws.com/jmeter-ec2/$JMETER_VERSION.tgz
> else
> # Fall back to the archive server
> echo "Downloading jmeter from Apache Archive"
> wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz http://archive.apache.org/dist/jmeter/binaries/$JMETER_VERSION.tgz
> fi
>
> # Untar downloaded file
> tar -xf $REMOTE_HOME/$JMETER_VERSION.tgz
> }
> JMETER_VERSION="apache-jmeter-2.13"
> REMOTE_HOME="/tmp"
> install_jmeter
>
>
>
>
>> On 5 Apr 2016, at 13:21, Felix Schumacher <fe...@internetallee.de> wrote:
>>
>>
>>
>> Am 4. April 2016 14:29:13 MESZ, schrieb Oliver Lloyd <ol...@hotmail.com>:
>>> That url is in theory exactly what I need and it works well in a
>>> browser but I'm not able to make it work from the CLI using something
>>> like wget or curl. Does anyone know how to make that cgi script serve
>>> me the file and not just the html for the download page?
>>
>> Have you tried "curl -L 'http://..."?
>>
>> Regards,
>> Felix
>>
>>>
>>>
>>>> On 3 Apr 2016, at 22:48, sebb <se...@gmail.com> wrote:
>>>>
>>>> On 3 April 2016 at 20:46, Oliver Lloyd <ol...@hotmail.com>
>>> wrote:
>>>>> Bintray sounds ideal. Speeds between dl.bintray.com and AWS are very
>>> fast.
>>>>>
>>>>> My issue is I can't really use a mirror because they change but
>>> linking to the host servers is bad because they're not designed to
>>> serve these files. A distribution as a service provider like bintray
>>> would definitely solve this problem.
>>>>
>>>> Why not?
>>>>
>>>> It's possible to use the automatic mirror chooser with a parameter
>>>> that automatically downloads:
>>>>
>>>>
>>> http://www.apache.org/dyn/closer.cgi?filename=jmeter/binaries/apache-jmeter-2.13.zip&action=download
>>>>
>>>>> In the meantime I'm experimenting with using 'www.apache.org/dist/
>>> <http://www.apache.org/dist/>...' vs. 'archive.apache.org/dist/
>>> <http://archive.apache.org/dist/>...' and this seems better.
>>>>
>>>> Neither of those should be used for normal downloads.
>>>> They are for last resort backup purposes / archived releases only.
>>>>
>>>>>
>>>>>> On 3 Apr 2016, at 20:31, Vladimir Sitnikov
>>> <si...@gmail.com> wrote:
>>>>>>
>>>>>> What if we deployed binary artifacts to maven, then they would get
>>>>>> mirrored to bintray?
>>>>>>
>>>>>> Groovy is using bintray.com for distribution of their binaries:
>>>>>> http://groovy-lang.org/download.html links to
>>>>>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>>>>>
>>>>>> Oliver,
>>>>>> Can I kindly ask you to download something like
>>>>>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>>>>> to check if that will be good enough?
>>>>>>
>>>>>> Vladimir
>>>>>>
>>>>>
>>>>
>>
>>
>
Re: Best location to download jmeter from using an unattended script?
Posted by Oliver Lloyd <ol...@hotmail.com>.
In the end I wrote a function. It solved the closer.cgi issue I was having by using the as_json=1 query param that Vladimir suggested.
#!/bin/bash
function install_jmeter() {
# ------------------------------------------------
# Decide where to download jmeter from
#
# Order of preference:
# 1. Mirror, if the desired version is current
# 2. S3, if not current and we have a copy
# 3. Archive, as a backup
# ------------------------------------------------
# Mirrors only host the current version, get the preferred mirror for this location
preferred_mirror=$(curl -s 'http://www.apache.org/dyn/closer.cgi?as_json=1' | grep "preferred" | cut -d ':' -f3 | cut -d'"' -f1 | awk -F// '{print $NF}' | sed 's/.$//')
# Scrape the main binaries page to see what the current version is
current=$(curl -s 'http://www.apache.org/dist/jmeter/binaries/')
if [ $(echo $current | grep -c "$JMETER_VERSION") -gt "0" ] ; then
# This is the current version of jmeter so use the preferred mirror to download
echo "downloading from preferred mirror: http://$preferred_mirror/jmeter/binaries/$JMETER_VERSION.tgz"
wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz http://$preferred_mirror/jmeter/binaries/$JMETER_VERSION.tgz
elif [ $(curl -sI https://s3.amazonaws.com/jmeter-ec2/$JMETER_VERSION.tgz | grep -c "403 Forbidden") -eq "0" ] ; then
# It wasn't the current version but we have a copy on S3 so use that
echo "Downloading jmeter from S3"
wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz https://s3.amazonaws.com/jmeter-ec2/$JMETER_VERSION.tgz
else
# Fall back to the archive server
echo "Downloading jmeter from Apache Archive"
wget -q -O $REMOTE_HOME/$JMETER_VERSION.tgz http://archive.apache.org/dist/jmeter/binaries/$JMETER_VERSION.tgz
fi
# Untar downloaded file
tar -xf $REMOTE_HOME/$JMETER_VERSION.tgz
}
JMETER_VERSION="apache-jmeter-2.13"
REMOTE_HOME="/tmp"
install_jmeter
> On 5 Apr 2016, at 13:21, Felix Schumacher <fe...@internetallee.de> wrote:
>
>
>
> Am 4. April 2016 14:29:13 MESZ, schrieb Oliver Lloyd <ol...@hotmail.com>:
>> That url is in theory exactly what I need and it works well in a
>> browser but I'm not able to make it work from the CLI using something
>> like wget or curl. Does anyone know how to make that cgi script serve
>> me the file and not just the html for the download page?
>
> Have you tried "curl -L 'http://..."?
>
> Regards,
> Felix
>
>>
>>
>>> On 3 Apr 2016, at 22:48, sebb <se...@gmail.com> wrote:
>>>
>>> On 3 April 2016 at 20:46, Oliver Lloyd <ol...@hotmail.com>
>> wrote:
>>>> Bintray sounds ideal. Speeds between dl.bintray.com and AWS are very
>> fast.
>>>>
>>>> My issue is I can't really use a mirror because they change but
>> linking to the host servers is bad because they're not designed to
>> serve these files. A distribution as a service provider like bintray
>> would definitely solve this problem.
>>>
>>> Why not?
>>>
>>> It's possible to use the automatic mirror chooser with a parameter
>>> that automatically downloads:
>>>
>>>
>> http://www.apache.org/dyn/closer.cgi?filename=jmeter/binaries/apache-jmeter-2.13.zip&action=download
>>>
>>>> In the meantime I'm experimenting with using 'www.apache.org/dist/
>> <http://www.apache.org/dist/>...' vs. 'archive.apache.org/dist/
>> <http://archive.apache.org/dist/>...' and this seems better.
>>>
>>> Neither of those should be used for normal downloads.
>>> They are for last resort backup purposes / archived releases only.
>>>
>>>>
>>>>> On 3 Apr 2016, at 20:31, Vladimir Sitnikov
>> <si...@gmail.com> wrote:
>>>>>
>>>>> What if we deployed binary artifacts to maven, then they would get
>>>>> mirrored to bintray?
>>>>>
>>>>> Groovy is using bintray.com for distribution of their binaries:
>>>>> http://groovy-lang.org/download.html links to
>>>>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>>>>
>>>>> Oliver,
>>>>> Can I kindly ask you to download something like
>>>>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>>>> to check if that will be good enough?
>>>>>
>>>>> Vladimir
>>>>>
>>>>
>>>
>
>
Re: Best location to download jmeter from using an unattended script?
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 4. April 2016 14:29:13 MESZ, schrieb Oliver Lloyd <ol...@hotmail.com>:
>That url is in theory exactly what I need and it works well in a
>browser but I'm not able to make it work from the CLI using something
>like wget or curl. Does anyone know how to make that cgi script serve
>me the file and not just the html for the download page?
Have you tried "curl -L 'http://..."?
Regards,
Felix
>
>
>> On 3 Apr 2016, at 22:48, sebb <se...@gmail.com> wrote:
>>
>> On 3 April 2016 at 20:46, Oliver Lloyd <ol...@hotmail.com>
>wrote:
>>> Bintray sounds ideal. Speeds between dl.bintray.com and AWS are very
>fast.
>>>
>>> My issue is I can't really use a mirror because they change but
>linking to the host servers is bad because they're not designed to
>serve these files. A distribution as a service provider like bintray
>would definitely solve this problem.
>>
>> Why not?
>>
>> It's possible to use the automatic mirror chooser with a parameter
>> that automatically downloads:
>>
>>
>http://www.apache.org/dyn/closer.cgi?filename=jmeter/binaries/apache-jmeter-2.13.zip&action=download
>>
>>> In the meantime I'm experimenting with using 'www.apache.org/dist/
><http://www.apache.org/dist/>...' vs. 'archive.apache.org/dist/
><http://archive.apache.org/dist/>...' and this seems better.
>>
>> Neither of those should be used for normal downloads.
>> They are for last resort backup purposes / archived releases only.
>>
>>>
>>>> On 3 Apr 2016, at 20:31, Vladimir Sitnikov
><si...@gmail.com> wrote:
>>>>
>>>> What if we deployed binary artifacts to maven, then they would get
>>>> mirrored to bintray?
>>>>
>>>> Groovy is using bintray.com for distribution of their binaries:
>>>> http://groovy-lang.org/download.html links to
>>>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>>>
>>>> Oliver,
>>>> Can I kindly ask you to download something like
>>>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>>> to check if that will be good enough?
>>>>
>>>> Vladimir
>>>>
>>>
>>
Re: Best location to download jmeter from using an unattended script?
Posted by Oliver Lloyd <ol...@hotmail.com>.
That url is in theory exactly what I need and it works well in a browser but I'm not able to make it work from the CLI using something like wget or curl. Does anyone know how to make that cgi script serve me the file and not just the html for the download page?
> On 3 Apr 2016, at 22:48, sebb <se...@gmail.com> wrote:
>
> On 3 April 2016 at 20:46, Oliver Lloyd <ol...@hotmail.com> wrote:
>> Bintray sounds ideal. Speeds between dl.bintray.com and AWS are very fast.
>>
>> My issue is I can't really use a mirror because they change but linking to the host servers is bad because they're not designed to serve these files. A distribution as a service provider like bintray would definitely solve this problem.
>
> Why not?
>
> It's possible to use the automatic mirror chooser with a parameter
> that automatically downloads:
>
> http://www.apache.org/dyn/closer.cgi?filename=jmeter/binaries/apache-jmeter-2.13.zip&action=download
>
>> In the meantime I'm experimenting with using 'www.apache.org/dist/ <http://www.apache.org/dist/>...' vs. 'archive.apache.org/dist/ <http://archive.apache.org/dist/>...' and this seems better.
>
> Neither of those should be used for normal downloads.
> They are for last resort backup purposes / archived releases only.
>
>>
>>> On 3 Apr 2016, at 20:31, Vladimir Sitnikov <si...@gmail.com> wrote:
>>>
>>> What if we deployed binary artifacts to maven, then they would get
>>> mirrored to bintray?
>>>
>>> Groovy is using bintray.com for distribution of their binaries:
>>> http://groovy-lang.org/download.html links to
>>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>>
>>> Oliver,
>>> Can I kindly ask you to download something like
>>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>> to check if that will be good enough?
>>>
>>> Vladimir
>>>
>>
>
Re: Best location to download jmeter from using an unattended script?
Posted by sebb <se...@gmail.com>.
On 3 April 2016 at 20:46, Oliver Lloyd <ol...@hotmail.com> wrote:
> Bintray sounds ideal. Speeds between dl.bintray.com and AWS are very fast.
>
> My issue is I can't really use a mirror because they change but linking to the host servers is bad because they're not designed to serve these files. A distribution as a service provider like bintray would definitely solve this problem.
Why not?
It's possible to use the automatic mirror chooser with a parameter
that automatically downloads:
http://www.apache.org/dyn/closer.cgi?filename=jmeter/binaries/apache-jmeter-2.13.zip&action=download
> In the meantime I'm experimenting with using 'www.apache.org/dist/ <http://www.apache.org/dist/>...' vs. 'archive.apache.org/dist/ <http://archive.apache.org/dist/>...' and this seems better.
Neither of those should be used for normal downloads.
They are for last resort backup purposes / archived releases only.
>
>> On 3 Apr 2016, at 20:31, Vladimir Sitnikov <si...@gmail.com> wrote:
>>
>> What if we deployed binary artifacts to maven, then they would get
>> mirrored to bintray?
>>
>> Groovy is using bintray.com for distribution of their binaries:
>> http://groovy-lang.org/download.html links to
>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>>
>> Oliver,
>> Can I kindly ask you to download something like
>> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>> to check if that will be good enough?
>>
>> Vladimir
>>
>
Re: Best location to download jmeter from using an unattended script?
Posted by Oliver Lloyd <ol...@hotmail.com>.
Bintray sounds ideal. Speeds between dl.bintray.com and AWS are very fast.
My issue is I can't really use a mirror because they change but linking to the host servers is bad because they're not designed to serve these files. A distribution as a service provider like bintray would definitely solve this problem.
In the meantime I'm experimenting with using 'www.apache.org/dist/ <http://www.apache.org/dist/>...' vs. 'archive.apache.org/dist/ <http://archive.apache.org/dist/>...' and this seems better.
> On 3 Apr 2016, at 20:31, Vladimir Sitnikov <si...@gmail.com> wrote:
>
> What if we deployed binary artifacts to maven, then they would get
> mirrored to bintray?
>
> Groovy is using bintray.com for distribution of their binaries:
> http://groovy-lang.org/download.html links to
> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
>
> Oliver,
> Can I kindly ask you to download something like
> https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
> to check if that will be good enough?
>
> Vladimir
>
Re: Best location to download jmeter from using an unattended script?
Posted by Vladimir Sitnikov <si...@gmail.com>.
What if we deployed binary artifacts to maven, then they would get
mirrored to bintray?
Groovy is using bintray.com for distribution of their binaries:
http://groovy-lang.org/download.html links to
https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
Oliver,
Can I kindly ask you to download something like
https://dl.bintray.com/groovy/maven/apache-groovy-sdk-2.4.6.zip
to check if that will be good enough?
Vladimir