You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Carsten Ziegeler <cz...@apache.org> on 2015/04/17 12:00:54 UTC

Slow/stopping check stages release scripts

Hi,

since some time I experience a very poor performce of our check script.
In many cases it even stops downloading.

However as soon as I remove the "--ignore-length" option from the wget
command, it gets fast and runs through. Anyone opposed to removing this
from the script?

Regards
Carsten
-- 
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org

Re: Slow/stopping check stages release scripts

Posted by Carsten Ziegeler <cz...@apache.org>.
> Ah, but still not 100% secure. The diff below should make the script
> cleaner and safer ( unless someone objects)
> #

I've applied the below check and also removed the ignore-length option.

Carsten

> Index: check_staged_release.sh
> ===================================================================
> --- check_staged_release.sh     (revision 1674258)
> +++ check_staged_release.sh     (working copy)
> @@ -16,15 +16,10 @@
>   echo "                           DOWNLOAD STAGED REPOSITORY
>                  "
>   echo "################################################################################"
> 
> - if [ `wget --help | grep "no-check-certificate" | wc -l` -eq 1 ]
> - then
> -   CHECK_SSL=--no-check-certificate
> - fi
> -
> - wget $CHECK_SSL \
> + wget \
>    -e "robots=off" --wait 1 -nv -r -np "--reject=html,txt" "--follow-tags=" \
>    -P "${DOWNLOAD}/${STAGING}" -nH "--cut-dirs=3" --ignore-length \
> -  "http://repository.apache.org/content/repositories/orgapachesling-${STAGING}/org/apache/sling/"
> +  "https://repository.apache.org/content/repositories/orgapachesling-${STAGING}/org/apache/sling/"
> 
>  else
>   echo "################################################################################"
> 
> Robert
> 


-- 
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org

Re: Slow/stopping check stages release scripts

Posted by Robert Munteanu <ro...@apache.org>.
On Fri, Apr 17, 2015 at 1:12 PM, Carsten Ziegeler <cz...@apache.org> wrote:
> Am 17.04.15 um 12:09 schrieb Robert Munteanu:
>> On Fri, Apr 17, 2015 at 1:00 PM, Carsten Ziegeler <cz...@apache.org> wrote:
>>> Hi,
>>>
>>> since some time I experience a very poor performce of our check script.
>>> In many cases it even stops downloading.
>>>
>>> However as soon as I remove the "--ignore-length" option from the wget
>>> command, it gets fast and runs through. Anyone opposed to removing this
>>> from the script?
>>
>> No, let's remove it and add it back if needed ( hopefully not ). While
>> looking at the script I noticed
>>
>>  if [ `wget --help | grep "no-check-certificate" | wc -l` -eq 1 ]
>>  then
>>    CHECK_SSL=--no-check-certificate
>>  fi
>>
>> which does not look good at all to me, especially since we access the
>> repository using http :-)
>>
> It seems nexus redirects to https :)

Ah, but still not 100% secure. The diff below should make the script
cleaner and safer ( unless someone objects)

Index: check_staged_release.sh
===================================================================
--- check_staged_release.sh     (revision 1674258)
+++ check_staged_release.sh     (working copy)
@@ -16,15 +16,10 @@
  echo "                           DOWNLOAD STAGED REPOSITORY
                 "
  echo "################################################################################"

- if [ `wget --help | grep "no-check-certificate" | wc -l` -eq 1 ]
- then
-   CHECK_SSL=--no-check-certificate
- fi
-
- wget $CHECK_SSL \
+ wget \
   -e "robots=off" --wait 1 -nv -r -np "--reject=html,txt" "--follow-tags=" \
   -P "${DOWNLOAD}/${STAGING}" -nH "--cut-dirs=3" --ignore-length \
-  "http://repository.apache.org/content/repositories/orgapachesling-${STAGING}/org/apache/sling/"
+  "https://repository.apache.org/content/repositories/orgapachesling-${STAGING}/org/apache/sling/"

 else
  echo "################################################################################"

Robert

Re: Slow/stopping check stages release scripts

Posted by Robert Munteanu <ro...@apache.org>.
On Fri, Apr 17, 2015 at 1:15 PM, Carsten Ziegeler <cz...@apache.org> wrote:
>
> I suggest we remove that section as well, having no cert check does not
> seem to be the right way


Sounds good to me.

Robert

Re: Slow/stopping check stages release scripts

Posted by Carsten Ziegeler <cz...@apache.org>.
Am 17.04.15 um 12:12 schrieb Carsten Ziegeler:
> Am 17.04.15 um 12:09 schrieb Robert Munteanu:
>> On Fri, Apr 17, 2015 at 1:00 PM, Carsten Ziegeler <cz...@apache.org> wrote:
>>> Hi,
>>>
>>> since some time I experience a very poor performce of our check script.
>>> In many cases it even stops downloading.
>>>
>>> However as soon as I remove the "--ignore-length" option from the wget
>>> command, it gets fast and runs through. Anyone opposed to removing this
>>> from the script?
>>
>> No, let's remove it and add it back if needed ( hopefully not ). While
>> looking at the script I noticed
>>
>>  if [ `wget --help | grep "no-check-certificate" | wc -l` -eq 1 ]
>>  then
>>    CHECK_SSL=--no-check-certificate
>>  fi
>>
>> which does not look good at all to me, especially since we access the
>> repository using http :-)
>>
> It seems nexus redirects to https :)
> 

I just checked running the script with directly using https and removing
the CHECK_SSL stuff

So that seems to work now.

I suggest we remove that section as well, having no cert check does not
seem to be the right way

Carsten
-- 
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org

Re: Slow/stopping check stages release scripts

Posted by Carsten Ziegeler <cz...@apache.org>.
Am 17.04.15 um 12:09 schrieb Robert Munteanu:
> On Fri, Apr 17, 2015 at 1:00 PM, Carsten Ziegeler <cz...@apache.org> wrote:
>> Hi,
>>
>> since some time I experience a very poor performce of our check script.
>> In many cases it even stops downloading.
>>
>> However as soon as I remove the "--ignore-length" option from the wget
>> command, it gets fast and runs through. Anyone opposed to removing this
>> from the script?
> 
> No, let's remove it and add it back if needed ( hopefully not ). While
> looking at the script I noticed
> 
>  if [ `wget --help | grep "no-check-certificate" | wc -l` -eq 1 ]
>  then
>    CHECK_SSL=--no-check-certificate
>  fi
> 
> which does not look good at all to me, especially since we access the
> repository using http :-)
> 
It seems nexus redirects to https :)

Carsten
-- 
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org

Re: Slow/stopping check stages release scripts

Posted by Robert Munteanu <ro...@apache.org>.
On Fri, Apr 17, 2015 at 1:00 PM, Carsten Ziegeler <cz...@apache.org> wrote:
> Hi,
>
> since some time I experience a very poor performce of our check script.
> In many cases it even stops downloading.
>
> However as soon as I remove the "--ignore-length" option from the wget
> command, it gets fast and runs through. Anyone opposed to removing this
> from the script?

No, let's remove it and add it back if needed ( hopefully not ). While
looking at the script I noticed

 if [ `wget --help | grep "no-check-certificate" | wc -l` -eq 1 ]
 then
   CHECK_SSL=--no-check-certificate
 fi

which does not look good at all to me, especially since we access the
repository using http :-)

Robert