You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Steve Rowe (JIRA)" <ji...@apache.org> on 2014/03/27 06:32:17 UTC
[jira] [Updated] (LUCENE-5556) buildAndPushRelease.py: scp to
people.apache.org stalls during transfer
[ https://issues.apache.org/jira/browse/LUCENE-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Rowe updated LUCENE-5556:
-------------------------------
Attachment: LUCENE-5556.patch
I tried some experiments with {{scp}} (including with the {{-l 8192}} option), disconnecting my ethernet cable to force a connection loss. {{scp}} didn't fail after 5 minutes or so, but on reconnecting the cable, {{scp}} did not resume the transfer, and did not fail either.
However, under the same conditions, {{rsync}} is capable of resuming transfers. {{rsync}} can also display progress, which is nice.
I'd like to replace {{scp}} with {{rsync}} in {{buildAndPushRelease.py}}.
The attached patch (against trunk) does this, and also retries remote operations, including file transfers, when they fail.
I'm testing this patch now. An {{rsync}} file transfer stalled for some reason, and when I typed Ctrl-C in the terminal running {{buildAndPushRelease.py}}, a retry was triggered for the file transfer.
{{rsync}} is also capable of resuming partial transfers, though in my testing the option to do so doesn't work unless the target file already exists. The patch performs a remote {{touch target-file}} to bring it into existence first if necessary.
> buildAndPushRelease.py: scp to people.apache.org stalls during transfer
> -----------------------------------------------------------------------
>
> Key: LUCENE-5556
> URL: https://issues.apache.org/jira/browse/LUCENE-5556
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build
> Reporter: Steve Rowe
> Priority: Minor
> Attachments: LUCENE-5556.patch
>
>
> When I first tried to build the first RC for 4.7.1 using {{dev-tools/scripts/buildAndPushRelease.py}}, after preparing both the Lucene and Solr releases, the {{scp}} command to transfer the Lucene release tarball to people.apache.org stalled after about 180MB had transferred out of 220MB. After an hour or so, I {{kill -9}}'d the {{scp}} process and started over. Thankfully, everything went smoothly the second go-around.
> But I'd like to see if we can reduce the likelihood of this happening again.
> I did some searching for causes/solutions to this problem and came across this recommendation to use {{scp}}'s {{-l}} option to limit bandwidth in order to prevent stalls: [http://www.aixmind.com/?p=1371] - the recommendation there limits bandwidth to 1MB/sec ({{-l 8192}} is 8192Kbit/sec = 8Mbit/sec = 1MB/sec).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org