[jira] [Commented] (BEAM-5052) beam_PreCommit_Website_Stage times out after 100m

["Udi Meiri (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564463#comment-16564463 ] 

Udi Meiri commented on BEAM-5052:
---------------------------------

[~Ardagan] [~melap]

> beam_PreCommit_Website_Stage times out after 100m
> -------------------------------------------------
>
>                 Key: BEAM-5052
>                 URL: https://issues.apache.org/jira/browse/BEAM-5052
>             Project: Beam
>          Issue Type: Bug
>          Components: testing, website
>            Reporter: Udi Meiri
>            Assignee: Jason Kuster
>            Priority: Major
>
> This job stages the website on GCS for easier review.
>  I believe this only happens on the 2nd and further runs on a specific PR. A lot of time is spent in deleting existing files (22k) and then copying over newer ones.
> Example log: [https://builds.apache.org/job/beam_PreCommit_Website_Stage/1276/consoleText]
> A large chunk of the files are pydoc and javadoc files for version we don't care about.
>  [https://github.com/apache/beam-site/tree/asf-site/content/documentation/sdks/javadoc]
>  [https://github.com/apache/beam-site/tree/asf-site/content/documentation/sdks/pydoc]
> Solution is to:
>  1. Delete unsupported versions' docs.
>  2. Only copy over to GCS the latest pydoc and javadoc, as the rest is irrelevant for review (for almost all PRs). This can be done by deleting the irrelevant directories before invoking "gsutil cp".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)