[jira] [Commented] (BEAM-3483) Introduce VirtualEnv for performance tests on Jenkins

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16327968#comment-16327968 ] 

ASF GitHub Bot commented on BEAM-3483:
--------------------------------------

chamikaramj closed pull request #4413: [BEAM-3483] use VirtualEnv for PerfKit jobs
URL: https://github.com/apache/beam/pull/4413
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.test-infra/jenkins/common_job_properties.groovy b/.test-infra/jenkins/common_job_properties.groovy
index e30c2cd6070..edcff32cd57 100644
--- a/.test-infra/jenkins/common_job_properties.groovy
+++ b/.test-infra/jenkins/common_job_properties.groovy
@@ -260,14 +260,19 @@ class common_job_properties {
     context.steps {
         // Clean up environment.
         shell('rm -rf PerfKitBenchmarker')
+        shell('rm -rf .env')
+
+        // create new VirtualEnv, inherit already existing packages
+        shell('virtualenv .env --system-site-packages')
+
         // Clone appropriate perfkit branch
         shell('git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git')
         // Install Perfkit benchmark requirements.
-        shell('pip install --user -r PerfKitBenchmarker/requirements.txt')
+        shell('.env/bin/pip install -r PerfKitBenchmarker/requirements.txt')
         // Install job requirements for Python SDK.
-        shell('pip install --user -e ' + common_job_properties.checkoutDir + '/sdks/python/[gcp,test]')
+        shell('.env/bin/pip install -e ' + common_job_properties.checkoutDir + '/sdks/python/[gcp,test]')
         // Launch performance test.
-        shell("python PerfKitBenchmarker/pkb.py $pkbArgs")
+        shell(".env/bin/python PerfKitBenchmarker/pkb.py $pkbArgs")
     }
   }
 


 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Introduce VirtualEnv for performance tests on Jenkins
> -----------------------------------------------------
>
>                 Key: BEAM-3483
>                 URL: https://issues.apache.org/jira/browse/BEAM-3483
>             Project: Beam
>          Issue Type: Improvement
>          Components: testing
>            Reporter: Dariusz Aniszewski
>            Assignee: Jason Kuster
>            Priority: Major
>
> Currently all performance tests are using python environment belonging to {{jenkins}} user. This started to be a problem when more that one test run at the same time on the same jenkins worker. In case of different packages, both jobs are altering the same environment and conflicting with each other.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)